OvercomingMethodologicalChallengesassets.press.princeton.edu/chapters/s9209.pdfmodels as well....
Transcript of OvercomingMethodologicalChallengesassets.press.princeton.edu/chapters/s9209.pdfmodels as well....
Copyrighted Material
C H A P T E R O N E
OvercomingMethodologicalChallenges
Questions about the relativemeritsofalternative research strategiespervadethesocialsciencesWhatcountsasanadequateexplanationforsocialphenomenaHowcanweevaluatecompetingexplanationsWhatstandards should we apply when weighing evidence How much andwhat typesof evidenceare convincingCan socialphenomena relatedtopolicyareasbestudiedscientificallySomeeminentscholarsappeartoagreeonbroadmethodological goalsor criteria (BradyandCollier2004Gerring2001Lieberman2005)Explanationsshouldbegeneralyetpreciseaccurateandwell-specifiedEvidenceshouldbetheoreticallyrelevant and should identify mechanisms linking explanations to out-comesAbundantevidenceiftheoreticallyrelevantisvaluedbecauseitenhancesconfidenceinfindings
Despite theapparentcommongroundunderlyingtheworkofmanyscholarsmethodologicaldivideswithinthesocialsciencesalsorundeepAslamentedbyMahoneyandGoertz(2006)andEOstrom(2006)rivalcampsoftencastaspersionsoneachotherrsquosworkratherthanengageinconstructivedialogueTheacrimonyhasseveralsourcesThedisagree-mentshavebeenprovokedinpartbybattlesoverinductionversusde-ductionpoormethodologicalpracticebysomescholarsandalackofsensitivity to diverse research goals The stakes of the methodologicaldebateareincreasedbytheintertwiningofmethodologicalchoicewithontologicalnormativeandtheoreticalpositionsandwithcompetitionforprofessionalstatusandresources(MosesandKnutsen2007)Thesedynamicsencourageintenseandsometimesgrosslyunfaircritiques
Thesubstantivefocusofthisbookisoncollectiveactionandthecom-monsItisafieldofresearchthatutilizesmultiplemethodsextensivelyaswellasbeingtheonemostfamiliartotheauthorsofthisbookWebelievethatthediscussionoftheuseofmultiplemethodsinthisresearchfieldandthelessonswedrawfromourpracticalexperiencesapplymorebroadlytosocialscienceingeneralThereforewestartthisfirstchapterwithabroaderdiscussiononthemethodologicalchallengesinthesocialsciences
Examplesofpoormethodologicalpracticepervadesocialsciencere-search Often scholars follow ldquothe rule of the hammerrdquo and apply asingle method indiscriminately regardless of its suitability for a given
Copyrighted Material
4 bull Chapter1
research project Harmonization of research goals theory data andmethoddoesnothoweverguaranteesoundpracticeOnecanfindqual-itative studies thatoverstate either theuniquenessor the generalityofparticularcasesfailtoutilizerelevantconceptsandtheories inthelit-eratureorworkwithconceptsthatconflatemultipledimensions(Sartori1991 compareGoldthorpe1997)Quantitative studies sometimesuseinadequate data and do not always use appropriate diagnostic checksandtechnicalfixes(Jackman1985Scruggs2007Shalev2007)Formalmodelsoftenworkwithunrealisticassumptionswithoutaddressingthegapbetweenassumptionsandreality(Bendor1988GreenandShapiro1994)Nomethodisimmunetopoorapplications
Critics sometimes conflatemethodological practicewith themethoditselfarguing thatexamplesofpoorapplicationdiscredit themethodAmethodneednotbeabandonedbecauseithasbeenpoorlyutilizeditmakesmoresense toencouragegreatermethodologicalawarenessandbetterpractices(Geddes2003Jackman1985KingKeohaneandVerba1994 Scruggs2007)Others fail to appreciate that researchgoals arevariedandrequirediversemethodsMorethanthreedecadesagoRobertClark(197710emphasisinoriginal)stronglywarnedagainstrelianceonasinglemethod
Afirstruleshouldbetobewareofoneresearcheronemethodor oneinstrumentThepointisnottoprovethatthehypothesisiscorrectbuttofind outsomethingTorelyonasingleapproachistobeshackled
Indiscriminateapplicationofamethodmakes little sensebut com-plete rejectionof amethodbecause it is inappropriate in aparticularsettingorforaparticularpurposeisnotmoresensibleItisimportantfor social scientists to recognize thatallmethodsgenerate results thatcontain some level of uncertainty While multiple scientific goals andtrade-offsinachievingthosegoalsarewidelyacknowledged(Coppedge1999Gerring2001)littleconsensusexistsontherelativeimportanceofparticulargoalsSomescholarsprioritizeoneorafewgoalstosuchanextentthattheydismissasunscientificresearchthatprioritizesothergoalsForexampleGoldthorpe(1997)includesgeneralityasthemostimportantcriterioninhisdefinitionofcausalexplanationratherthanasoneofseveralcriteria(compareGerring2001)ConsequentlyheseesuniqueeventsandcontingencyasmarkingthelimitsofscientificinquiryBythisdefinitionanalysesofsucheventsarenotscientificandcannotsupportcausalinferencesProponentsofpath-dependentexplanationsanalyticnarrativesinterpretivemethodsandotherapproachesstronglydisagree(Batesetal1998BennettandElman2006Rogowski2004R Smith 2004) As in this example and as discussed further below
Copyrighted Material
OvercomingChallenges bull 5
methodologicalcontroversiesoftenreflectcompetitionbetweenresearchtraditions
Fortunately social scientists increasingly recognize trade-offs acrossmethods (Bates 2007 Brady and Collier 2004 Gerring 2001)1 KingKeohaneandVerba(1994)forexamplepointoutthatallmethodolo-gieshavelimitationsscholarsshouldbemoreawareoftheselimitsandmoretransparentaboutthe limitsaswellas thesolidcontributionsoftheir work To overcome the limits of any one method one needs todrawonmultiplemethods(Batesetal1998Coppedge1999GranatoandScioli2004Jackman1985KingKeohaneandVerba1994Laitin2003Lieberman2005Scharpf2000Tarrow2004)Ifsocialscientistshave shared standardsno singlemethod fullyaddressesall standardsMethods offer different strengths and weaknesses Rigorous researchthatcombinescomplementarymethodswillbesuperiortoresearchthatreliesonanysinglemethod(Grayetal2007)
Thepragmatismandrespectfordiversemethodologicaltraditionsinthese reflections are welcome Too often however the challenges in-volvedinusingmultiplemethodsarethemselvesoverlookedProponentsof mixed methods justify their preferred combination in logical termsandillustratetheapproachwithafewexamplesWithsomeexceptions(Lieberman2005Scharpf2000)thisliteratureoffersfewspecificpracti-calsuggestions
Practical challenges canbe formidableNotallmethodsare equallyfeasibleorevenappropriateforallresearchtopics(BennettandElman2006PoteeteandOstrom2008)Liebermanrsquos (2005)nestedanalysisfor example involves large-N analysis prior to any case study workTherearemany important topics forwhichbroadlycomparativedataare scarce difficult to access or of dubiousqualityLieberman how-everdoesnotaddressthesechallengesEvenifdataavailabilityisnotaproblemthevalueofamultimethodapproachrequiressufficientcom-mandofmultiplemethodsYet considerable investment is required togaincompetencyinanymethodologyandthebenefitsofmethodologi-calspecializationaresubstantialWhilethesechallengesaresometimesacknowledgedfewsocialscientistsmakepracticalsuggestionstoaddressthem
Thisbookfocusesonthepracticalchallengesthatinfluencemethod-ologicalchoiceWeareparticularlyconcernedwithresearchontopicsforwhichdataarescarcedifficulttocollectandnotreadilycomparableTheseconditionsaffect researchonawidevarietyof topics includingthose concerned with informal institutions subnational organizationsand nonelite populations We focus on collective action for the man-agementofnaturalresourcesanareaofresearchinwhichallofthese
Copyrighted Material
6 bull Chapter1
conditionsapplyForsuchtopicsdataforlarge-Nanalysisareneitheravailable nor readily accessible andfield research is unavoidableRe-searchersoftenneedconsiderablecontextualknowledgeeventorecog-nizethephenomenonofinterestTheneedtoconductintensivefieldworklimitsthepotentialforcollectingenoughdatatosupportbroadlycom-parativeanalysis
WehavebecomestronglyawareofthesechallengesthroughourownworkoncollectiveactionandnaturalresourcemanagementWefeelthatthe practical challenges of conducting rigorous social science researchontopicsforwhichdataarescarceordifficulttoaccessortointerprethavenotreceivedadequateattentionindiscussionsaboutsocialscienceresearchWehaveseenthebenefitsofcollaborationandthecombinationofmultiplemethodsinourresearchWealsohavefirsthandexperienceof the challenges involved in such research and we will discuss thesethroughoutthisbook
Inthischapterweintroducefourthemesthatrecurthroughthebook(1)theinterlinkingofmethodologicaldebateswiththeoreticaldevelop-ment (2) theadvantagesand limitationsofmultiplemethodsandcol-laborativeresearch(3)practicalconstraintsonmethodologicalchoicesand(4)theoftenproblematicinfluenceofcareerincentivesonmethod-ologicalpracticeInthisbookweexplicitlyacknowledgethepracticalchallengesthataffectmethodologicalchoicesevaluateseveralstrategiesforaddressingthesechallengesanddirectattentiontotheinfluenceofcareer incentives on methodological choices in social science researchWediscussarangeofoptionsforbalancingcompetingmethodologicaldemandsundertheinevitableconditionsoflimitedresourcesincludingavarietyoftechniquesthatwefeelhavebeenunderutilizedinthesocialsciencesWediscussthemeritsandlimitsofeachmethodaswellasthepossibilities forandconstraintsoncombiningvariousmethods Inourdiscussionofconstraintsonmethodologicalchoicewehopetostimulatea debate about professional incentives and other structural aspects ofacademiathatinfluencehowresearchisconducted
Thisbookismoreaboutmethodologicalpracticethanaboutmethod-ologicalidealsWethusbeginthischapterwithahistoricaloverviewofmethodologicaldebateshighlightinginteractionsamongmethodologicalpracticeschangingtheoreticalorientationsandcompetitionforprofes-sionalstatusandresourcesWethenlookmorecloselyatissuessurround-ingresearchthatusesmultiplemethodsanapproachthathasgainedinacceptanceinrecentyearsThisleadstoadiscussionofconstraintsonmethodologicalchoicebothpracticalandprofessionalWethenexplainhowoursubstantivefocusmdashthestudyofcollectiveactioninnaturalre-sourcemanagementmdashhelpsusaddressourfourthematicconcernsThechapterconcludeswithanoutlineoftherestofthebook
Copyrighted Material
OvercomingChallenges bull 7
Social Science Debates over the Superiority of Particular Methods
Thehistoryofthesocialsciencescanberecountedwithreferencetoma-jormethodologicalshiftsAninitialrelianceonqualitativeanalysisgavewaydramaticallytoquantificationintheearlytomid-twentiethcenturyWhenthistransformationbeganquantificationlargelymeantstatisticalanalysisoflarge-NdatasetsofpublicopinionsurveysThelastthirdofthetwentiethcenturysawasurgeintheuseofformalmodelsaswellDebatesabout the relativemeritsofqualitative statisticaland formalmethodscontributedtoseveraldevelopments inthe late twentiethandearlytwenty-firstcenturiesrefinementsofquantitativemethodsthatat-tempttobettermatchsocialconditionstheriseofformalmodelsgreaterappreciation for combiningmultiplemethods and the spreadofpost-positivistmethodssuchasdiscourseanalysis
Thequalitativeorientationoftheearlysocialsciencescanbeseenintheemphasisoncase studiesandparticipantobservation in sociologyethnographicfield-basedresearchinanthropologyanddescriptiveandnormativeanalysesofformallegalarrangementsIntheearlydecadesofthetwentiethcenturymanyscholarsembracedquantitativemethodsaspartofadrivetomakethesocialsciencesmorescientific2 Quantitativemethodsbegantogaincurrencyacrossthesocialsciencesinthe1920sand1930sTheadoptionofthesemethodsacceleratedatmidcenturyasconveyedbyreferencestothebehavioralrevolution
ThebranchesofthesocialsciencesdifferedintheirtimingpaceandpreferredformsofquantificationNonethelessthemethodologicalshiftfromqualitativetoquantitativemethodsinthesocialscienceswasdra-maticPsychologyrapidlyadoptedexperimentalandstatisticalmethodsQuantitativemethodsineconomicsencompassedformalmodelsaswellas experiments and statistics For sociology research activities duringWorldWarIImarkedtheascendanceofsurveyresearchexperimentsand statistical formsofanalysis (Platt1986)Postwarpolitical scienceshared the enthusiasm for survey research and statistical analysis butformalmodelingbecamewidespreadonly in the1980s and1990s Insocioculturalanthropologysomeinterestwasexpressedinmathematicalmodelsintheearlypostwarperiodbutmultivariatestatisticalanalysesremainedrelativelyrareuntilthe1970s(Chibnik1985)
Theroleofquantitativemethodsinthesocialscienceshasalwaysbeencontentious Current methodological debates echo those of a centuryago even if framed in somewhat different terms3 Scholars concernedwithmethodshavedisagreedover (1) thegoalsof social research (2)philosophical and theoretical issues and (3) practical considerationsespecially related to data quality Methodological choices should be
Copyrighted Material
8 bull Chapter1
drivenbytheoreticalandontologicalassumptions(Hall2003)buttheyalso reflectunderlying values andbeliefs (MahoneyandGoertz2006)andpractical considerations (Platt1986)Theontologicalandnorma-tive dimensions of methodological choices are not widely recognized(MahoneyandGoertz2006)Asa result social sciencedebatesaboutmethodsinvolvefrequentmisunderstandingswithproponentsofdiffer-entapproachestalkingpasteachother(EOstrom2006)Furthermorebecause methodological discussions rarely acknowledge practical andprofessionalconsiderationstheyofferlittleguidanceonhowtoaddresstheseconstraintsInthissectionwediscusscontroversiesoverthegoalsofsocialresearchandhowphilosophicalandtheoreticalissuesinteractwithprofessionalcompetitionWeexpandourtreatmentofpracticalandprofessionalconsiderationsinsubsequentsections
During the1920sand1930s the social sciencesbecamemore insti-tutionalized in North America The social sciences sought recognitionassciencesandeachdisciplinedevelopedamoreorlessdistinctprofes-sionalidentity(Guy2003Platt1986)ThisprocessofinstitutionalizationinfluencedmethodologicaldebatesDuringtheprewarperioddisagree-ments focused on the goals of social research Should sociological re-searchsupportsocialworktoimprovesocialconditionsseeksubjectiveunderstandingoflifeexperiencesorattempttoidentifygeneralpatterns(Platt1986)Shouldthestudyofpoliticsprovidenormativeandpracti-cal guidance for administratorsorobjectiveunderstandingofpoliticalphenomena(Guy2003Lasswell1951)Asuniversitiessetupschoolsofsocialworkpublicadministrationandbusinessadministrationalong-sidedepartmentsof sociologypolitical scienceandeconomicsdiffer-encesovergoalswerealleviatedmdashbutnotreallyaddressedmdashthroughtheinstitutionalizationofmorefocusedprogramsofstudy
Yetdifferencesovertherelativeimportanceoftheoryandpraxiscan-not fully account for methodological debates Scholars with commongoalsdisagreeovermethodsandscholarsdrawonthesamemethodstopursuedivergentgoalsAlackofconsensusonfundamentalphilosophi-cal issues contributes to disagreements over methods What counts asscienceWhatmodelormodelsofcausalityandexplanationmakesenseforsocialphenomenaInparticulardomodelsofscienceandexplana-tiondevelopedinthenaturalandespeciallythephysicalsciencesmakesenseforthesocialsciences
Overthepastcenturysomehaveembraceddeductivemodelsofscienceinspiredbythenaturalsciencesasawaytogainmorereliableinsightsaboutsocialprocesses(KingKeohaneandVerba1994PrzeworskiandTeune 1970) Deduction involves the logical derivation of universalis-ticlawlikestatementsofthesetsofconditionsassociatedwiththeout-comeofinterestfromtheoreticalassumptionsLawlikestatementsmay
Copyrighted Material
OvercomingChallenges bull 9
be derived from formal or mathematical models as in rational-choiceapproachesorlogicalanalysisasinsomequalitativestudiesEmpiricalevaluationsrelyontheanalysisofcorrelationasinbehavioralresearchor paired comparisons4 The journal Public Choice devoted a specialissue in December 2008 to the topic ldquoHomo Economicus and HomoPoliticusrdquo(editedbyGeoffreyBrennanandMichaelGillespie)withninearticlesaddressingthequestionofhowtoreconcilethebasicdifferencesbetweentheoriesofhumanbehaviorineconomicsandpoliticalscienceIntheintroductionBrennan(2008431)reflectsthat
theambitiontofindcommongroundonwhichpublicchoicescholarsandldquopoliticaltheoristsrdquoofamoretraditionalkindmighthaveprofit-ableexchangeisnotatrivialonewestartfromverydifferentconcep-tionsofwhatcountsas theorymdashevenofwhatcountsasworthwhilescholarshipmdashandfromratherdifferentdisciplinarypresuppositionsastohowdifferences inapproachcanmostprofitablybeengagedandresolved
Criticshoweverarguethatdeductivemethodsdonotallowforhu-managencyandreflexivitytheinfluenceofmeaningandinterpretationorcontingentrelationships(AlmondandGenco1977Hall2003Ragin1987seereviewinPlatt1986)Ifagencyistakenseriouslywemustal-lowforbothcreativityanddifferencesinperspectivesButcreativityanddifferencesininterpretationmeanthatlawlikesocialpatternsareunlikelytoariseContingentrelationshipsarepossibleevenifquestionsofagencyareputasideThesedifferencesoverthenatureofcausalityhavefueledheatedmethodologicaldebatesInpoliticalscienceboththebehavioralrevolutionoftheearlypostwarperiodandtheriseofrational-choicethe-oryinthe1980sand1990sassumedthevalueofdeductive-nomologicalreasoningScholarswhousedmethodsthatreflectedalternativeontologi-calassumptionshaddifficultygainingrecognitionfortheirworkTheirfrustrationgaverisetotherecentperestroikamovementinwhichcon-structivistsandotherschallengedboththeuniversalityofsocialpatternsassumedbyrationalchoiceandbehavioraltheoriesandthedominanceofstatisticalandformalmethodsassociatedwiththeseapproachesintheprofession(Monroe2005)Withineconomicstheconcernthatnarrowrational-choicemodelshavecometodominatemuchofeconomicschol-arshipisregularlyexpressedintheonlinejournalReal-World Economics Review5
Deductive-nomological reasoning suggests a mechanical view of theworldinwhichthesamestimulusproducesthesameeffectceterispa-ribusTheoriesthatviewsocialphenomenaasproductsofeitherevolu-tionaryprocessesor intentionalactionchallenge thismechanicalviewBothevolutionaryandintentionaltheoriesassumethatindividualsand
Copyrighted Material
10 bull Chapter1
organizationsadjusttheirresponsestosocialconditions(Alchian1950Brady2004EOstrom2000Thelen2003)Intentionaltheoriesofhu-manbehaviorassumethatadaptationoccursaspeoplestruggletosolvepuzzlesrelatedtothepursuitoftheirgoals(AlmondandGenco1977Elster1983Knight1992)While some intentional theories emphasizeroutines and heuristics there is always a possibility for creativity andinnovation (March andOlsen1984 Simon1955)Evolutionary theo-riesdonotrequire intentionalitybutdorequiresomesortofselectionmechanismsuchasmarketorelectoralcompetitiontodriveadaptationBothformsofadaptationimplythatthesamecircumstanceswillgener-atediverse responsesacrossactorsandchanges in individualbehaviorover timebut thatadaptationswill reflecthistorical trajectoriesThusthesamestimuluswillnot producethesameeffectonaverageandcon-stanteffectscannotbeassumedBothperspectivesraisequestionsaboutthesuitabilityof researchmethods thatassumeconstanteffects (Elster1998Hall2003Ragin19872000)
ThechoiceofmethodtendstosignalonersquostheoreticalperspectiveasdoesthenatureofmethodologicalcritiqueThosewhodiscountquali-tativemethodsasincapableofevaluatinggeneralrelationshipssignalabelief inbothlawlikesocialrelationsandtherelativeunimportanceoffactorssuchasagencyhistoryandinformalcontextNotsurprisinglycritiquesofquantitativemethodsoftenchargethattheydonotcapturethemost important aspects of social conditionsLikewise thosewaryofformalmodelsworryaboutthelevelofabstractionHowcanformalmodelsadequatelyrepresentthedensenetworksofformalandinformalinstitutionsandculturalunderstandingsinwhichhumanactionoccursNoneof these critiques really concerns themethodasmethod ratherthey target the theoretical assumptions as reflected in methodologicalchoicesWhatvariablesareimportantWhatistherelativeimportanceofformalinstitutionsculturesocialstructureorinformalinstitutionsHow important aremass beliefs andbehavior or individual interestsbeliefsandstrategicactionHowarethosevariablesrelatedWhilethebehavioralrevolutionduringthemid-twentiethcenturycertainlyfosteredtherapidspreadofquantitativeanalysisitalsoredirectedtheoreticalem-phasisfromformalinstitutionstothebehaviorandattitudesofindividu-als interactingwithinboth formal and informal institutionsLikewiserational-choiceanalysisoftenreliesongametheoryandothervarietiesof formal modeling but is defined by assumptions of methodologicalindividualismandintentionalaction
Yettheinfluenceoftheorymdashandtheimpliedinfluenceofontologymdashonmethodologicalpracticecannotbeassumedandshouldnotbeover-stated6 Theoreticalchangescananddooccurindependentlyofchangesinmethodologicalpractice(Hall2003Platt1986)Sometimesmethod-
Copyrighted Material
OvercomingChallenges bull 11
ologicalchallengesseemtodrivetheoreticalargumentsratherthantheotherwayaround(Lieberson1991318)Indeedsophisticatedmethodssometimescrowdouttheoryaltogether(Achen20022005)WearguethatmethodologicalchoicesareoftendrivenasmuchbydataavailabilityorcareerincentivesWhencareersurvivalisatstakepracticalconsider-ationscansqueezeoutconcernsaboutmatchingtheoryandmethodThelinkbetweenmethodsandcareerprospectscanhoweverbeexpectedtoinfluencethetenorofmethodologicalmdashandtheoreticalmdashdebates
SometimesmethodologicalandtheoreticaldebatestakeonexistentialovertonesWhenaparticulartheoryandassociatedmethodsbecomeex-tremelywidespreadforexampleproponentsofalternativeapproachesmayworryabouttheirownacademicsurvivalProponentsofnewtheo-riesmdashandassociatedmethodsmdashalsofaceanexistentialfightforrecogni-tionandsurvivalThedegreeof(perceived)existentialthreatdependsonthe extent to which fellowships job opportunities publishing outletsandresearchgrantsareopen(orclosed)todiversetheoriesandmethodsTherecentperestroikamovementinpoliticalscienceforexamplepre-sented itself asdefendingagainstmethodologicalhegemony conjuringimages of political scientists conspiring to control journal outlets andprofessionalassociations7ThiswasnotsimplyamethodologicalcritiquebutacalltoactionagainstpresumedtyrannyWhiletheinflammatorypublic languageassociatedwiththeperestroikamovementmaybeun-usualthelayeringofmethodologicaldebateswithvaluejudgmentsandcompetitionforprofessionalrecognitionandresourcesiscommonplace
Despite references to ldquorevolutionsrdquo and paradigm shifts new socialsciencetheoriesandmethodshavenotfullydisplacedtheirpredecessorsRathereachnewtheoryandmethodhasaddedanotherstrandConstruc-tivists institutionalists andpostmodernists coexistwithbehavioralistsandstructuralistsDespitethehistoryoftheoreticalandmethodologicalcompetitionandcritiquescholarsalsoengageincreativesynthesisThecurrentappreciationformethodologicalpluralismmaybeinterpretedasaproductofthesurvivalandadaptationofapproachesthatwereonceperceived to be under existential threat Promotion of methodologicalpluralismfavorsatheoreticaleclecticismthatshoulddecreaseconcernsaboutexistential threats toparticularapproachesandshouldthusde-creasetheintensityofmethodologicaldebates
Multiple Methods Promises and Challenges
Therearemanyreasonsforsocialscientiststowelcomemethodologicalpluralism and greater use of mixed methods No single method over-comes all challenges Case studies and small-N comparative research
Copyrighted Material
12 bull Chapter1
designs offer advantages for concept and theory development as wellasevaluationofhypothesizedcausalsequencesandmechanisms(Bates2008Batesetal1998CollierBradyandSeawright2004Coppedge1999 Lieberman 2005) Rich explanations of particular cases are of-tenvaluablesubstantivelyandtheoretically(MahoneyandGoertz2006Rogowski2004)Yetasiswidelyrecognizedsmall-Nstudiesofferanuncertainfoundationforpositingorevaluatinggeneralrelationships
Formalmethodsseektobuild logicallycoherentmodelsanddiscerntheir logical implicationsTheir emphasison logical consistency facili-tatesthedistillationofparsimoniousyetgeneralhypothesesandguidesthechoiceofstatisticaltechniques(Achen20022005Batesetal1998GranatoandScioli2004)8 Thehighlevelofabstractioninformalmod-elshoweverraisesquestionsabouttheirempiricalapplicability(GreenandShapiro1994)Thecontrolledconditionsinexperimentalresearchprovidegreaterconfidenceintheinternalvalidityofobservedrelation-shipsTheexternalvalidityofgeneralrelationshipscanbestbeevaluatedhoweverthroughanalysisofalargenumberofnonexperimentalobser-vations(Goldthorpe1997KingKeohaneandVerba1994)aswellasthroughfieldexperiments(seeCardenas2003CardenasStranlundandWillis2000Henrichetal2004List2004)
Small-Nqualitativestudiescansuggesttheplausibilityofformalmod-elsbutprovidelittleleverageinassessingthegeneralityofrelationshipsThebroad comparisons required to evaluate thegeneralityofhypoth-esizedrelationshipsdemandsomeformofquantitativeanalysisWherequantitativeanalysisoncemeantregression-basedanalysisoptions forquantitativeanalysisofempiricalsocialsciencedatanowincludeQuali-tative Comparative Analysis (QCA) and fuzzy-set Qualitative Com-parativeAnalysis(fsQCA)(Ragin19872000)aswellasprobabilisticlikelihood-basedandBayesianstatistics(Gill2004)9 Thismethodologi-calmenuincludesoptionsforscholarswhoholdvariedontologicalas-sumptionsaboutthesocialworld
MixedmethodstakeavarietyofformsAresearchermightusedifferentmethodstoaddressdifferentresearchquestionsorcontextsOrdifferentmethodsmightguidedifferentstagesofaresearchprogram(Lieberman2005)IncreasinglyscholarsstrivetousetwoormoremethodsateachstageofresearchThoseconcernedwithgeneralcausalpatternsdrawonquantitativeandqualitativemethods(Coppedge1999Lieberman2005Tarrow2004)Combinationsofformalandqualitativemethodsaddressconcernsaboutlogicalcoherenceandcausalprocessesincontingentrela-tionshipswherethereisnoexpectationofgenerality(Batesetal1998)Otherscontendthatscholarsshouldseeklogicalcoherenceandevidencefor causalprocesses and should test for thegeneralityof relationsbydrawingonformalqualitativeandquantitativemethods(Granatoand
Copyrighted Material
OvercomingChallenges bull 13
Scioli2004Laitin2003)Scholarswhodevelopagent-basedmodelsuserolegamesandexperiments tocollectdataaswellas involving stake-holdersinthevalidationoftheirmodels(BarreteauLePageandAquino2003Bousquetetal2002GurungBousquetandTreacutebuil2006)Otherscholars combine their formal models with ethnographic observations(Bharwanietal2005HuigenOvermarsanddeGroot2006)
Theuseofmultiplemethodshoweverdoesnotguaranteemethod-ologicallysuperiorsocialscienceresearchSomequestiontheextenttowhichformalqualitativeandquantitativeresearchmethodsareactuallycomplementary Several recent publications have argued that differentmethodsreflectdifferentassumptionsaboutthenatureofcausalityandhavecalledforgreatercareinmatchingmethodstoontologicalassump-tions(BennettandElman2006ClarkGilliganandGolder2006Hall2003Mahoney2003Ragin19872000)
Therearealso limits to the feasibilityofmultimethodresearchHy-potheses about complex causal relationships imply complex statisticalmodelsthatstretchthelimitsofavailabledataStatisticalanalysesoftenadd interaction termsordummyvariables tomodel contingent effectsandmultiplecausalpaths(ClarkGilliganandGolder2006Pontusson2007)but theseadditionalvariablesconsumedegreesof freedominacontextoflimiteddataavailability(Shalev2007)Othertechniquesde-velopedtoaddresscausalcomplexitysuchastheanalysisoftime-series-cross-sectional data and hierarchical models may strain the technicalskillsofboththeresearcherandtheaudience(Shalev2007)
MahoneyandGoertz(2006)contendthatinteractioneffectsdummyvariableshierarchicalmodelsandothersimilarstatisticalfixesdonotaccuratelyreflecttherelationshipspositedintheunderlyingtheoriesTheassumptionthatobservationsareindependentforexampleiscalledintoquestionbyglobalizationdiffusioneffectsandactor-centeredtheoriesthatemphasizestrategicinteractionsEvensomequantitativelyorientedscholarsquestiontheappropriatenessofstandardstatisticaltechniquesInrecentyearsnewtechniqueshavebeenproposedtoincorporateinter-dependence(Signorino1999)Bayesianstatistics(Dion1998Gill2004)andBooleanlogic(Braumoeller2003Ragin19872000)Theverdictisstilloutonwhetherthesenewtechniquesmatchunderlyingassumptionsbetterthandoesregression-basedstatistics
Too often the development of ever-more sophisticated techniquesseemstobeanendinitselfThelatesttechniquesaresometimesadoptedwithlittlereferencetotheoreticalconsiderationsorunderstandingoftheunderlyingassumptionsButmethodologicalsophisticationcannotsub-stitutefortheoryAchen(20022005)warnsthatquantitativeanalysesthat are not supported by theoretical microfoundations or careful ex-plorationofthedatayieldunreliableresultsandshouldnotbetrusted
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
4 bull Chapter1
research project Harmonization of research goals theory data andmethoddoesnothoweverguaranteesoundpracticeOnecanfindqual-itative studies thatoverstate either theuniquenessor the generalityofparticularcasesfailtoutilizerelevantconceptsandtheories inthelit-eratureorworkwithconceptsthatconflatemultipledimensions(Sartori1991 compareGoldthorpe1997)Quantitative studies sometimesuseinadequate data and do not always use appropriate diagnostic checksandtechnicalfixes(Jackman1985Scruggs2007Shalev2007)Formalmodelsoftenworkwithunrealisticassumptionswithoutaddressingthegapbetweenassumptionsandreality(Bendor1988GreenandShapiro1994)Nomethodisimmunetopoorapplications
Critics sometimes conflatemethodological practicewith themethoditselfarguing thatexamplesofpoorapplicationdiscredit themethodAmethodneednotbeabandonedbecauseithasbeenpoorlyutilizeditmakesmoresense toencouragegreatermethodologicalawarenessandbetterpractices(Geddes2003Jackman1985KingKeohaneandVerba1994 Scruggs2007)Others fail to appreciate that researchgoals arevariedandrequirediversemethodsMorethanthreedecadesagoRobertClark(197710emphasisinoriginal)stronglywarnedagainstrelianceonasinglemethod
Afirstruleshouldbetobewareofoneresearcheronemethodor oneinstrumentThepointisnottoprovethatthehypothesisiscorrectbuttofind outsomethingTorelyonasingleapproachistobeshackled
Indiscriminateapplicationofamethodmakes little sensebut com-plete rejectionof amethodbecause it is inappropriate in aparticularsettingorforaparticularpurposeisnotmoresensibleItisimportantfor social scientists to recognize thatallmethodsgenerate results thatcontain some level of uncertainty While multiple scientific goals andtrade-offsinachievingthosegoalsarewidelyacknowledged(Coppedge1999Gerring2001)littleconsensusexistsontherelativeimportanceofparticulargoalsSomescholarsprioritizeoneorafewgoalstosuchanextentthattheydismissasunscientificresearchthatprioritizesothergoalsForexampleGoldthorpe(1997)includesgeneralityasthemostimportantcriterioninhisdefinitionofcausalexplanationratherthanasoneofseveralcriteria(compareGerring2001)ConsequentlyheseesuniqueeventsandcontingencyasmarkingthelimitsofscientificinquiryBythisdefinitionanalysesofsucheventsarenotscientificandcannotsupportcausalinferencesProponentsofpath-dependentexplanationsanalyticnarrativesinterpretivemethodsandotherapproachesstronglydisagree(Batesetal1998BennettandElman2006Rogowski2004R Smith 2004) As in this example and as discussed further below
Copyrighted Material
OvercomingChallenges bull 5
methodologicalcontroversiesoftenreflectcompetitionbetweenresearchtraditions
Fortunately social scientists increasingly recognize trade-offs acrossmethods (Bates 2007 Brady and Collier 2004 Gerring 2001)1 KingKeohaneandVerba(1994)forexamplepointoutthatallmethodolo-gieshavelimitationsscholarsshouldbemoreawareoftheselimitsandmoretransparentaboutthe limitsaswellas thesolidcontributionsoftheir work To overcome the limits of any one method one needs todrawonmultiplemethods(Batesetal1998Coppedge1999GranatoandScioli2004Jackman1985KingKeohaneandVerba1994Laitin2003Lieberman2005Scharpf2000Tarrow2004)Ifsocialscientistshave shared standardsno singlemethod fullyaddressesall standardsMethods offer different strengths and weaknesses Rigorous researchthatcombinescomplementarymethodswillbesuperiortoresearchthatreliesonanysinglemethod(Grayetal2007)
Thepragmatismandrespectfordiversemethodologicaltraditionsinthese reflections are welcome Too often however the challenges in-volvedinusingmultiplemethodsarethemselvesoverlookedProponentsof mixed methods justify their preferred combination in logical termsandillustratetheapproachwithafewexamplesWithsomeexceptions(Lieberman2005Scharpf2000)thisliteratureoffersfewspecificpracti-calsuggestions
Practical challenges canbe formidableNotallmethodsare equallyfeasibleorevenappropriateforallresearchtopics(BennettandElman2006PoteeteandOstrom2008)Liebermanrsquos (2005)nestedanalysisfor example involves large-N analysis prior to any case study workTherearemany important topics forwhichbroadlycomparativedataare scarce difficult to access or of dubiousqualityLieberman how-everdoesnotaddressthesechallengesEvenifdataavailabilityisnotaproblemthevalueofamultimethodapproachrequiressufficientcom-mandofmultiplemethodsYet considerable investment is required togaincompetencyinanymethodologyandthebenefitsofmethodologi-calspecializationaresubstantialWhilethesechallengesaresometimesacknowledgedfewsocialscientistsmakepracticalsuggestionstoaddressthem
Thisbookfocusesonthepracticalchallengesthatinfluencemethod-ologicalchoiceWeareparticularlyconcernedwithresearchontopicsforwhichdataarescarcedifficulttocollectandnotreadilycomparableTheseconditionsaffect researchonawidevarietyof topics includingthose concerned with informal institutions subnational organizationsand nonelite populations We focus on collective action for the man-agementofnaturalresourcesanareaofresearchinwhichallofthese
Copyrighted Material
6 bull Chapter1
conditionsapplyForsuchtopicsdataforlarge-Nanalysisareneitheravailable nor readily accessible andfield research is unavoidableRe-searchersoftenneedconsiderablecontextualknowledgeeventorecog-nizethephenomenonofinterestTheneedtoconductintensivefieldworklimitsthepotentialforcollectingenoughdatatosupportbroadlycom-parativeanalysis
WehavebecomestronglyawareofthesechallengesthroughourownworkoncollectiveactionandnaturalresourcemanagementWefeelthatthe practical challenges of conducting rigorous social science researchontopicsforwhichdataarescarceordifficulttoaccessortointerprethavenotreceivedadequateattentionindiscussionsaboutsocialscienceresearchWehaveseenthebenefitsofcollaborationandthecombinationofmultiplemethodsinourresearchWealsohavefirsthandexperienceof the challenges involved in such research and we will discuss thesethroughoutthisbook
Inthischapterweintroducefourthemesthatrecurthroughthebook(1)theinterlinkingofmethodologicaldebateswiththeoreticaldevelop-ment (2) theadvantagesand limitationsofmultiplemethodsandcol-laborativeresearch(3)practicalconstraintsonmethodologicalchoicesand(4)theoftenproblematicinfluenceofcareerincentivesonmethod-ologicalpracticeInthisbookweexplicitlyacknowledgethepracticalchallengesthataffectmethodologicalchoicesevaluateseveralstrategiesforaddressingthesechallengesanddirectattentiontotheinfluenceofcareer incentives on methodological choices in social science researchWediscussarangeofoptionsforbalancingcompetingmethodologicaldemandsundertheinevitableconditionsoflimitedresourcesincludingavarietyoftechniquesthatwefeelhavebeenunderutilizedinthesocialsciencesWediscussthemeritsandlimitsofeachmethodaswellasthepossibilities forandconstraintsoncombiningvariousmethods Inourdiscussionofconstraintsonmethodologicalchoicewehopetostimulatea debate about professional incentives and other structural aspects ofacademiathatinfluencehowresearchisconducted
Thisbookismoreaboutmethodologicalpracticethanaboutmethod-ologicalidealsWethusbeginthischapterwithahistoricaloverviewofmethodologicaldebateshighlightinginteractionsamongmethodologicalpracticeschangingtheoreticalorientationsandcompetitionforprofes-sionalstatusandresourcesWethenlookmorecloselyatissuessurround-ingresearchthatusesmultiplemethodsanapproachthathasgainedinacceptanceinrecentyearsThisleadstoadiscussionofconstraintsonmethodologicalchoicebothpracticalandprofessionalWethenexplainhowoursubstantivefocusmdashthestudyofcollectiveactioninnaturalre-sourcemanagementmdashhelpsusaddressourfourthematicconcernsThechapterconcludeswithanoutlineoftherestofthebook
Copyrighted Material
OvercomingChallenges bull 7
Social Science Debates over the Superiority of Particular Methods
Thehistoryofthesocialsciencescanberecountedwithreferencetoma-jormethodologicalshiftsAninitialrelianceonqualitativeanalysisgavewaydramaticallytoquantificationintheearlytomid-twentiethcenturyWhenthistransformationbeganquantificationlargelymeantstatisticalanalysisoflarge-NdatasetsofpublicopinionsurveysThelastthirdofthetwentiethcenturysawasurgeintheuseofformalmodelsaswellDebatesabout the relativemeritsofqualitative statisticaland formalmethodscontributedtoseveraldevelopments inthe late twentiethandearlytwenty-firstcenturiesrefinementsofquantitativemethodsthatat-tempttobettermatchsocialconditionstheriseofformalmodelsgreaterappreciation for combiningmultiplemethods and the spreadofpost-positivistmethodssuchasdiscourseanalysis
Thequalitativeorientationoftheearlysocialsciencescanbeseenintheemphasisoncase studiesandparticipantobservation in sociologyethnographicfield-basedresearchinanthropologyanddescriptiveandnormativeanalysesofformallegalarrangementsIntheearlydecadesofthetwentiethcenturymanyscholarsembracedquantitativemethodsaspartofadrivetomakethesocialsciencesmorescientific2 Quantitativemethodsbegantogaincurrencyacrossthesocialsciencesinthe1920sand1930sTheadoptionofthesemethodsacceleratedatmidcenturyasconveyedbyreferencestothebehavioralrevolution
ThebranchesofthesocialsciencesdifferedintheirtimingpaceandpreferredformsofquantificationNonethelessthemethodologicalshiftfromqualitativetoquantitativemethodsinthesocialscienceswasdra-maticPsychologyrapidlyadoptedexperimentalandstatisticalmethodsQuantitativemethodsineconomicsencompassedformalmodelsaswellas experiments and statistics For sociology research activities duringWorldWarIImarkedtheascendanceofsurveyresearchexperimentsand statistical formsofanalysis (Platt1986)Postwarpolitical scienceshared the enthusiasm for survey research and statistical analysis butformalmodelingbecamewidespreadonly in the1980s and1990s Insocioculturalanthropologysomeinterestwasexpressedinmathematicalmodelsintheearlypostwarperiodbutmultivariatestatisticalanalysesremainedrelativelyrareuntilthe1970s(Chibnik1985)
Theroleofquantitativemethodsinthesocialscienceshasalwaysbeencontentious Current methodological debates echo those of a centuryago even if framed in somewhat different terms3 Scholars concernedwithmethodshavedisagreedover (1) thegoalsof social research (2)philosophical and theoretical issues and (3) practical considerationsespecially related to data quality Methodological choices should be
Copyrighted Material
8 bull Chapter1
drivenbytheoreticalandontologicalassumptions(Hall2003)buttheyalso reflectunderlying values andbeliefs (MahoneyandGoertz2006)andpractical considerations (Platt1986)Theontologicalandnorma-tive dimensions of methodological choices are not widely recognized(MahoneyandGoertz2006)Asa result social sciencedebatesaboutmethodsinvolvefrequentmisunderstandingswithproponentsofdiffer-entapproachestalkingpasteachother(EOstrom2006)Furthermorebecause methodological discussions rarely acknowledge practical andprofessionalconsiderationstheyofferlittleguidanceonhowtoaddresstheseconstraintsInthissectionwediscusscontroversiesoverthegoalsofsocialresearchandhowphilosophicalandtheoreticalissuesinteractwithprofessionalcompetitionWeexpandourtreatmentofpracticalandprofessionalconsiderationsinsubsequentsections
During the1920sand1930s the social sciencesbecamemore insti-tutionalized in North America The social sciences sought recognitionassciencesandeachdisciplinedevelopedamoreorlessdistinctprofes-sionalidentity(Guy2003Platt1986)ThisprocessofinstitutionalizationinfluencedmethodologicaldebatesDuringtheprewarperioddisagree-ments focused on the goals of social research Should sociological re-searchsupportsocialworktoimprovesocialconditionsseeksubjectiveunderstandingoflifeexperiencesorattempttoidentifygeneralpatterns(Platt1986)Shouldthestudyofpoliticsprovidenormativeandpracti-cal guidance for administratorsorobjectiveunderstandingofpoliticalphenomena(Guy2003Lasswell1951)Asuniversitiessetupschoolsofsocialworkpublicadministrationandbusinessadministrationalong-sidedepartmentsof sociologypolitical scienceandeconomicsdiffer-encesovergoalswerealleviatedmdashbutnotreallyaddressedmdashthroughtheinstitutionalizationofmorefocusedprogramsofstudy
Yetdifferencesovertherelativeimportanceoftheoryandpraxiscan-not fully account for methodological debates Scholars with commongoalsdisagreeovermethodsandscholarsdrawonthesamemethodstopursuedivergentgoalsAlackofconsensusonfundamentalphilosophi-cal issues contributes to disagreements over methods What counts asscienceWhatmodelormodelsofcausalityandexplanationmakesenseforsocialphenomenaInparticulardomodelsofscienceandexplana-tiondevelopedinthenaturalandespeciallythephysicalsciencesmakesenseforthesocialsciences
Overthepastcenturysomehaveembraceddeductivemodelsofscienceinspiredbythenaturalsciencesasawaytogainmorereliableinsightsaboutsocialprocesses(KingKeohaneandVerba1994PrzeworskiandTeune 1970) Deduction involves the logical derivation of universalis-ticlawlikestatementsofthesetsofconditionsassociatedwiththeout-comeofinterestfromtheoreticalassumptionsLawlikestatementsmay
Copyrighted Material
OvercomingChallenges bull 9
be derived from formal or mathematical models as in rational-choiceapproachesorlogicalanalysisasinsomequalitativestudiesEmpiricalevaluationsrelyontheanalysisofcorrelationasinbehavioralresearchor paired comparisons4 The journal Public Choice devoted a specialissue in December 2008 to the topic ldquoHomo Economicus and HomoPoliticusrdquo(editedbyGeoffreyBrennanandMichaelGillespie)withninearticlesaddressingthequestionofhowtoreconcilethebasicdifferencesbetweentheoriesofhumanbehaviorineconomicsandpoliticalscienceIntheintroductionBrennan(2008431)reflectsthat
theambitiontofindcommongroundonwhichpublicchoicescholarsandldquopoliticaltheoristsrdquoofamoretraditionalkindmighthaveprofit-ableexchangeisnotatrivialonewestartfromverydifferentconcep-tionsofwhatcountsas theorymdashevenofwhatcountsasworthwhilescholarshipmdashandfromratherdifferentdisciplinarypresuppositionsastohowdifferences inapproachcanmostprofitablybeengagedandresolved
Criticshoweverarguethatdeductivemethodsdonotallowforhu-managencyandreflexivitytheinfluenceofmeaningandinterpretationorcontingentrelationships(AlmondandGenco1977Hall2003Ragin1987seereviewinPlatt1986)Ifagencyistakenseriouslywemustal-lowforbothcreativityanddifferencesinperspectivesButcreativityanddifferencesininterpretationmeanthatlawlikesocialpatternsareunlikelytoariseContingentrelationshipsarepossibleevenifquestionsofagencyareputasideThesedifferencesoverthenatureofcausalityhavefueledheatedmethodologicaldebatesInpoliticalscienceboththebehavioralrevolutionoftheearlypostwarperiodandtheriseofrational-choicethe-oryinthe1980sand1990sassumedthevalueofdeductive-nomologicalreasoningScholarswhousedmethodsthatreflectedalternativeontologi-calassumptionshaddifficultygainingrecognitionfortheirworkTheirfrustrationgaverisetotherecentperestroikamovementinwhichcon-structivistsandotherschallengedboththeuniversalityofsocialpatternsassumedbyrationalchoiceandbehavioraltheoriesandthedominanceofstatisticalandformalmethodsassociatedwiththeseapproachesintheprofession(Monroe2005)Withineconomicstheconcernthatnarrowrational-choicemodelshavecometodominatemuchofeconomicschol-arshipisregularlyexpressedintheonlinejournalReal-World Economics Review5
Deductive-nomological reasoning suggests a mechanical view of theworldinwhichthesamestimulusproducesthesameeffectceterispa-ribusTheoriesthatviewsocialphenomenaasproductsofeitherevolu-tionaryprocessesor intentionalactionchallenge thismechanicalviewBothevolutionaryandintentionaltheoriesassumethatindividualsand
Copyrighted Material
10 bull Chapter1
organizationsadjusttheirresponsestosocialconditions(Alchian1950Brady2004EOstrom2000Thelen2003)Intentionaltheoriesofhu-manbehaviorassumethatadaptationoccursaspeoplestruggletosolvepuzzlesrelatedtothepursuitoftheirgoals(AlmondandGenco1977Elster1983Knight1992)While some intentional theories emphasizeroutines and heuristics there is always a possibility for creativity andinnovation (March andOlsen1984 Simon1955)Evolutionary theo-riesdonotrequire intentionalitybutdorequiresomesortofselectionmechanismsuchasmarketorelectoralcompetitiontodriveadaptationBothformsofadaptationimplythatthesamecircumstanceswillgener-atediverse responsesacrossactorsandchanges in individualbehaviorover timebut thatadaptationswill reflecthistorical trajectoriesThusthesamestimuluswillnot producethesameeffectonaverageandcon-stanteffectscannotbeassumedBothperspectivesraisequestionsaboutthesuitabilityof researchmethods thatassumeconstanteffects (Elster1998Hall2003Ragin19872000)
ThechoiceofmethodtendstosignalonersquostheoreticalperspectiveasdoesthenatureofmethodologicalcritiqueThosewhodiscountquali-tativemethodsasincapableofevaluatinggeneralrelationshipssignalabelief inbothlawlikesocialrelationsandtherelativeunimportanceoffactorssuchasagencyhistoryandinformalcontextNotsurprisinglycritiquesofquantitativemethodsoftenchargethattheydonotcapturethemost important aspects of social conditionsLikewise thosewaryofformalmodelsworryaboutthelevelofabstractionHowcanformalmodelsadequatelyrepresentthedensenetworksofformalandinformalinstitutionsandculturalunderstandingsinwhichhumanactionoccursNoneof these critiques really concerns themethodasmethod ratherthey target the theoretical assumptions as reflected in methodologicalchoicesWhatvariablesareimportantWhatistherelativeimportanceofformalinstitutionsculturesocialstructureorinformalinstitutionsHow important aremass beliefs andbehavior or individual interestsbeliefsandstrategicactionHowarethosevariablesrelatedWhilethebehavioralrevolutionduringthemid-twentiethcenturycertainlyfosteredtherapidspreadofquantitativeanalysisitalsoredirectedtheoreticalem-phasisfromformalinstitutionstothebehaviorandattitudesofindividu-als interactingwithinboth formal and informal institutionsLikewiserational-choiceanalysisoftenreliesongametheoryandothervarietiesof formal modeling but is defined by assumptions of methodologicalindividualismandintentionalaction
Yettheinfluenceoftheorymdashandtheimpliedinfluenceofontologymdashonmethodologicalpracticecannotbeassumedandshouldnotbeover-stated6 Theoreticalchangescananddooccurindependentlyofchangesinmethodologicalpractice(Hall2003Platt1986)Sometimesmethod-
Copyrighted Material
OvercomingChallenges bull 11
ologicalchallengesseemtodrivetheoreticalargumentsratherthantheotherwayaround(Lieberson1991318)Indeedsophisticatedmethodssometimescrowdouttheoryaltogether(Achen20022005)WearguethatmethodologicalchoicesareoftendrivenasmuchbydataavailabilityorcareerincentivesWhencareersurvivalisatstakepracticalconsider-ationscansqueezeoutconcernsaboutmatchingtheoryandmethodThelinkbetweenmethodsandcareerprospectscanhoweverbeexpectedtoinfluencethetenorofmethodologicalmdashandtheoreticalmdashdebates
SometimesmethodologicalandtheoreticaldebatestakeonexistentialovertonesWhenaparticulartheoryandassociatedmethodsbecomeex-tremelywidespreadforexampleproponentsofalternativeapproachesmayworryabouttheirownacademicsurvivalProponentsofnewtheo-riesmdashandassociatedmethodsmdashalsofaceanexistentialfightforrecogni-tionandsurvivalThedegreeof(perceived)existentialthreatdependsonthe extent to which fellowships job opportunities publishing outletsandresearchgrantsareopen(orclosed)todiversetheoriesandmethodsTherecentperestroikamovementinpoliticalscienceforexamplepre-sented itself asdefendingagainstmethodologicalhegemony conjuringimages of political scientists conspiring to control journal outlets andprofessionalassociations7ThiswasnotsimplyamethodologicalcritiquebutacalltoactionagainstpresumedtyrannyWhiletheinflammatorypublic languageassociatedwiththeperestroikamovementmaybeun-usualthelayeringofmethodologicaldebateswithvaluejudgmentsandcompetitionforprofessionalrecognitionandresourcesiscommonplace
Despite references to ldquorevolutionsrdquo and paradigm shifts new socialsciencetheoriesandmethodshavenotfullydisplacedtheirpredecessorsRathereachnewtheoryandmethodhasaddedanotherstrandConstruc-tivists institutionalists andpostmodernists coexistwithbehavioralistsandstructuralistsDespitethehistoryoftheoreticalandmethodologicalcompetitionandcritiquescholarsalsoengageincreativesynthesisThecurrentappreciationformethodologicalpluralismmaybeinterpretedasaproductofthesurvivalandadaptationofapproachesthatwereonceperceived to be under existential threat Promotion of methodologicalpluralismfavorsatheoreticaleclecticismthatshoulddecreaseconcernsaboutexistential threats toparticularapproachesandshouldthusde-creasetheintensityofmethodologicaldebates
Multiple Methods Promises and Challenges
Therearemanyreasonsforsocialscientiststowelcomemethodologicalpluralism and greater use of mixed methods No single method over-comes all challenges Case studies and small-N comparative research
Copyrighted Material
12 bull Chapter1
designs offer advantages for concept and theory development as wellasevaluationofhypothesizedcausalsequencesandmechanisms(Bates2008Batesetal1998CollierBradyandSeawright2004Coppedge1999 Lieberman 2005) Rich explanations of particular cases are of-tenvaluablesubstantivelyandtheoretically(MahoneyandGoertz2006Rogowski2004)Yetasiswidelyrecognizedsmall-Nstudiesofferanuncertainfoundationforpositingorevaluatinggeneralrelationships
Formalmethodsseektobuild logicallycoherentmodelsanddiscerntheir logical implicationsTheir emphasison logical consistency facili-tatesthedistillationofparsimoniousyetgeneralhypothesesandguidesthechoiceofstatisticaltechniques(Achen20022005Batesetal1998GranatoandScioli2004)8 Thehighlevelofabstractioninformalmod-elshoweverraisesquestionsabouttheirempiricalapplicability(GreenandShapiro1994)Thecontrolledconditionsinexperimentalresearchprovidegreaterconfidenceintheinternalvalidityofobservedrelation-shipsTheexternalvalidityofgeneralrelationshipscanbestbeevaluatedhoweverthroughanalysisofalargenumberofnonexperimentalobser-vations(Goldthorpe1997KingKeohaneandVerba1994)aswellasthroughfieldexperiments(seeCardenas2003CardenasStranlundandWillis2000Henrichetal2004List2004)
Small-Nqualitativestudiescansuggesttheplausibilityofformalmod-elsbutprovidelittleleverageinassessingthegeneralityofrelationshipsThebroad comparisons required to evaluate thegeneralityofhypoth-esizedrelationshipsdemandsomeformofquantitativeanalysisWherequantitativeanalysisoncemeantregression-basedanalysisoptions forquantitativeanalysisofempiricalsocialsciencedatanowincludeQuali-tative Comparative Analysis (QCA) and fuzzy-set Qualitative Com-parativeAnalysis(fsQCA)(Ragin19872000)aswellasprobabilisticlikelihood-basedandBayesianstatistics(Gill2004)9 Thismethodologi-calmenuincludesoptionsforscholarswhoholdvariedontologicalas-sumptionsaboutthesocialworld
MixedmethodstakeavarietyofformsAresearchermightusedifferentmethodstoaddressdifferentresearchquestionsorcontextsOrdifferentmethodsmightguidedifferentstagesofaresearchprogram(Lieberman2005)IncreasinglyscholarsstrivetousetwoormoremethodsateachstageofresearchThoseconcernedwithgeneralcausalpatternsdrawonquantitativeandqualitativemethods(Coppedge1999Lieberman2005Tarrow2004)Combinationsofformalandqualitativemethodsaddressconcernsaboutlogicalcoherenceandcausalprocessesincontingentrela-tionshipswherethereisnoexpectationofgenerality(Batesetal1998)Otherscontendthatscholarsshouldseeklogicalcoherenceandevidencefor causalprocesses and should test for thegeneralityof relationsbydrawingonformalqualitativeandquantitativemethods(Granatoand
Copyrighted Material
OvercomingChallenges bull 13
Scioli2004Laitin2003)Scholarswhodevelopagent-basedmodelsuserolegamesandexperiments tocollectdataaswellas involving stake-holdersinthevalidationoftheirmodels(BarreteauLePageandAquino2003Bousquetetal2002GurungBousquetandTreacutebuil2006)Otherscholars combine their formal models with ethnographic observations(Bharwanietal2005HuigenOvermarsanddeGroot2006)
Theuseofmultiplemethodshoweverdoesnotguaranteemethod-ologicallysuperiorsocialscienceresearchSomequestiontheextenttowhichformalqualitativeandquantitativeresearchmethodsareactuallycomplementary Several recent publications have argued that differentmethodsreflectdifferentassumptionsaboutthenatureofcausalityandhavecalledforgreatercareinmatchingmethodstoontologicalassump-tions(BennettandElman2006ClarkGilliganandGolder2006Hall2003Mahoney2003Ragin19872000)
Therearealso limits to the feasibilityofmultimethodresearchHy-potheses about complex causal relationships imply complex statisticalmodelsthatstretchthelimitsofavailabledataStatisticalanalysesoftenadd interaction termsordummyvariables tomodel contingent effectsandmultiplecausalpaths(ClarkGilliganandGolder2006Pontusson2007)but theseadditionalvariablesconsumedegreesof freedominacontextoflimiteddataavailability(Shalev2007)Othertechniquesde-velopedtoaddresscausalcomplexitysuchastheanalysisoftime-series-cross-sectional data and hierarchical models may strain the technicalskillsofboththeresearcherandtheaudience(Shalev2007)
MahoneyandGoertz(2006)contendthatinteractioneffectsdummyvariableshierarchicalmodelsandothersimilarstatisticalfixesdonotaccuratelyreflecttherelationshipspositedintheunderlyingtheoriesTheassumptionthatobservationsareindependentforexampleiscalledintoquestionbyglobalizationdiffusioneffectsandactor-centeredtheoriesthatemphasizestrategicinteractionsEvensomequantitativelyorientedscholarsquestiontheappropriatenessofstandardstatisticaltechniquesInrecentyearsnewtechniqueshavebeenproposedtoincorporateinter-dependence(Signorino1999)Bayesianstatistics(Dion1998Gill2004)andBooleanlogic(Braumoeller2003Ragin19872000)Theverdictisstilloutonwhetherthesenewtechniquesmatchunderlyingassumptionsbetterthandoesregression-basedstatistics
Too often the development of ever-more sophisticated techniquesseemstobeanendinitselfThelatesttechniquesaresometimesadoptedwithlittlereferencetotheoreticalconsiderationsorunderstandingoftheunderlyingassumptionsButmethodologicalsophisticationcannotsub-stitutefortheoryAchen(20022005)warnsthatquantitativeanalysesthat are not supported by theoretical microfoundations or careful ex-plorationofthedatayieldunreliableresultsandshouldnotbetrusted
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 5
methodologicalcontroversiesoftenreflectcompetitionbetweenresearchtraditions
Fortunately social scientists increasingly recognize trade-offs acrossmethods (Bates 2007 Brady and Collier 2004 Gerring 2001)1 KingKeohaneandVerba(1994)forexamplepointoutthatallmethodolo-gieshavelimitationsscholarsshouldbemoreawareoftheselimitsandmoretransparentaboutthe limitsaswellas thesolidcontributionsoftheir work To overcome the limits of any one method one needs todrawonmultiplemethods(Batesetal1998Coppedge1999GranatoandScioli2004Jackman1985KingKeohaneandVerba1994Laitin2003Lieberman2005Scharpf2000Tarrow2004)Ifsocialscientistshave shared standardsno singlemethod fullyaddressesall standardsMethods offer different strengths and weaknesses Rigorous researchthatcombinescomplementarymethodswillbesuperiortoresearchthatreliesonanysinglemethod(Grayetal2007)
Thepragmatismandrespectfordiversemethodologicaltraditionsinthese reflections are welcome Too often however the challenges in-volvedinusingmultiplemethodsarethemselvesoverlookedProponentsof mixed methods justify their preferred combination in logical termsandillustratetheapproachwithafewexamplesWithsomeexceptions(Lieberman2005Scharpf2000)thisliteratureoffersfewspecificpracti-calsuggestions
Practical challenges canbe formidableNotallmethodsare equallyfeasibleorevenappropriateforallresearchtopics(BennettandElman2006PoteeteandOstrom2008)Liebermanrsquos (2005)nestedanalysisfor example involves large-N analysis prior to any case study workTherearemany important topics forwhichbroadlycomparativedataare scarce difficult to access or of dubiousqualityLieberman how-everdoesnotaddressthesechallengesEvenifdataavailabilityisnotaproblemthevalueofamultimethodapproachrequiressufficientcom-mandofmultiplemethodsYet considerable investment is required togaincompetencyinanymethodologyandthebenefitsofmethodologi-calspecializationaresubstantialWhilethesechallengesaresometimesacknowledgedfewsocialscientistsmakepracticalsuggestionstoaddressthem
Thisbookfocusesonthepracticalchallengesthatinfluencemethod-ologicalchoiceWeareparticularlyconcernedwithresearchontopicsforwhichdataarescarcedifficulttocollectandnotreadilycomparableTheseconditionsaffect researchonawidevarietyof topics includingthose concerned with informal institutions subnational organizationsand nonelite populations We focus on collective action for the man-agementofnaturalresourcesanareaofresearchinwhichallofthese
Copyrighted Material
6 bull Chapter1
conditionsapplyForsuchtopicsdataforlarge-Nanalysisareneitheravailable nor readily accessible andfield research is unavoidableRe-searchersoftenneedconsiderablecontextualknowledgeeventorecog-nizethephenomenonofinterestTheneedtoconductintensivefieldworklimitsthepotentialforcollectingenoughdatatosupportbroadlycom-parativeanalysis
WehavebecomestronglyawareofthesechallengesthroughourownworkoncollectiveactionandnaturalresourcemanagementWefeelthatthe practical challenges of conducting rigorous social science researchontopicsforwhichdataarescarceordifficulttoaccessortointerprethavenotreceivedadequateattentionindiscussionsaboutsocialscienceresearchWehaveseenthebenefitsofcollaborationandthecombinationofmultiplemethodsinourresearchWealsohavefirsthandexperienceof the challenges involved in such research and we will discuss thesethroughoutthisbook
Inthischapterweintroducefourthemesthatrecurthroughthebook(1)theinterlinkingofmethodologicaldebateswiththeoreticaldevelop-ment (2) theadvantagesand limitationsofmultiplemethodsandcol-laborativeresearch(3)practicalconstraintsonmethodologicalchoicesand(4)theoftenproblematicinfluenceofcareerincentivesonmethod-ologicalpracticeInthisbookweexplicitlyacknowledgethepracticalchallengesthataffectmethodologicalchoicesevaluateseveralstrategiesforaddressingthesechallengesanddirectattentiontotheinfluenceofcareer incentives on methodological choices in social science researchWediscussarangeofoptionsforbalancingcompetingmethodologicaldemandsundertheinevitableconditionsoflimitedresourcesincludingavarietyoftechniquesthatwefeelhavebeenunderutilizedinthesocialsciencesWediscussthemeritsandlimitsofeachmethodaswellasthepossibilities forandconstraintsoncombiningvariousmethods Inourdiscussionofconstraintsonmethodologicalchoicewehopetostimulatea debate about professional incentives and other structural aspects ofacademiathatinfluencehowresearchisconducted
Thisbookismoreaboutmethodologicalpracticethanaboutmethod-ologicalidealsWethusbeginthischapterwithahistoricaloverviewofmethodologicaldebateshighlightinginteractionsamongmethodologicalpracticeschangingtheoreticalorientationsandcompetitionforprofes-sionalstatusandresourcesWethenlookmorecloselyatissuessurround-ingresearchthatusesmultiplemethodsanapproachthathasgainedinacceptanceinrecentyearsThisleadstoadiscussionofconstraintsonmethodologicalchoicebothpracticalandprofessionalWethenexplainhowoursubstantivefocusmdashthestudyofcollectiveactioninnaturalre-sourcemanagementmdashhelpsusaddressourfourthematicconcernsThechapterconcludeswithanoutlineoftherestofthebook
Copyrighted Material
OvercomingChallenges bull 7
Social Science Debates over the Superiority of Particular Methods
Thehistoryofthesocialsciencescanberecountedwithreferencetoma-jormethodologicalshiftsAninitialrelianceonqualitativeanalysisgavewaydramaticallytoquantificationintheearlytomid-twentiethcenturyWhenthistransformationbeganquantificationlargelymeantstatisticalanalysisoflarge-NdatasetsofpublicopinionsurveysThelastthirdofthetwentiethcenturysawasurgeintheuseofformalmodelsaswellDebatesabout the relativemeritsofqualitative statisticaland formalmethodscontributedtoseveraldevelopments inthe late twentiethandearlytwenty-firstcenturiesrefinementsofquantitativemethodsthatat-tempttobettermatchsocialconditionstheriseofformalmodelsgreaterappreciation for combiningmultiplemethods and the spreadofpost-positivistmethodssuchasdiscourseanalysis
Thequalitativeorientationoftheearlysocialsciencescanbeseenintheemphasisoncase studiesandparticipantobservation in sociologyethnographicfield-basedresearchinanthropologyanddescriptiveandnormativeanalysesofformallegalarrangementsIntheearlydecadesofthetwentiethcenturymanyscholarsembracedquantitativemethodsaspartofadrivetomakethesocialsciencesmorescientific2 Quantitativemethodsbegantogaincurrencyacrossthesocialsciencesinthe1920sand1930sTheadoptionofthesemethodsacceleratedatmidcenturyasconveyedbyreferencestothebehavioralrevolution
ThebranchesofthesocialsciencesdifferedintheirtimingpaceandpreferredformsofquantificationNonethelessthemethodologicalshiftfromqualitativetoquantitativemethodsinthesocialscienceswasdra-maticPsychologyrapidlyadoptedexperimentalandstatisticalmethodsQuantitativemethodsineconomicsencompassedformalmodelsaswellas experiments and statistics For sociology research activities duringWorldWarIImarkedtheascendanceofsurveyresearchexperimentsand statistical formsofanalysis (Platt1986)Postwarpolitical scienceshared the enthusiasm for survey research and statistical analysis butformalmodelingbecamewidespreadonly in the1980s and1990s Insocioculturalanthropologysomeinterestwasexpressedinmathematicalmodelsintheearlypostwarperiodbutmultivariatestatisticalanalysesremainedrelativelyrareuntilthe1970s(Chibnik1985)
Theroleofquantitativemethodsinthesocialscienceshasalwaysbeencontentious Current methodological debates echo those of a centuryago even if framed in somewhat different terms3 Scholars concernedwithmethodshavedisagreedover (1) thegoalsof social research (2)philosophical and theoretical issues and (3) practical considerationsespecially related to data quality Methodological choices should be
Copyrighted Material
8 bull Chapter1
drivenbytheoreticalandontologicalassumptions(Hall2003)buttheyalso reflectunderlying values andbeliefs (MahoneyandGoertz2006)andpractical considerations (Platt1986)Theontologicalandnorma-tive dimensions of methodological choices are not widely recognized(MahoneyandGoertz2006)Asa result social sciencedebatesaboutmethodsinvolvefrequentmisunderstandingswithproponentsofdiffer-entapproachestalkingpasteachother(EOstrom2006)Furthermorebecause methodological discussions rarely acknowledge practical andprofessionalconsiderationstheyofferlittleguidanceonhowtoaddresstheseconstraintsInthissectionwediscusscontroversiesoverthegoalsofsocialresearchandhowphilosophicalandtheoreticalissuesinteractwithprofessionalcompetitionWeexpandourtreatmentofpracticalandprofessionalconsiderationsinsubsequentsections
During the1920sand1930s the social sciencesbecamemore insti-tutionalized in North America The social sciences sought recognitionassciencesandeachdisciplinedevelopedamoreorlessdistinctprofes-sionalidentity(Guy2003Platt1986)ThisprocessofinstitutionalizationinfluencedmethodologicaldebatesDuringtheprewarperioddisagree-ments focused on the goals of social research Should sociological re-searchsupportsocialworktoimprovesocialconditionsseeksubjectiveunderstandingoflifeexperiencesorattempttoidentifygeneralpatterns(Platt1986)Shouldthestudyofpoliticsprovidenormativeandpracti-cal guidance for administratorsorobjectiveunderstandingofpoliticalphenomena(Guy2003Lasswell1951)Asuniversitiessetupschoolsofsocialworkpublicadministrationandbusinessadministrationalong-sidedepartmentsof sociologypolitical scienceandeconomicsdiffer-encesovergoalswerealleviatedmdashbutnotreallyaddressedmdashthroughtheinstitutionalizationofmorefocusedprogramsofstudy
Yetdifferencesovertherelativeimportanceoftheoryandpraxiscan-not fully account for methodological debates Scholars with commongoalsdisagreeovermethodsandscholarsdrawonthesamemethodstopursuedivergentgoalsAlackofconsensusonfundamentalphilosophi-cal issues contributes to disagreements over methods What counts asscienceWhatmodelormodelsofcausalityandexplanationmakesenseforsocialphenomenaInparticulardomodelsofscienceandexplana-tiondevelopedinthenaturalandespeciallythephysicalsciencesmakesenseforthesocialsciences
Overthepastcenturysomehaveembraceddeductivemodelsofscienceinspiredbythenaturalsciencesasawaytogainmorereliableinsightsaboutsocialprocesses(KingKeohaneandVerba1994PrzeworskiandTeune 1970) Deduction involves the logical derivation of universalis-ticlawlikestatementsofthesetsofconditionsassociatedwiththeout-comeofinterestfromtheoreticalassumptionsLawlikestatementsmay
Copyrighted Material
OvercomingChallenges bull 9
be derived from formal or mathematical models as in rational-choiceapproachesorlogicalanalysisasinsomequalitativestudiesEmpiricalevaluationsrelyontheanalysisofcorrelationasinbehavioralresearchor paired comparisons4 The journal Public Choice devoted a specialissue in December 2008 to the topic ldquoHomo Economicus and HomoPoliticusrdquo(editedbyGeoffreyBrennanandMichaelGillespie)withninearticlesaddressingthequestionofhowtoreconcilethebasicdifferencesbetweentheoriesofhumanbehaviorineconomicsandpoliticalscienceIntheintroductionBrennan(2008431)reflectsthat
theambitiontofindcommongroundonwhichpublicchoicescholarsandldquopoliticaltheoristsrdquoofamoretraditionalkindmighthaveprofit-ableexchangeisnotatrivialonewestartfromverydifferentconcep-tionsofwhatcountsas theorymdashevenofwhatcountsasworthwhilescholarshipmdashandfromratherdifferentdisciplinarypresuppositionsastohowdifferences inapproachcanmostprofitablybeengagedandresolved
Criticshoweverarguethatdeductivemethodsdonotallowforhu-managencyandreflexivitytheinfluenceofmeaningandinterpretationorcontingentrelationships(AlmondandGenco1977Hall2003Ragin1987seereviewinPlatt1986)Ifagencyistakenseriouslywemustal-lowforbothcreativityanddifferencesinperspectivesButcreativityanddifferencesininterpretationmeanthatlawlikesocialpatternsareunlikelytoariseContingentrelationshipsarepossibleevenifquestionsofagencyareputasideThesedifferencesoverthenatureofcausalityhavefueledheatedmethodologicaldebatesInpoliticalscienceboththebehavioralrevolutionoftheearlypostwarperiodandtheriseofrational-choicethe-oryinthe1980sand1990sassumedthevalueofdeductive-nomologicalreasoningScholarswhousedmethodsthatreflectedalternativeontologi-calassumptionshaddifficultygainingrecognitionfortheirworkTheirfrustrationgaverisetotherecentperestroikamovementinwhichcon-structivistsandotherschallengedboththeuniversalityofsocialpatternsassumedbyrationalchoiceandbehavioraltheoriesandthedominanceofstatisticalandformalmethodsassociatedwiththeseapproachesintheprofession(Monroe2005)Withineconomicstheconcernthatnarrowrational-choicemodelshavecometodominatemuchofeconomicschol-arshipisregularlyexpressedintheonlinejournalReal-World Economics Review5
Deductive-nomological reasoning suggests a mechanical view of theworldinwhichthesamestimulusproducesthesameeffectceterispa-ribusTheoriesthatviewsocialphenomenaasproductsofeitherevolu-tionaryprocessesor intentionalactionchallenge thismechanicalviewBothevolutionaryandintentionaltheoriesassumethatindividualsand
Copyrighted Material
10 bull Chapter1
organizationsadjusttheirresponsestosocialconditions(Alchian1950Brady2004EOstrom2000Thelen2003)Intentionaltheoriesofhu-manbehaviorassumethatadaptationoccursaspeoplestruggletosolvepuzzlesrelatedtothepursuitoftheirgoals(AlmondandGenco1977Elster1983Knight1992)While some intentional theories emphasizeroutines and heuristics there is always a possibility for creativity andinnovation (March andOlsen1984 Simon1955)Evolutionary theo-riesdonotrequire intentionalitybutdorequiresomesortofselectionmechanismsuchasmarketorelectoralcompetitiontodriveadaptationBothformsofadaptationimplythatthesamecircumstanceswillgener-atediverse responsesacrossactorsandchanges in individualbehaviorover timebut thatadaptationswill reflecthistorical trajectoriesThusthesamestimuluswillnot producethesameeffectonaverageandcon-stanteffectscannotbeassumedBothperspectivesraisequestionsaboutthesuitabilityof researchmethods thatassumeconstanteffects (Elster1998Hall2003Ragin19872000)
ThechoiceofmethodtendstosignalonersquostheoreticalperspectiveasdoesthenatureofmethodologicalcritiqueThosewhodiscountquali-tativemethodsasincapableofevaluatinggeneralrelationshipssignalabelief inbothlawlikesocialrelationsandtherelativeunimportanceoffactorssuchasagencyhistoryandinformalcontextNotsurprisinglycritiquesofquantitativemethodsoftenchargethattheydonotcapturethemost important aspects of social conditionsLikewise thosewaryofformalmodelsworryaboutthelevelofabstractionHowcanformalmodelsadequatelyrepresentthedensenetworksofformalandinformalinstitutionsandculturalunderstandingsinwhichhumanactionoccursNoneof these critiques really concerns themethodasmethod ratherthey target the theoretical assumptions as reflected in methodologicalchoicesWhatvariablesareimportantWhatistherelativeimportanceofformalinstitutionsculturesocialstructureorinformalinstitutionsHow important aremass beliefs andbehavior or individual interestsbeliefsandstrategicactionHowarethosevariablesrelatedWhilethebehavioralrevolutionduringthemid-twentiethcenturycertainlyfosteredtherapidspreadofquantitativeanalysisitalsoredirectedtheoreticalem-phasisfromformalinstitutionstothebehaviorandattitudesofindividu-als interactingwithinboth formal and informal institutionsLikewiserational-choiceanalysisoftenreliesongametheoryandothervarietiesof formal modeling but is defined by assumptions of methodologicalindividualismandintentionalaction
Yettheinfluenceoftheorymdashandtheimpliedinfluenceofontologymdashonmethodologicalpracticecannotbeassumedandshouldnotbeover-stated6 Theoreticalchangescananddooccurindependentlyofchangesinmethodologicalpractice(Hall2003Platt1986)Sometimesmethod-
Copyrighted Material
OvercomingChallenges bull 11
ologicalchallengesseemtodrivetheoreticalargumentsratherthantheotherwayaround(Lieberson1991318)Indeedsophisticatedmethodssometimescrowdouttheoryaltogether(Achen20022005)WearguethatmethodologicalchoicesareoftendrivenasmuchbydataavailabilityorcareerincentivesWhencareersurvivalisatstakepracticalconsider-ationscansqueezeoutconcernsaboutmatchingtheoryandmethodThelinkbetweenmethodsandcareerprospectscanhoweverbeexpectedtoinfluencethetenorofmethodologicalmdashandtheoreticalmdashdebates
SometimesmethodologicalandtheoreticaldebatestakeonexistentialovertonesWhenaparticulartheoryandassociatedmethodsbecomeex-tremelywidespreadforexampleproponentsofalternativeapproachesmayworryabouttheirownacademicsurvivalProponentsofnewtheo-riesmdashandassociatedmethodsmdashalsofaceanexistentialfightforrecogni-tionandsurvivalThedegreeof(perceived)existentialthreatdependsonthe extent to which fellowships job opportunities publishing outletsandresearchgrantsareopen(orclosed)todiversetheoriesandmethodsTherecentperestroikamovementinpoliticalscienceforexamplepre-sented itself asdefendingagainstmethodologicalhegemony conjuringimages of political scientists conspiring to control journal outlets andprofessionalassociations7ThiswasnotsimplyamethodologicalcritiquebutacalltoactionagainstpresumedtyrannyWhiletheinflammatorypublic languageassociatedwiththeperestroikamovementmaybeun-usualthelayeringofmethodologicaldebateswithvaluejudgmentsandcompetitionforprofessionalrecognitionandresourcesiscommonplace
Despite references to ldquorevolutionsrdquo and paradigm shifts new socialsciencetheoriesandmethodshavenotfullydisplacedtheirpredecessorsRathereachnewtheoryandmethodhasaddedanotherstrandConstruc-tivists institutionalists andpostmodernists coexistwithbehavioralistsandstructuralistsDespitethehistoryoftheoreticalandmethodologicalcompetitionandcritiquescholarsalsoengageincreativesynthesisThecurrentappreciationformethodologicalpluralismmaybeinterpretedasaproductofthesurvivalandadaptationofapproachesthatwereonceperceived to be under existential threat Promotion of methodologicalpluralismfavorsatheoreticaleclecticismthatshoulddecreaseconcernsaboutexistential threats toparticularapproachesandshouldthusde-creasetheintensityofmethodologicaldebates
Multiple Methods Promises and Challenges
Therearemanyreasonsforsocialscientiststowelcomemethodologicalpluralism and greater use of mixed methods No single method over-comes all challenges Case studies and small-N comparative research
Copyrighted Material
12 bull Chapter1
designs offer advantages for concept and theory development as wellasevaluationofhypothesizedcausalsequencesandmechanisms(Bates2008Batesetal1998CollierBradyandSeawright2004Coppedge1999 Lieberman 2005) Rich explanations of particular cases are of-tenvaluablesubstantivelyandtheoretically(MahoneyandGoertz2006Rogowski2004)Yetasiswidelyrecognizedsmall-Nstudiesofferanuncertainfoundationforpositingorevaluatinggeneralrelationships
Formalmethodsseektobuild logicallycoherentmodelsanddiscerntheir logical implicationsTheir emphasison logical consistency facili-tatesthedistillationofparsimoniousyetgeneralhypothesesandguidesthechoiceofstatisticaltechniques(Achen20022005Batesetal1998GranatoandScioli2004)8 Thehighlevelofabstractioninformalmod-elshoweverraisesquestionsabouttheirempiricalapplicability(GreenandShapiro1994)Thecontrolledconditionsinexperimentalresearchprovidegreaterconfidenceintheinternalvalidityofobservedrelation-shipsTheexternalvalidityofgeneralrelationshipscanbestbeevaluatedhoweverthroughanalysisofalargenumberofnonexperimentalobser-vations(Goldthorpe1997KingKeohaneandVerba1994)aswellasthroughfieldexperiments(seeCardenas2003CardenasStranlundandWillis2000Henrichetal2004List2004)
Small-Nqualitativestudiescansuggesttheplausibilityofformalmod-elsbutprovidelittleleverageinassessingthegeneralityofrelationshipsThebroad comparisons required to evaluate thegeneralityofhypoth-esizedrelationshipsdemandsomeformofquantitativeanalysisWherequantitativeanalysisoncemeantregression-basedanalysisoptions forquantitativeanalysisofempiricalsocialsciencedatanowincludeQuali-tative Comparative Analysis (QCA) and fuzzy-set Qualitative Com-parativeAnalysis(fsQCA)(Ragin19872000)aswellasprobabilisticlikelihood-basedandBayesianstatistics(Gill2004)9 Thismethodologi-calmenuincludesoptionsforscholarswhoholdvariedontologicalas-sumptionsaboutthesocialworld
MixedmethodstakeavarietyofformsAresearchermightusedifferentmethodstoaddressdifferentresearchquestionsorcontextsOrdifferentmethodsmightguidedifferentstagesofaresearchprogram(Lieberman2005)IncreasinglyscholarsstrivetousetwoormoremethodsateachstageofresearchThoseconcernedwithgeneralcausalpatternsdrawonquantitativeandqualitativemethods(Coppedge1999Lieberman2005Tarrow2004)Combinationsofformalandqualitativemethodsaddressconcernsaboutlogicalcoherenceandcausalprocessesincontingentrela-tionshipswherethereisnoexpectationofgenerality(Batesetal1998)Otherscontendthatscholarsshouldseeklogicalcoherenceandevidencefor causalprocesses and should test for thegeneralityof relationsbydrawingonformalqualitativeandquantitativemethods(Granatoand
Copyrighted Material
OvercomingChallenges bull 13
Scioli2004Laitin2003)Scholarswhodevelopagent-basedmodelsuserolegamesandexperiments tocollectdataaswellas involving stake-holdersinthevalidationoftheirmodels(BarreteauLePageandAquino2003Bousquetetal2002GurungBousquetandTreacutebuil2006)Otherscholars combine their formal models with ethnographic observations(Bharwanietal2005HuigenOvermarsanddeGroot2006)
Theuseofmultiplemethodshoweverdoesnotguaranteemethod-ologicallysuperiorsocialscienceresearchSomequestiontheextenttowhichformalqualitativeandquantitativeresearchmethodsareactuallycomplementary Several recent publications have argued that differentmethodsreflectdifferentassumptionsaboutthenatureofcausalityandhavecalledforgreatercareinmatchingmethodstoontologicalassump-tions(BennettandElman2006ClarkGilliganandGolder2006Hall2003Mahoney2003Ragin19872000)
Therearealso limits to the feasibilityofmultimethodresearchHy-potheses about complex causal relationships imply complex statisticalmodelsthatstretchthelimitsofavailabledataStatisticalanalysesoftenadd interaction termsordummyvariables tomodel contingent effectsandmultiplecausalpaths(ClarkGilliganandGolder2006Pontusson2007)but theseadditionalvariablesconsumedegreesof freedominacontextoflimiteddataavailability(Shalev2007)Othertechniquesde-velopedtoaddresscausalcomplexitysuchastheanalysisoftime-series-cross-sectional data and hierarchical models may strain the technicalskillsofboththeresearcherandtheaudience(Shalev2007)
MahoneyandGoertz(2006)contendthatinteractioneffectsdummyvariableshierarchicalmodelsandothersimilarstatisticalfixesdonotaccuratelyreflecttherelationshipspositedintheunderlyingtheoriesTheassumptionthatobservationsareindependentforexampleiscalledintoquestionbyglobalizationdiffusioneffectsandactor-centeredtheoriesthatemphasizestrategicinteractionsEvensomequantitativelyorientedscholarsquestiontheappropriatenessofstandardstatisticaltechniquesInrecentyearsnewtechniqueshavebeenproposedtoincorporateinter-dependence(Signorino1999)Bayesianstatistics(Dion1998Gill2004)andBooleanlogic(Braumoeller2003Ragin19872000)Theverdictisstilloutonwhetherthesenewtechniquesmatchunderlyingassumptionsbetterthandoesregression-basedstatistics
Too often the development of ever-more sophisticated techniquesseemstobeanendinitselfThelatesttechniquesaresometimesadoptedwithlittlereferencetotheoreticalconsiderationsorunderstandingoftheunderlyingassumptionsButmethodologicalsophisticationcannotsub-stitutefortheoryAchen(20022005)warnsthatquantitativeanalysesthat are not supported by theoretical microfoundations or careful ex-plorationofthedatayieldunreliableresultsandshouldnotbetrusted
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
6 bull Chapter1
conditionsapplyForsuchtopicsdataforlarge-Nanalysisareneitheravailable nor readily accessible andfield research is unavoidableRe-searchersoftenneedconsiderablecontextualknowledgeeventorecog-nizethephenomenonofinterestTheneedtoconductintensivefieldworklimitsthepotentialforcollectingenoughdatatosupportbroadlycom-parativeanalysis
WehavebecomestronglyawareofthesechallengesthroughourownworkoncollectiveactionandnaturalresourcemanagementWefeelthatthe practical challenges of conducting rigorous social science researchontopicsforwhichdataarescarceordifficulttoaccessortointerprethavenotreceivedadequateattentionindiscussionsaboutsocialscienceresearchWehaveseenthebenefitsofcollaborationandthecombinationofmultiplemethodsinourresearchWealsohavefirsthandexperienceof the challenges involved in such research and we will discuss thesethroughoutthisbook
Inthischapterweintroducefourthemesthatrecurthroughthebook(1)theinterlinkingofmethodologicaldebateswiththeoreticaldevelop-ment (2) theadvantagesand limitationsofmultiplemethodsandcol-laborativeresearch(3)practicalconstraintsonmethodologicalchoicesand(4)theoftenproblematicinfluenceofcareerincentivesonmethod-ologicalpracticeInthisbookweexplicitlyacknowledgethepracticalchallengesthataffectmethodologicalchoicesevaluateseveralstrategiesforaddressingthesechallengesanddirectattentiontotheinfluenceofcareer incentives on methodological choices in social science researchWediscussarangeofoptionsforbalancingcompetingmethodologicaldemandsundertheinevitableconditionsoflimitedresourcesincludingavarietyoftechniquesthatwefeelhavebeenunderutilizedinthesocialsciencesWediscussthemeritsandlimitsofeachmethodaswellasthepossibilities forandconstraintsoncombiningvariousmethods Inourdiscussionofconstraintsonmethodologicalchoicewehopetostimulatea debate about professional incentives and other structural aspects ofacademiathatinfluencehowresearchisconducted
Thisbookismoreaboutmethodologicalpracticethanaboutmethod-ologicalidealsWethusbeginthischapterwithahistoricaloverviewofmethodologicaldebateshighlightinginteractionsamongmethodologicalpracticeschangingtheoreticalorientationsandcompetitionforprofes-sionalstatusandresourcesWethenlookmorecloselyatissuessurround-ingresearchthatusesmultiplemethodsanapproachthathasgainedinacceptanceinrecentyearsThisleadstoadiscussionofconstraintsonmethodologicalchoicebothpracticalandprofessionalWethenexplainhowoursubstantivefocusmdashthestudyofcollectiveactioninnaturalre-sourcemanagementmdashhelpsusaddressourfourthematicconcernsThechapterconcludeswithanoutlineoftherestofthebook
Copyrighted Material
OvercomingChallenges bull 7
Social Science Debates over the Superiority of Particular Methods
Thehistoryofthesocialsciencescanberecountedwithreferencetoma-jormethodologicalshiftsAninitialrelianceonqualitativeanalysisgavewaydramaticallytoquantificationintheearlytomid-twentiethcenturyWhenthistransformationbeganquantificationlargelymeantstatisticalanalysisoflarge-NdatasetsofpublicopinionsurveysThelastthirdofthetwentiethcenturysawasurgeintheuseofformalmodelsaswellDebatesabout the relativemeritsofqualitative statisticaland formalmethodscontributedtoseveraldevelopments inthe late twentiethandearlytwenty-firstcenturiesrefinementsofquantitativemethodsthatat-tempttobettermatchsocialconditionstheriseofformalmodelsgreaterappreciation for combiningmultiplemethods and the spreadofpost-positivistmethodssuchasdiscourseanalysis
Thequalitativeorientationoftheearlysocialsciencescanbeseenintheemphasisoncase studiesandparticipantobservation in sociologyethnographicfield-basedresearchinanthropologyanddescriptiveandnormativeanalysesofformallegalarrangementsIntheearlydecadesofthetwentiethcenturymanyscholarsembracedquantitativemethodsaspartofadrivetomakethesocialsciencesmorescientific2 Quantitativemethodsbegantogaincurrencyacrossthesocialsciencesinthe1920sand1930sTheadoptionofthesemethodsacceleratedatmidcenturyasconveyedbyreferencestothebehavioralrevolution
ThebranchesofthesocialsciencesdifferedintheirtimingpaceandpreferredformsofquantificationNonethelessthemethodologicalshiftfromqualitativetoquantitativemethodsinthesocialscienceswasdra-maticPsychologyrapidlyadoptedexperimentalandstatisticalmethodsQuantitativemethodsineconomicsencompassedformalmodelsaswellas experiments and statistics For sociology research activities duringWorldWarIImarkedtheascendanceofsurveyresearchexperimentsand statistical formsofanalysis (Platt1986)Postwarpolitical scienceshared the enthusiasm for survey research and statistical analysis butformalmodelingbecamewidespreadonly in the1980s and1990s Insocioculturalanthropologysomeinterestwasexpressedinmathematicalmodelsintheearlypostwarperiodbutmultivariatestatisticalanalysesremainedrelativelyrareuntilthe1970s(Chibnik1985)
Theroleofquantitativemethodsinthesocialscienceshasalwaysbeencontentious Current methodological debates echo those of a centuryago even if framed in somewhat different terms3 Scholars concernedwithmethodshavedisagreedover (1) thegoalsof social research (2)philosophical and theoretical issues and (3) practical considerationsespecially related to data quality Methodological choices should be
Copyrighted Material
8 bull Chapter1
drivenbytheoreticalandontologicalassumptions(Hall2003)buttheyalso reflectunderlying values andbeliefs (MahoneyandGoertz2006)andpractical considerations (Platt1986)Theontologicalandnorma-tive dimensions of methodological choices are not widely recognized(MahoneyandGoertz2006)Asa result social sciencedebatesaboutmethodsinvolvefrequentmisunderstandingswithproponentsofdiffer-entapproachestalkingpasteachother(EOstrom2006)Furthermorebecause methodological discussions rarely acknowledge practical andprofessionalconsiderationstheyofferlittleguidanceonhowtoaddresstheseconstraintsInthissectionwediscusscontroversiesoverthegoalsofsocialresearchandhowphilosophicalandtheoreticalissuesinteractwithprofessionalcompetitionWeexpandourtreatmentofpracticalandprofessionalconsiderationsinsubsequentsections
During the1920sand1930s the social sciencesbecamemore insti-tutionalized in North America The social sciences sought recognitionassciencesandeachdisciplinedevelopedamoreorlessdistinctprofes-sionalidentity(Guy2003Platt1986)ThisprocessofinstitutionalizationinfluencedmethodologicaldebatesDuringtheprewarperioddisagree-ments focused on the goals of social research Should sociological re-searchsupportsocialworktoimprovesocialconditionsseeksubjectiveunderstandingoflifeexperiencesorattempttoidentifygeneralpatterns(Platt1986)Shouldthestudyofpoliticsprovidenormativeandpracti-cal guidance for administratorsorobjectiveunderstandingofpoliticalphenomena(Guy2003Lasswell1951)Asuniversitiessetupschoolsofsocialworkpublicadministrationandbusinessadministrationalong-sidedepartmentsof sociologypolitical scienceandeconomicsdiffer-encesovergoalswerealleviatedmdashbutnotreallyaddressedmdashthroughtheinstitutionalizationofmorefocusedprogramsofstudy
Yetdifferencesovertherelativeimportanceoftheoryandpraxiscan-not fully account for methodological debates Scholars with commongoalsdisagreeovermethodsandscholarsdrawonthesamemethodstopursuedivergentgoalsAlackofconsensusonfundamentalphilosophi-cal issues contributes to disagreements over methods What counts asscienceWhatmodelormodelsofcausalityandexplanationmakesenseforsocialphenomenaInparticulardomodelsofscienceandexplana-tiondevelopedinthenaturalandespeciallythephysicalsciencesmakesenseforthesocialsciences
Overthepastcenturysomehaveembraceddeductivemodelsofscienceinspiredbythenaturalsciencesasawaytogainmorereliableinsightsaboutsocialprocesses(KingKeohaneandVerba1994PrzeworskiandTeune 1970) Deduction involves the logical derivation of universalis-ticlawlikestatementsofthesetsofconditionsassociatedwiththeout-comeofinterestfromtheoreticalassumptionsLawlikestatementsmay
Copyrighted Material
OvercomingChallenges bull 9
be derived from formal or mathematical models as in rational-choiceapproachesorlogicalanalysisasinsomequalitativestudiesEmpiricalevaluationsrelyontheanalysisofcorrelationasinbehavioralresearchor paired comparisons4 The journal Public Choice devoted a specialissue in December 2008 to the topic ldquoHomo Economicus and HomoPoliticusrdquo(editedbyGeoffreyBrennanandMichaelGillespie)withninearticlesaddressingthequestionofhowtoreconcilethebasicdifferencesbetweentheoriesofhumanbehaviorineconomicsandpoliticalscienceIntheintroductionBrennan(2008431)reflectsthat
theambitiontofindcommongroundonwhichpublicchoicescholarsandldquopoliticaltheoristsrdquoofamoretraditionalkindmighthaveprofit-ableexchangeisnotatrivialonewestartfromverydifferentconcep-tionsofwhatcountsas theorymdashevenofwhatcountsasworthwhilescholarshipmdashandfromratherdifferentdisciplinarypresuppositionsastohowdifferences inapproachcanmostprofitablybeengagedandresolved
Criticshoweverarguethatdeductivemethodsdonotallowforhu-managencyandreflexivitytheinfluenceofmeaningandinterpretationorcontingentrelationships(AlmondandGenco1977Hall2003Ragin1987seereviewinPlatt1986)Ifagencyistakenseriouslywemustal-lowforbothcreativityanddifferencesinperspectivesButcreativityanddifferencesininterpretationmeanthatlawlikesocialpatternsareunlikelytoariseContingentrelationshipsarepossibleevenifquestionsofagencyareputasideThesedifferencesoverthenatureofcausalityhavefueledheatedmethodologicaldebatesInpoliticalscienceboththebehavioralrevolutionoftheearlypostwarperiodandtheriseofrational-choicethe-oryinthe1980sand1990sassumedthevalueofdeductive-nomologicalreasoningScholarswhousedmethodsthatreflectedalternativeontologi-calassumptionshaddifficultygainingrecognitionfortheirworkTheirfrustrationgaverisetotherecentperestroikamovementinwhichcon-structivistsandotherschallengedboththeuniversalityofsocialpatternsassumedbyrationalchoiceandbehavioraltheoriesandthedominanceofstatisticalandformalmethodsassociatedwiththeseapproachesintheprofession(Monroe2005)Withineconomicstheconcernthatnarrowrational-choicemodelshavecometodominatemuchofeconomicschol-arshipisregularlyexpressedintheonlinejournalReal-World Economics Review5
Deductive-nomological reasoning suggests a mechanical view of theworldinwhichthesamestimulusproducesthesameeffectceterispa-ribusTheoriesthatviewsocialphenomenaasproductsofeitherevolu-tionaryprocessesor intentionalactionchallenge thismechanicalviewBothevolutionaryandintentionaltheoriesassumethatindividualsand
Copyrighted Material
10 bull Chapter1
organizationsadjusttheirresponsestosocialconditions(Alchian1950Brady2004EOstrom2000Thelen2003)Intentionaltheoriesofhu-manbehaviorassumethatadaptationoccursaspeoplestruggletosolvepuzzlesrelatedtothepursuitoftheirgoals(AlmondandGenco1977Elster1983Knight1992)While some intentional theories emphasizeroutines and heuristics there is always a possibility for creativity andinnovation (March andOlsen1984 Simon1955)Evolutionary theo-riesdonotrequire intentionalitybutdorequiresomesortofselectionmechanismsuchasmarketorelectoralcompetitiontodriveadaptationBothformsofadaptationimplythatthesamecircumstanceswillgener-atediverse responsesacrossactorsandchanges in individualbehaviorover timebut thatadaptationswill reflecthistorical trajectoriesThusthesamestimuluswillnot producethesameeffectonaverageandcon-stanteffectscannotbeassumedBothperspectivesraisequestionsaboutthesuitabilityof researchmethods thatassumeconstanteffects (Elster1998Hall2003Ragin19872000)
ThechoiceofmethodtendstosignalonersquostheoreticalperspectiveasdoesthenatureofmethodologicalcritiqueThosewhodiscountquali-tativemethodsasincapableofevaluatinggeneralrelationshipssignalabelief inbothlawlikesocialrelationsandtherelativeunimportanceoffactorssuchasagencyhistoryandinformalcontextNotsurprisinglycritiquesofquantitativemethodsoftenchargethattheydonotcapturethemost important aspects of social conditionsLikewise thosewaryofformalmodelsworryaboutthelevelofabstractionHowcanformalmodelsadequatelyrepresentthedensenetworksofformalandinformalinstitutionsandculturalunderstandingsinwhichhumanactionoccursNoneof these critiques really concerns themethodasmethod ratherthey target the theoretical assumptions as reflected in methodologicalchoicesWhatvariablesareimportantWhatistherelativeimportanceofformalinstitutionsculturesocialstructureorinformalinstitutionsHow important aremass beliefs andbehavior or individual interestsbeliefsandstrategicactionHowarethosevariablesrelatedWhilethebehavioralrevolutionduringthemid-twentiethcenturycertainlyfosteredtherapidspreadofquantitativeanalysisitalsoredirectedtheoreticalem-phasisfromformalinstitutionstothebehaviorandattitudesofindividu-als interactingwithinboth formal and informal institutionsLikewiserational-choiceanalysisoftenreliesongametheoryandothervarietiesof formal modeling but is defined by assumptions of methodologicalindividualismandintentionalaction
Yettheinfluenceoftheorymdashandtheimpliedinfluenceofontologymdashonmethodologicalpracticecannotbeassumedandshouldnotbeover-stated6 Theoreticalchangescananddooccurindependentlyofchangesinmethodologicalpractice(Hall2003Platt1986)Sometimesmethod-
Copyrighted Material
OvercomingChallenges bull 11
ologicalchallengesseemtodrivetheoreticalargumentsratherthantheotherwayaround(Lieberson1991318)Indeedsophisticatedmethodssometimescrowdouttheoryaltogether(Achen20022005)WearguethatmethodologicalchoicesareoftendrivenasmuchbydataavailabilityorcareerincentivesWhencareersurvivalisatstakepracticalconsider-ationscansqueezeoutconcernsaboutmatchingtheoryandmethodThelinkbetweenmethodsandcareerprospectscanhoweverbeexpectedtoinfluencethetenorofmethodologicalmdashandtheoreticalmdashdebates
SometimesmethodologicalandtheoreticaldebatestakeonexistentialovertonesWhenaparticulartheoryandassociatedmethodsbecomeex-tremelywidespreadforexampleproponentsofalternativeapproachesmayworryabouttheirownacademicsurvivalProponentsofnewtheo-riesmdashandassociatedmethodsmdashalsofaceanexistentialfightforrecogni-tionandsurvivalThedegreeof(perceived)existentialthreatdependsonthe extent to which fellowships job opportunities publishing outletsandresearchgrantsareopen(orclosed)todiversetheoriesandmethodsTherecentperestroikamovementinpoliticalscienceforexamplepre-sented itself asdefendingagainstmethodologicalhegemony conjuringimages of political scientists conspiring to control journal outlets andprofessionalassociations7ThiswasnotsimplyamethodologicalcritiquebutacalltoactionagainstpresumedtyrannyWhiletheinflammatorypublic languageassociatedwiththeperestroikamovementmaybeun-usualthelayeringofmethodologicaldebateswithvaluejudgmentsandcompetitionforprofessionalrecognitionandresourcesiscommonplace
Despite references to ldquorevolutionsrdquo and paradigm shifts new socialsciencetheoriesandmethodshavenotfullydisplacedtheirpredecessorsRathereachnewtheoryandmethodhasaddedanotherstrandConstruc-tivists institutionalists andpostmodernists coexistwithbehavioralistsandstructuralistsDespitethehistoryoftheoreticalandmethodologicalcompetitionandcritiquescholarsalsoengageincreativesynthesisThecurrentappreciationformethodologicalpluralismmaybeinterpretedasaproductofthesurvivalandadaptationofapproachesthatwereonceperceived to be under existential threat Promotion of methodologicalpluralismfavorsatheoreticaleclecticismthatshoulddecreaseconcernsaboutexistential threats toparticularapproachesandshouldthusde-creasetheintensityofmethodologicaldebates
Multiple Methods Promises and Challenges
Therearemanyreasonsforsocialscientiststowelcomemethodologicalpluralism and greater use of mixed methods No single method over-comes all challenges Case studies and small-N comparative research
Copyrighted Material
12 bull Chapter1
designs offer advantages for concept and theory development as wellasevaluationofhypothesizedcausalsequencesandmechanisms(Bates2008Batesetal1998CollierBradyandSeawright2004Coppedge1999 Lieberman 2005) Rich explanations of particular cases are of-tenvaluablesubstantivelyandtheoretically(MahoneyandGoertz2006Rogowski2004)Yetasiswidelyrecognizedsmall-Nstudiesofferanuncertainfoundationforpositingorevaluatinggeneralrelationships
Formalmethodsseektobuild logicallycoherentmodelsanddiscerntheir logical implicationsTheir emphasison logical consistency facili-tatesthedistillationofparsimoniousyetgeneralhypothesesandguidesthechoiceofstatisticaltechniques(Achen20022005Batesetal1998GranatoandScioli2004)8 Thehighlevelofabstractioninformalmod-elshoweverraisesquestionsabouttheirempiricalapplicability(GreenandShapiro1994)Thecontrolledconditionsinexperimentalresearchprovidegreaterconfidenceintheinternalvalidityofobservedrelation-shipsTheexternalvalidityofgeneralrelationshipscanbestbeevaluatedhoweverthroughanalysisofalargenumberofnonexperimentalobser-vations(Goldthorpe1997KingKeohaneandVerba1994)aswellasthroughfieldexperiments(seeCardenas2003CardenasStranlundandWillis2000Henrichetal2004List2004)
Small-Nqualitativestudiescansuggesttheplausibilityofformalmod-elsbutprovidelittleleverageinassessingthegeneralityofrelationshipsThebroad comparisons required to evaluate thegeneralityofhypoth-esizedrelationshipsdemandsomeformofquantitativeanalysisWherequantitativeanalysisoncemeantregression-basedanalysisoptions forquantitativeanalysisofempiricalsocialsciencedatanowincludeQuali-tative Comparative Analysis (QCA) and fuzzy-set Qualitative Com-parativeAnalysis(fsQCA)(Ragin19872000)aswellasprobabilisticlikelihood-basedandBayesianstatistics(Gill2004)9 Thismethodologi-calmenuincludesoptionsforscholarswhoholdvariedontologicalas-sumptionsaboutthesocialworld
MixedmethodstakeavarietyofformsAresearchermightusedifferentmethodstoaddressdifferentresearchquestionsorcontextsOrdifferentmethodsmightguidedifferentstagesofaresearchprogram(Lieberman2005)IncreasinglyscholarsstrivetousetwoormoremethodsateachstageofresearchThoseconcernedwithgeneralcausalpatternsdrawonquantitativeandqualitativemethods(Coppedge1999Lieberman2005Tarrow2004)Combinationsofformalandqualitativemethodsaddressconcernsaboutlogicalcoherenceandcausalprocessesincontingentrela-tionshipswherethereisnoexpectationofgenerality(Batesetal1998)Otherscontendthatscholarsshouldseeklogicalcoherenceandevidencefor causalprocesses and should test for thegeneralityof relationsbydrawingonformalqualitativeandquantitativemethods(Granatoand
Copyrighted Material
OvercomingChallenges bull 13
Scioli2004Laitin2003)Scholarswhodevelopagent-basedmodelsuserolegamesandexperiments tocollectdataaswellas involving stake-holdersinthevalidationoftheirmodels(BarreteauLePageandAquino2003Bousquetetal2002GurungBousquetandTreacutebuil2006)Otherscholars combine their formal models with ethnographic observations(Bharwanietal2005HuigenOvermarsanddeGroot2006)
Theuseofmultiplemethodshoweverdoesnotguaranteemethod-ologicallysuperiorsocialscienceresearchSomequestiontheextenttowhichformalqualitativeandquantitativeresearchmethodsareactuallycomplementary Several recent publications have argued that differentmethodsreflectdifferentassumptionsaboutthenatureofcausalityandhavecalledforgreatercareinmatchingmethodstoontologicalassump-tions(BennettandElman2006ClarkGilliganandGolder2006Hall2003Mahoney2003Ragin19872000)
Therearealso limits to the feasibilityofmultimethodresearchHy-potheses about complex causal relationships imply complex statisticalmodelsthatstretchthelimitsofavailabledataStatisticalanalysesoftenadd interaction termsordummyvariables tomodel contingent effectsandmultiplecausalpaths(ClarkGilliganandGolder2006Pontusson2007)but theseadditionalvariablesconsumedegreesof freedominacontextoflimiteddataavailability(Shalev2007)Othertechniquesde-velopedtoaddresscausalcomplexitysuchastheanalysisoftime-series-cross-sectional data and hierarchical models may strain the technicalskillsofboththeresearcherandtheaudience(Shalev2007)
MahoneyandGoertz(2006)contendthatinteractioneffectsdummyvariableshierarchicalmodelsandothersimilarstatisticalfixesdonotaccuratelyreflecttherelationshipspositedintheunderlyingtheoriesTheassumptionthatobservationsareindependentforexampleiscalledintoquestionbyglobalizationdiffusioneffectsandactor-centeredtheoriesthatemphasizestrategicinteractionsEvensomequantitativelyorientedscholarsquestiontheappropriatenessofstandardstatisticaltechniquesInrecentyearsnewtechniqueshavebeenproposedtoincorporateinter-dependence(Signorino1999)Bayesianstatistics(Dion1998Gill2004)andBooleanlogic(Braumoeller2003Ragin19872000)Theverdictisstilloutonwhetherthesenewtechniquesmatchunderlyingassumptionsbetterthandoesregression-basedstatistics
Too often the development of ever-more sophisticated techniquesseemstobeanendinitselfThelatesttechniquesaresometimesadoptedwithlittlereferencetotheoreticalconsiderationsorunderstandingoftheunderlyingassumptionsButmethodologicalsophisticationcannotsub-stitutefortheoryAchen(20022005)warnsthatquantitativeanalysesthat are not supported by theoretical microfoundations or careful ex-plorationofthedatayieldunreliableresultsandshouldnotbetrusted
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 7
Social Science Debates over the Superiority of Particular Methods
Thehistoryofthesocialsciencescanberecountedwithreferencetoma-jormethodologicalshiftsAninitialrelianceonqualitativeanalysisgavewaydramaticallytoquantificationintheearlytomid-twentiethcenturyWhenthistransformationbeganquantificationlargelymeantstatisticalanalysisoflarge-NdatasetsofpublicopinionsurveysThelastthirdofthetwentiethcenturysawasurgeintheuseofformalmodelsaswellDebatesabout the relativemeritsofqualitative statisticaland formalmethodscontributedtoseveraldevelopments inthe late twentiethandearlytwenty-firstcenturiesrefinementsofquantitativemethodsthatat-tempttobettermatchsocialconditionstheriseofformalmodelsgreaterappreciation for combiningmultiplemethods and the spreadofpost-positivistmethodssuchasdiscourseanalysis
Thequalitativeorientationoftheearlysocialsciencescanbeseenintheemphasisoncase studiesandparticipantobservation in sociologyethnographicfield-basedresearchinanthropologyanddescriptiveandnormativeanalysesofformallegalarrangementsIntheearlydecadesofthetwentiethcenturymanyscholarsembracedquantitativemethodsaspartofadrivetomakethesocialsciencesmorescientific2 Quantitativemethodsbegantogaincurrencyacrossthesocialsciencesinthe1920sand1930sTheadoptionofthesemethodsacceleratedatmidcenturyasconveyedbyreferencestothebehavioralrevolution
ThebranchesofthesocialsciencesdifferedintheirtimingpaceandpreferredformsofquantificationNonethelessthemethodologicalshiftfromqualitativetoquantitativemethodsinthesocialscienceswasdra-maticPsychologyrapidlyadoptedexperimentalandstatisticalmethodsQuantitativemethodsineconomicsencompassedformalmodelsaswellas experiments and statistics For sociology research activities duringWorldWarIImarkedtheascendanceofsurveyresearchexperimentsand statistical formsofanalysis (Platt1986)Postwarpolitical scienceshared the enthusiasm for survey research and statistical analysis butformalmodelingbecamewidespreadonly in the1980s and1990s Insocioculturalanthropologysomeinterestwasexpressedinmathematicalmodelsintheearlypostwarperiodbutmultivariatestatisticalanalysesremainedrelativelyrareuntilthe1970s(Chibnik1985)
Theroleofquantitativemethodsinthesocialscienceshasalwaysbeencontentious Current methodological debates echo those of a centuryago even if framed in somewhat different terms3 Scholars concernedwithmethodshavedisagreedover (1) thegoalsof social research (2)philosophical and theoretical issues and (3) practical considerationsespecially related to data quality Methodological choices should be
Copyrighted Material
8 bull Chapter1
drivenbytheoreticalandontologicalassumptions(Hall2003)buttheyalso reflectunderlying values andbeliefs (MahoneyandGoertz2006)andpractical considerations (Platt1986)Theontologicalandnorma-tive dimensions of methodological choices are not widely recognized(MahoneyandGoertz2006)Asa result social sciencedebatesaboutmethodsinvolvefrequentmisunderstandingswithproponentsofdiffer-entapproachestalkingpasteachother(EOstrom2006)Furthermorebecause methodological discussions rarely acknowledge practical andprofessionalconsiderationstheyofferlittleguidanceonhowtoaddresstheseconstraintsInthissectionwediscusscontroversiesoverthegoalsofsocialresearchandhowphilosophicalandtheoreticalissuesinteractwithprofessionalcompetitionWeexpandourtreatmentofpracticalandprofessionalconsiderationsinsubsequentsections
During the1920sand1930s the social sciencesbecamemore insti-tutionalized in North America The social sciences sought recognitionassciencesandeachdisciplinedevelopedamoreorlessdistinctprofes-sionalidentity(Guy2003Platt1986)ThisprocessofinstitutionalizationinfluencedmethodologicaldebatesDuringtheprewarperioddisagree-ments focused on the goals of social research Should sociological re-searchsupportsocialworktoimprovesocialconditionsseeksubjectiveunderstandingoflifeexperiencesorattempttoidentifygeneralpatterns(Platt1986)Shouldthestudyofpoliticsprovidenormativeandpracti-cal guidance for administratorsorobjectiveunderstandingofpoliticalphenomena(Guy2003Lasswell1951)Asuniversitiessetupschoolsofsocialworkpublicadministrationandbusinessadministrationalong-sidedepartmentsof sociologypolitical scienceandeconomicsdiffer-encesovergoalswerealleviatedmdashbutnotreallyaddressedmdashthroughtheinstitutionalizationofmorefocusedprogramsofstudy
Yetdifferencesovertherelativeimportanceoftheoryandpraxiscan-not fully account for methodological debates Scholars with commongoalsdisagreeovermethodsandscholarsdrawonthesamemethodstopursuedivergentgoalsAlackofconsensusonfundamentalphilosophi-cal issues contributes to disagreements over methods What counts asscienceWhatmodelormodelsofcausalityandexplanationmakesenseforsocialphenomenaInparticulardomodelsofscienceandexplana-tiondevelopedinthenaturalandespeciallythephysicalsciencesmakesenseforthesocialsciences
Overthepastcenturysomehaveembraceddeductivemodelsofscienceinspiredbythenaturalsciencesasawaytogainmorereliableinsightsaboutsocialprocesses(KingKeohaneandVerba1994PrzeworskiandTeune 1970) Deduction involves the logical derivation of universalis-ticlawlikestatementsofthesetsofconditionsassociatedwiththeout-comeofinterestfromtheoreticalassumptionsLawlikestatementsmay
Copyrighted Material
OvercomingChallenges bull 9
be derived from formal or mathematical models as in rational-choiceapproachesorlogicalanalysisasinsomequalitativestudiesEmpiricalevaluationsrelyontheanalysisofcorrelationasinbehavioralresearchor paired comparisons4 The journal Public Choice devoted a specialissue in December 2008 to the topic ldquoHomo Economicus and HomoPoliticusrdquo(editedbyGeoffreyBrennanandMichaelGillespie)withninearticlesaddressingthequestionofhowtoreconcilethebasicdifferencesbetweentheoriesofhumanbehaviorineconomicsandpoliticalscienceIntheintroductionBrennan(2008431)reflectsthat
theambitiontofindcommongroundonwhichpublicchoicescholarsandldquopoliticaltheoristsrdquoofamoretraditionalkindmighthaveprofit-ableexchangeisnotatrivialonewestartfromverydifferentconcep-tionsofwhatcountsas theorymdashevenofwhatcountsasworthwhilescholarshipmdashandfromratherdifferentdisciplinarypresuppositionsastohowdifferences inapproachcanmostprofitablybeengagedandresolved
Criticshoweverarguethatdeductivemethodsdonotallowforhu-managencyandreflexivitytheinfluenceofmeaningandinterpretationorcontingentrelationships(AlmondandGenco1977Hall2003Ragin1987seereviewinPlatt1986)Ifagencyistakenseriouslywemustal-lowforbothcreativityanddifferencesinperspectivesButcreativityanddifferencesininterpretationmeanthatlawlikesocialpatternsareunlikelytoariseContingentrelationshipsarepossibleevenifquestionsofagencyareputasideThesedifferencesoverthenatureofcausalityhavefueledheatedmethodologicaldebatesInpoliticalscienceboththebehavioralrevolutionoftheearlypostwarperiodandtheriseofrational-choicethe-oryinthe1980sand1990sassumedthevalueofdeductive-nomologicalreasoningScholarswhousedmethodsthatreflectedalternativeontologi-calassumptionshaddifficultygainingrecognitionfortheirworkTheirfrustrationgaverisetotherecentperestroikamovementinwhichcon-structivistsandotherschallengedboththeuniversalityofsocialpatternsassumedbyrationalchoiceandbehavioraltheoriesandthedominanceofstatisticalandformalmethodsassociatedwiththeseapproachesintheprofession(Monroe2005)Withineconomicstheconcernthatnarrowrational-choicemodelshavecometodominatemuchofeconomicschol-arshipisregularlyexpressedintheonlinejournalReal-World Economics Review5
Deductive-nomological reasoning suggests a mechanical view of theworldinwhichthesamestimulusproducesthesameeffectceterispa-ribusTheoriesthatviewsocialphenomenaasproductsofeitherevolu-tionaryprocessesor intentionalactionchallenge thismechanicalviewBothevolutionaryandintentionaltheoriesassumethatindividualsand
Copyrighted Material
10 bull Chapter1
organizationsadjusttheirresponsestosocialconditions(Alchian1950Brady2004EOstrom2000Thelen2003)Intentionaltheoriesofhu-manbehaviorassumethatadaptationoccursaspeoplestruggletosolvepuzzlesrelatedtothepursuitoftheirgoals(AlmondandGenco1977Elster1983Knight1992)While some intentional theories emphasizeroutines and heuristics there is always a possibility for creativity andinnovation (March andOlsen1984 Simon1955)Evolutionary theo-riesdonotrequire intentionalitybutdorequiresomesortofselectionmechanismsuchasmarketorelectoralcompetitiontodriveadaptationBothformsofadaptationimplythatthesamecircumstanceswillgener-atediverse responsesacrossactorsandchanges in individualbehaviorover timebut thatadaptationswill reflecthistorical trajectoriesThusthesamestimuluswillnot producethesameeffectonaverageandcon-stanteffectscannotbeassumedBothperspectivesraisequestionsaboutthesuitabilityof researchmethods thatassumeconstanteffects (Elster1998Hall2003Ragin19872000)
ThechoiceofmethodtendstosignalonersquostheoreticalperspectiveasdoesthenatureofmethodologicalcritiqueThosewhodiscountquali-tativemethodsasincapableofevaluatinggeneralrelationshipssignalabelief inbothlawlikesocialrelationsandtherelativeunimportanceoffactorssuchasagencyhistoryandinformalcontextNotsurprisinglycritiquesofquantitativemethodsoftenchargethattheydonotcapturethemost important aspects of social conditionsLikewise thosewaryofformalmodelsworryaboutthelevelofabstractionHowcanformalmodelsadequatelyrepresentthedensenetworksofformalandinformalinstitutionsandculturalunderstandingsinwhichhumanactionoccursNoneof these critiques really concerns themethodasmethod ratherthey target the theoretical assumptions as reflected in methodologicalchoicesWhatvariablesareimportantWhatistherelativeimportanceofformalinstitutionsculturesocialstructureorinformalinstitutionsHow important aremass beliefs andbehavior or individual interestsbeliefsandstrategicactionHowarethosevariablesrelatedWhilethebehavioralrevolutionduringthemid-twentiethcenturycertainlyfosteredtherapidspreadofquantitativeanalysisitalsoredirectedtheoreticalem-phasisfromformalinstitutionstothebehaviorandattitudesofindividu-als interactingwithinboth formal and informal institutionsLikewiserational-choiceanalysisoftenreliesongametheoryandothervarietiesof formal modeling but is defined by assumptions of methodologicalindividualismandintentionalaction
Yettheinfluenceoftheorymdashandtheimpliedinfluenceofontologymdashonmethodologicalpracticecannotbeassumedandshouldnotbeover-stated6 Theoreticalchangescananddooccurindependentlyofchangesinmethodologicalpractice(Hall2003Platt1986)Sometimesmethod-
Copyrighted Material
OvercomingChallenges bull 11
ologicalchallengesseemtodrivetheoreticalargumentsratherthantheotherwayaround(Lieberson1991318)Indeedsophisticatedmethodssometimescrowdouttheoryaltogether(Achen20022005)WearguethatmethodologicalchoicesareoftendrivenasmuchbydataavailabilityorcareerincentivesWhencareersurvivalisatstakepracticalconsider-ationscansqueezeoutconcernsaboutmatchingtheoryandmethodThelinkbetweenmethodsandcareerprospectscanhoweverbeexpectedtoinfluencethetenorofmethodologicalmdashandtheoreticalmdashdebates
SometimesmethodologicalandtheoreticaldebatestakeonexistentialovertonesWhenaparticulartheoryandassociatedmethodsbecomeex-tremelywidespreadforexampleproponentsofalternativeapproachesmayworryabouttheirownacademicsurvivalProponentsofnewtheo-riesmdashandassociatedmethodsmdashalsofaceanexistentialfightforrecogni-tionandsurvivalThedegreeof(perceived)existentialthreatdependsonthe extent to which fellowships job opportunities publishing outletsandresearchgrantsareopen(orclosed)todiversetheoriesandmethodsTherecentperestroikamovementinpoliticalscienceforexamplepre-sented itself asdefendingagainstmethodologicalhegemony conjuringimages of political scientists conspiring to control journal outlets andprofessionalassociations7ThiswasnotsimplyamethodologicalcritiquebutacalltoactionagainstpresumedtyrannyWhiletheinflammatorypublic languageassociatedwiththeperestroikamovementmaybeun-usualthelayeringofmethodologicaldebateswithvaluejudgmentsandcompetitionforprofessionalrecognitionandresourcesiscommonplace
Despite references to ldquorevolutionsrdquo and paradigm shifts new socialsciencetheoriesandmethodshavenotfullydisplacedtheirpredecessorsRathereachnewtheoryandmethodhasaddedanotherstrandConstruc-tivists institutionalists andpostmodernists coexistwithbehavioralistsandstructuralistsDespitethehistoryoftheoreticalandmethodologicalcompetitionandcritiquescholarsalsoengageincreativesynthesisThecurrentappreciationformethodologicalpluralismmaybeinterpretedasaproductofthesurvivalandadaptationofapproachesthatwereonceperceived to be under existential threat Promotion of methodologicalpluralismfavorsatheoreticaleclecticismthatshoulddecreaseconcernsaboutexistential threats toparticularapproachesandshouldthusde-creasetheintensityofmethodologicaldebates
Multiple Methods Promises and Challenges
Therearemanyreasonsforsocialscientiststowelcomemethodologicalpluralism and greater use of mixed methods No single method over-comes all challenges Case studies and small-N comparative research
Copyrighted Material
12 bull Chapter1
designs offer advantages for concept and theory development as wellasevaluationofhypothesizedcausalsequencesandmechanisms(Bates2008Batesetal1998CollierBradyandSeawright2004Coppedge1999 Lieberman 2005) Rich explanations of particular cases are of-tenvaluablesubstantivelyandtheoretically(MahoneyandGoertz2006Rogowski2004)Yetasiswidelyrecognizedsmall-Nstudiesofferanuncertainfoundationforpositingorevaluatinggeneralrelationships
Formalmethodsseektobuild logicallycoherentmodelsanddiscerntheir logical implicationsTheir emphasison logical consistency facili-tatesthedistillationofparsimoniousyetgeneralhypothesesandguidesthechoiceofstatisticaltechniques(Achen20022005Batesetal1998GranatoandScioli2004)8 Thehighlevelofabstractioninformalmod-elshoweverraisesquestionsabouttheirempiricalapplicability(GreenandShapiro1994)Thecontrolledconditionsinexperimentalresearchprovidegreaterconfidenceintheinternalvalidityofobservedrelation-shipsTheexternalvalidityofgeneralrelationshipscanbestbeevaluatedhoweverthroughanalysisofalargenumberofnonexperimentalobser-vations(Goldthorpe1997KingKeohaneandVerba1994)aswellasthroughfieldexperiments(seeCardenas2003CardenasStranlundandWillis2000Henrichetal2004List2004)
Small-Nqualitativestudiescansuggesttheplausibilityofformalmod-elsbutprovidelittleleverageinassessingthegeneralityofrelationshipsThebroad comparisons required to evaluate thegeneralityofhypoth-esizedrelationshipsdemandsomeformofquantitativeanalysisWherequantitativeanalysisoncemeantregression-basedanalysisoptions forquantitativeanalysisofempiricalsocialsciencedatanowincludeQuali-tative Comparative Analysis (QCA) and fuzzy-set Qualitative Com-parativeAnalysis(fsQCA)(Ragin19872000)aswellasprobabilisticlikelihood-basedandBayesianstatistics(Gill2004)9 Thismethodologi-calmenuincludesoptionsforscholarswhoholdvariedontologicalas-sumptionsaboutthesocialworld
MixedmethodstakeavarietyofformsAresearchermightusedifferentmethodstoaddressdifferentresearchquestionsorcontextsOrdifferentmethodsmightguidedifferentstagesofaresearchprogram(Lieberman2005)IncreasinglyscholarsstrivetousetwoormoremethodsateachstageofresearchThoseconcernedwithgeneralcausalpatternsdrawonquantitativeandqualitativemethods(Coppedge1999Lieberman2005Tarrow2004)Combinationsofformalandqualitativemethodsaddressconcernsaboutlogicalcoherenceandcausalprocessesincontingentrela-tionshipswherethereisnoexpectationofgenerality(Batesetal1998)Otherscontendthatscholarsshouldseeklogicalcoherenceandevidencefor causalprocesses and should test for thegeneralityof relationsbydrawingonformalqualitativeandquantitativemethods(Granatoand
Copyrighted Material
OvercomingChallenges bull 13
Scioli2004Laitin2003)Scholarswhodevelopagent-basedmodelsuserolegamesandexperiments tocollectdataaswellas involving stake-holdersinthevalidationoftheirmodels(BarreteauLePageandAquino2003Bousquetetal2002GurungBousquetandTreacutebuil2006)Otherscholars combine their formal models with ethnographic observations(Bharwanietal2005HuigenOvermarsanddeGroot2006)
Theuseofmultiplemethodshoweverdoesnotguaranteemethod-ologicallysuperiorsocialscienceresearchSomequestiontheextenttowhichformalqualitativeandquantitativeresearchmethodsareactuallycomplementary Several recent publications have argued that differentmethodsreflectdifferentassumptionsaboutthenatureofcausalityandhavecalledforgreatercareinmatchingmethodstoontologicalassump-tions(BennettandElman2006ClarkGilliganandGolder2006Hall2003Mahoney2003Ragin19872000)
Therearealso limits to the feasibilityofmultimethodresearchHy-potheses about complex causal relationships imply complex statisticalmodelsthatstretchthelimitsofavailabledataStatisticalanalysesoftenadd interaction termsordummyvariables tomodel contingent effectsandmultiplecausalpaths(ClarkGilliganandGolder2006Pontusson2007)but theseadditionalvariablesconsumedegreesof freedominacontextoflimiteddataavailability(Shalev2007)Othertechniquesde-velopedtoaddresscausalcomplexitysuchastheanalysisoftime-series-cross-sectional data and hierarchical models may strain the technicalskillsofboththeresearcherandtheaudience(Shalev2007)
MahoneyandGoertz(2006)contendthatinteractioneffectsdummyvariableshierarchicalmodelsandothersimilarstatisticalfixesdonotaccuratelyreflecttherelationshipspositedintheunderlyingtheoriesTheassumptionthatobservationsareindependentforexampleiscalledintoquestionbyglobalizationdiffusioneffectsandactor-centeredtheoriesthatemphasizestrategicinteractionsEvensomequantitativelyorientedscholarsquestiontheappropriatenessofstandardstatisticaltechniquesInrecentyearsnewtechniqueshavebeenproposedtoincorporateinter-dependence(Signorino1999)Bayesianstatistics(Dion1998Gill2004)andBooleanlogic(Braumoeller2003Ragin19872000)Theverdictisstilloutonwhetherthesenewtechniquesmatchunderlyingassumptionsbetterthandoesregression-basedstatistics
Too often the development of ever-more sophisticated techniquesseemstobeanendinitselfThelatesttechniquesaresometimesadoptedwithlittlereferencetotheoreticalconsiderationsorunderstandingoftheunderlyingassumptionsButmethodologicalsophisticationcannotsub-stitutefortheoryAchen(20022005)warnsthatquantitativeanalysesthat are not supported by theoretical microfoundations or careful ex-plorationofthedatayieldunreliableresultsandshouldnotbetrusted
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
8 bull Chapter1
drivenbytheoreticalandontologicalassumptions(Hall2003)buttheyalso reflectunderlying values andbeliefs (MahoneyandGoertz2006)andpractical considerations (Platt1986)Theontologicalandnorma-tive dimensions of methodological choices are not widely recognized(MahoneyandGoertz2006)Asa result social sciencedebatesaboutmethodsinvolvefrequentmisunderstandingswithproponentsofdiffer-entapproachestalkingpasteachother(EOstrom2006)Furthermorebecause methodological discussions rarely acknowledge practical andprofessionalconsiderationstheyofferlittleguidanceonhowtoaddresstheseconstraintsInthissectionwediscusscontroversiesoverthegoalsofsocialresearchandhowphilosophicalandtheoreticalissuesinteractwithprofessionalcompetitionWeexpandourtreatmentofpracticalandprofessionalconsiderationsinsubsequentsections
During the1920sand1930s the social sciencesbecamemore insti-tutionalized in North America The social sciences sought recognitionassciencesandeachdisciplinedevelopedamoreorlessdistinctprofes-sionalidentity(Guy2003Platt1986)ThisprocessofinstitutionalizationinfluencedmethodologicaldebatesDuringtheprewarperioddisagree-ments focused on the goals of social research Should sociological re-searchsupportsocialworktoimprovesocialconditionsseeksubjectiveunderstandingoflifeexperiencesorattempttoidentifygeneralpatterns(Platt1986)Shouldthestudyofpoliticsprovidenormativeandpracti-cal guidance for administratorsorobjectiveunderstandingofpoliticalphenomena(Guy2003Lasswell1951)Asuniversitiessetupschoolsofsocialworkpublicadministrationandbusinessadministrationalong-sidedepartmentsof sociologypolitical scienceandeconomicsdiffer-encesovergoalswerealleviatedmdashbutnotreallyaddressedmdashthroughtheinstitutionalizationofmorefocusedprogramsofstudy
Yetdifferencesovertherelativeimportanceoftheoryandpraxiscan-not fully account for methodological debates Scholars with commongoalsdisagreeovermethodsandscholarsdrawonthesamemethodstopursuedivergentgoalsAlackofconsensusonfundamentalphilosophi-cal issues contributes to disagreements over methods What counts asscienceWhatmodelormodelsofcausalityandexplanationmakesenseforsocialphenomenaInparticulardomodelsofscienceandexplana-tiondevelopedinthenaturalandespeciallythephysicalsciencesmakesenseforthesocialsciences
Overthepastcenturysomehaveembraceddeductivemodelsofscienceinspiredbythenaturalsciencesasawaytogainmorereliableinsightsaboutsocialprocesses(KingKeohaneandVerba1994PrzeworskiandTeune 1970) Deduction involves the logical derivation of universalis-ticlawlikestatementsofthesetsofconditionsassociatedwiththeout-comeofinterestfromtheoreticalassumptionsLawlikestatementsmay
Copyrighted Material
OvercomingChallenges bull 9
be derived from formal or mathematical models as in rational-choiceapproachesorlogicalanalysisasinsomequalitativestudiesEmpiricalevaluationsrelyontheanalysisofcorrelationasinbehavioralresearchor paired comparisons4 The journal Public Choice devoted a specialissue in December 2008 to the topic ldquoHomo Economicus and HomoPoliticusrdquo(editedbyGeoffreyBrennanandMichaelGillespie)withninearticlesaddressingthequestionofhowtoreconcilethebasicdifferencesbetweentheoriesofhumanbehaviorineconomicsandpoliticalscienceIntheintroductionBrennan(2008431)reflectsthat
theambitiontofindcommongroundonwhichpublicchoicescholarsandldquopoliticaltheoristsrdquoofamoretraditionalkindmighthaveprofit-ableexchangeisnotatrivialonewestartfromverydifferentconcep-tionsofwhatcountsas theorymdashevenofwhatcountsasworthwhilescholarshipmdashandfromratherdifferentdisciplinarypresuppositionsastohowdifferences inapproachcanmostprofitablybeengagedandresolved
Criticshoweverarguethatdeductivemethodsdonotallowforhu-managencyandreflexivitytheinfluenceofmeaningandinterpretationorcontingentrelationships(AlmondandGenco1977Hall2003Ragin1987seereviewinPlatt1986)Ifagencyistakenseriouslywemustal-lowforbothcreativityanddifferencesinperspectivesButcreativityanddifferencesininterpretationmeanthatlawlikesocialpatternsareunlikelytoariseContingentrelationshipsarepossibleevenifquestionsofagencyareputasideThesedifferencesoverthenatureofcausalityhavefueledheatedmethodologicaldebatesInpoliticalscienceboththebehavioralrevolutionoftheearlypostwarperiodandtheriseofrational-choicethe-oryinthe1980sand1990sassumedthevalueofdeductive-nomologicalreasoningScholarswhousedmethodsthatreflectedalternativeontologi-calassumptionshaddifficultygainingrecognitionfortheirworkTheirfrustrationgaverisetotherecentperestroikamovementinwhichcon-structivistsandotherschallengedboththeuniversalityofsocialpatternsassumedbyrationalchoiceandbehavioraltheoriesandthedominanceofstatisticalandformalmethodsassociatedwiththeseapproachesintheprofession(Monroe2005)Withineconomicstheconcernthatnarrowrational-choicemodelshavecometodominatemuchofeconomicschol-arshipisregularlyexpressedintheonlinejournalReal-World Economics Review5
Deductive-nomological reasoning suggests a mechanical view of theworldinwhichthesamestimulusproducesthesameeffectceterispa-ribusTheoriesthatviewsocialphenomenaasproductsofeitherevolu-tionaryprocessesor intentionalactionchallenge thismechanicalviewBothevolutionaryandintentionaltheoriesassumethatindividualsand
Copyrighted Material
10 bull Chapter1
organizationsadjusttheirresponsestosocialconditions(Alchian1950Brady2004EOstrom2000Thelen2003)Intentionaltheoriesofhu-manbehaviorassumethatadaptationoccursaspeoplestruggletosolvepuzzlesrelatedtothepursuitoftheirgoals(AlmondandGenco1977Elster1983Knight1992)While some intentional theories emphasizeroutines and heuristics there is always a possibility for creativity andinnovation (March andOlsen1984 Simon1955)Evolutionary theo-riesdonotrequire intentionalitybutdorequiresomesortofselectionmechanismsuchasmarketorelectoralcompetitiontodriveadaptationBothformsofadaptationimplythatthesamecircumstanceswillgener-atediverse responsesacrossactorsandchanges in individualbehaviorover timebut thatadaptationswill reflecthistorical trajectoriesThusthesamestimuluswillnot producethesameeffectonaverageandcon-stanteffectscannotbeassumedBothperspectivesraisequestionsaboutthesuitabilityof researchmethods thatassumeconstanteffects (Elster1998Hall2003Ragin19872000)
ThechoiceofmethodtendstosignalonersquostheoreticalperspectiveasdoesthenatureofmethodologicalcritiqueThosewhodiscountquali-tativemethodsasincapableofevaluatinggeneralrelationshipssignalabelief inbothlawlikesocialrelationsandtherelativeunimportanceoffactorssuchasagencyhistoryandinformalcontextNotsurprisinglycritiquesofquantitativemethodsoftenchargethattheydonotcapturethemost important aspects of social conditionsLikewise thosewaryofformalmodelsworryaboutthelevelofabstractionHowcanformalmodelsadequatelyrepresentthedensenetworksofformalandinformalinstitutionsandculturalunderstandingsinwhichhumanactionoccursNoneof these critiques really concerns themethodasmethod ratherthey target the theoretical assumptions as reflected in methodologicalchoicesWhatvariablesareimportantWhatistherelativeimportanceofformalinstitutionsculturesocialstructureorinformalinstitutionsHow important aremass beliefs andbehavior or individual interestsbeliefsandstrategicactionHowarethosevariablesrelatedWhilethebehavioralrevolutionduringthemid-twentiethcenturycertainlyfosteredtherapidspreadofquantitativeanalysisitalsoredirectedtheoreticalem-phasisfromformalinstitutionstothebehaviorandattitudesofindividu-als interactingwithinboth formal and informal institutionsLikewiserational-choiceanalysisoftenreliesongametheoryandothervarietiesof formal modeling but is defined by assumptions of methodologicalindividualismandintentionalaction
Yettheinfluenceoftheorymdashandtheimpliedinfluenceofontologymdashonmethodologicalpracticecannotbeassumedandshouldnotbeover-stated6 Theoreticalchangescananddooccurindependentlyofchangesinmethodologicalpractice(Hall2003Platt1986)Sometimesmethod-
Copyrighted Material
OvercomingChallenges bull 11
ologicalchallengesseemtodrivetheoreticalargumentsratherthantheotherwayaround(Lieberson1991318)Indeedsophisticatedmethodssometimescrowdouttheoryaltogether(Achen20022005)WearguethatmethodologicalchoicesareoftendrivenasmuchbydataavailabilityorcareerincentivesWhencareersurvivalisatstakepracticalconsider-ationscansqueezeoutconcernsaboutmatchingtheoryandmethodThelinkbetweenmethodsandcareerprospectscanhoweverbeexpectedtoinfluencethetenorofmethodologicalmdashandtheoreticalmdashdebates
SometimesmethodologicalandtheoreticaldebatestakeonexistentialovertonesWhenaparticulartheoryandassociatedmethodsbecomeex-tremelywidespreadforexampleproponentsofalternativeapproachesmayworryabouttheirownacademicsurvivalProponentsofnewtheo-riesmdashandassociatedmethodsmdashalsofaceanexistentialfightforrecogni-tionandsurvivalThedegreeof(perceived)existentialthreatdependsonthe extent to which fellowships job opportunities publishing outletsandresearchgrantsareopen(orclosed)todiversetheoriesandmethodsTherecentperestroikamovementinpoliticalscienceforexamplepre-sented itself asdefendingagainstmethodologicalhegemony conjuringimages of political scientists conspiring to control journal outlets andprofessionalassociations7ThiswasnotsimplyamethodologicalcritiquebutacalltoactionagainstpresumedtyrannyWhiletheinflammatorypublic languageassociatedwiththeperestroikamovementmaybeun-usualthelayeringofmethodologicaldebateswithvaluejudgmentsandcompetitionforprofessionalrecognitionandresourcesiscommonplace
Despite references to ldquorevolutionsrdquo and paradigm shifts new socialsciencetheoriesandmethodshavenotfullydisplacedtheirpredecessorsRathereachnewtheoryandmethodhasaddedanotherstrandConstruc-tivists institutionalists andpostmodernists coexistwithbehavioralistsandstructuralistsDespitethehistoryoftheoreticalandmethodologicalcompetitionandcritiquescholarsalsoengageincreativesynthesisThecurrentappreciationformethodologicalpluralismmaybeinterpretedasaproductofthesurvivalandadaptationofapproachesthatwereonceperceived to be under existential threat Promotion of methodologicalpluralismfavorsatheoreticaleclecticismthatshoulddecreaseconcernsaboutexistential threats toparticularapproachesandshouldthusde-creasetheintensityofmethodologicaldebates
Multiple Methods Promises and Challenges
Therearemanyreasonsforsocialscientiststowelcomemethodologicalpluralism and greater use of mixed methods No single method over-comes all challenges Case studies and small-N comparative research
Copyrighted Material
12 bull Chapter1
designs offer advantages for concept and theory development as wellasevaluationofhypothesizedcausalsequencesandmechanisms(Bates2008Batesetal1998CollierBradyandSeawright2004Coppedge1999 Lieberman 2005) Rich explanations of particular cases are of-tenvaluablesubstantivelyandtheoretically(MahoneyandGoertz2006Rogowski2004)Yetasiswidelyrecognizedsmall-Nstudiesofferanuncertainfoundationforpositingorevaluatinggeneralrelationships
Formalmethodsseektobuild logicallycoherentmodelsanddiscerntheir logical implicationsTheir emphasison logical consistency facili-tatesthedistillationofparsimoniousyetgeneralhypothesesandguidesthechoiceofstatisticaltechniques(Achen20022005Batesetal1998GranatoandScioli2004)8 Thehighlevelofabstractioninformalmod-elshoweverraisesquestionsabouttheirempiricalapplicability(GreenandShapiro1994)Thecontrolledconditionsinexperimentalresearchprovidegreaterconfidenceintheinternalvalidityofobservedrelation-shipsTheexternalvalidityofgeneralrelationshipscanbestbeevaluatedhoweverthroughanalysisofalargenumberofnonexperimentalobser-vations(Goldthorpe1997KingKeohaneandVerba1994)aswellasthroughfieldexperiments(seeCardenas2003CardenasStranlundandWillis2000Henrichetal2004List2004)
Small-Nqualitativestudiescansuggesttheplausibilityofformalmod-elsbutprovidelittleleverageinassessingthegeneralityofrelationshipsThebroad comparisons required to evaluate thegeneralityofhypoth-esizedrelationshipsdemandsomeformofquantitativeanalysisWherequantitativeanalysisoncemeantregression-basedanalysisoptions forquantitativeanalysisofempiricalsocialsciencedatanowincludeQuali-tative Comparative Analysis (QCA) and fuzzy-set Qualitative Com-parativeAnalysis(fsQCA)(Ragin19872000)aswellasprobabilisticlikelihood-basedandBayesianstatistics(Gill2004)9 Thismethodologi-calmenuincludesoptionsforscholarswhoholdvariedontologicalas-sumptionsaboutthesocialworld
MixedmethodstakeavarietyofformsAresearchermightusedifferentmethodstoaddressdifferentresearchquestionsorcontextsOrdifferentmethodsmightguidedifferentstagesofaresearchprogram(Lieberman2005)IncreasinglyscholarsstrivetousetwoormoremethodsateachstageofresearchThoseconcernedwithgeneralcausalpatternsdrawonquantitativeandqualitativemethods(Coppedge1999Lieberman2005Tarrow2004)Combinationsofformalandqualitativemethodsaddressconcernsaboutlogicalcoherenceandcausalprocessesincontingentrela-tionshipswherethereisnoexpectationofgenerality(Batesetal1998)Otherscontendthatscholarsshouldseeklogicalcoherenceandevidencefor causalprocesses and should test for thegeneralityof relationsbydrawingonformalqualitativeandquantitativemethods(Granatoand
Copyrighted Material
OvercomingChallenges bull 13
Scioli2004Laitin2003)Scholarswhodevelopagent-basedmodelsuserolegamesandexperiments tocollectdataaswellas involving stake-holdersinthevalidationoftheirmodels(BarreteauLePageandAquino2003Bousquetetal2002GurungBousquetandTreacutebuil2006)Otherscholars combine their formal models with ethnographic observations(Bharwanietal2005HuigenOvermarsanddeGroot2006)
Theuseofmultiplemethodshoweverdoesnotguaranteemethod-ologicallysuperiorsocialscienceresearchSomequestiontheextenttowhichformalqualitativeandquantitativeresearchmethodsareactuallycomplementary Several recent publications have argued that differentmethodsreflectdifferentassumptionsaboutthenatureofcausalityandhavecalledforgreatercareinmatchingmethodstoontologicalassump-tions(BennettandElman2006ClarkGilliganandGolder2006Hall2003Mahoney2003Ragin19872000)
Therearealso limits to the feasibilityofmultimethodresearchHy-potheses about complex causal relationships imply complex statisticalmodelsthatstretchthelimitsofavailabledataStatisticalanalysesoftenadd interaction termsordummyvariables tomodel contingent effectsandmultiplecausalpaths(ClarkGilliganandGolder2006Pontusson2007)but theseadditionalvariablesconsumedegreesof freedominacontextoflimiteddataavailability(Shalev2007)Othertechniquesde-velopedtoaddresscausalcomplexitysuchastheanalysisoftime-series-cross-sectional data and hierarchical models may strain the technicalskillsofboththeresearcherandtheaudience(Shalev2007)
MahoneyandGoertz(2006)contendthatinteractioneffectsdummyvariableshierarchicalmodelsandothersimilarstatisticalfixesdonotaccuratelyreflecttherelationshipspositedintheunderlyingtheoriesTheassumptionthatobservationsareindependentforexampleiscalledintoquestionbyglobalizationdiffusioneffectsandactor-centeredtheoriesthatemphasizestrategicinteractionsEvensomequantitativelyorientedscholarsquestiontheappropriatenessofstandardstatisticaltechniquesInrecentyearsnewtechniqueshavebeenproposedtoincorporateinter-dependence(Signorino1999)Bayesianstatistics(Dion1998Gill2004)andBooleanlogic(Braumoeller2003Ragin19872000)Theverdictisstilloutonwhetherthesenewtechniquesmatchunderlyingassumptionsbetterthandoesregression-basedstatistics
Too often the development of ever-more sophisticated techniquesseemstobeanendinitselfThelatesttechniquesaresometimesadoptedwithlittlereferencetotheoreticalconsiderationsorunderstandingoftheunderlyingassumptionsButmethodologicalsophisticationcannotsub-stitutefortheoryAchen(20022005)warnsthatquantitativeanalysesthat are not supported by theoretical microfoundations or careful ex-plorationofthedatayieldunreliableresultsandshouldnotbetrusted
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 9
be derived from formal or mathematical models as in rational-choiceapproachesorlogicalanalysisasinsomequalitativestudiesEmpiricalevaluationsrelyontheanalysisofcorrelationasinbehavioralresearchor paired comparisons4 The journal Public Choice devoted a specialissue in December 2008 to the topic ldquoHomo Economicus and HomoPoliticusrdquo(editedbyGeoffreyBrennanandMichaelGillespie)withninearticlesaddressingthequestionofhowtoreconcilethebasicdifferencesbetweentheoriesofhumanbehaviorineconomicsandpoliticalscienceIntheintroductionBrennan(2008431)reflectsthat
theambitiontofindcommongroundonwhichpublicchoicescholarsandldquopoliticaltheoristsrdquoofamoretraditionalkindmighthaveprofit-ableexchangeisnotatrivialonewestartfromverydifferentconcep-tionsofwhatcountsas theorymdashevenofwhatcountsasworthwhilescholarshipmdashandfromratherdifferentdisciplinarypresuppositionsastohowdifferences inapproachcanmostprofitablybeengagedandresolved
Criticshoweverarguethatdeductivemethodsdonotallowforhu-managencyandreflexivitytheinfluenceofmeaningandinterpretationorcontingentrelationships(AlmondandGenco1977Hall2003Ragin1987seereviewinPlatt1986)Ifagencyistakenseriouslywemustal-lowforbothcreativityanddifferencesinperspectivesButcreativityanddifferencesininterpretationmeanthatlawlikesocialpatternsareunlikelytoariseContingentrelationshipsarepossibleevenifquestionsofagencyareputasideThesedifferencesoverthenatureofcausalityhavefueledheatedmethodologicaldebatesInpoliticalscienceboththebehavioralrevolutionoftheearlypostwarperiodandtheriseofrational-choicethe-oryinthe1980sand1990sassumedthevalueofdeductive-nomologicalreasoningScholarswhousedmethodsthatreflectedalternativeontologi-calassumptionshaddifficultygainingrecognitionfortheirworkTheirfrustrationgaverisetotherecentperestroikamovementinwhichcon-structivistsandotherschallengedboththeuniversalityofsocialpatternsassumedbyrationalchoiceandbehavioraltheoriesandthedominanceofstatisticalandformalmethodsassociatedwiththeseapproachesintheprofession(Monroe2005)Withineconomicstheconcernthatnarrowrational-choicemodelshavecometodominatemuchofeconomicschol-arshipisregularlyexpressedintheonlinejournalReal-World Economics Review5
Deductive-nomological reasoning suggests a mechanical view of theworldinwhichthesamestimulusproducesthesameeffectceterispa-ribusTheoriesthatviewsocialphenomenaasproductsofeitherevolu-tionaryprocessesor intentionalactionchallenge thismechanicalviewBothevolutionaryandintentionaltheoriesassumethatindividualsand
Copyrighted Material
10 bull Chapter1
organizationsadjusttheirresponsestosocialconditions(Alchian1950Brady2004EOstrom2000Thelen2003)Intentionaltheoriesofhu-manbehaviorassumethatadaptationoccursaspeoplestruggletosolvepuzzlesrelatedtothepursuitoftheirgoals(AlmondandGenco1977Elster1983Knight1992)While some intentional theories emphasizeroutines and heuristics there is always a possibility for creativity andinnovation (March andOlsen1984 Simon1955)Evolutionary theo-riesdonotrequire intentionalitybutdorequiresomesortofselectionmechanismsuchasmarketorelectoralcompetitiontodriveadaptationBothformsofadaptationimplythatthesamecircumstanceswillgener-atediverse responsesacrossactorsandchanges in individualbehaviorover timebut thatadaptationswill reflecthistorical trajectoriesThusthesamestimuluswillnot producethesameeffectonaverageandcon-stanteffectscannotbeassumedBothperspectivesraisequestionsaboutthesuitabilityof researchmethods thatassumeconstanteffects (Elster1998Hall2003Ragin19872000)
ThechoiceofmethodtendstosignalonersquostheoreticalperspectiveasdoesthenatureofmethodologicalcritiqueThosewhodiscountquali-tativemethodsasincapableofevaluatinggeneralrelationshipssignalabelief inbothlawlikesocialrelationsandtherelativeunimportanceoffactorssuchasagencyhistoryandinformalcontextNotsurprisinglycritiquesofquantitativemethodsoftenchargethattheydonotcapturethemost important aspects of social conditionsLikewise thosewaryofformalmodelsworryaboutthelevelofabstractionHowcanformalmodelsadequatelyrepresentthedensenetworksofformalandinformalinstitutionsandculturalunderstandingsinwhichhumanactionoccursNoneof these critiques really concerns themethodasmethod ratherthey target the theoretical assumptions as reflected in methodologicalchoicesWhatvariablesareimportantWhatistherelativeimportanceofformalinstitutionsculturesocialstructureorinformalinstitutionsHow important aremass beliefs andbehavior or individual interestsbeliefsandstrategicactionHowarethosevariablesrelatedWhilethebehavioralrevolutionduringthemid-twentiethcenturycertainlyfosteredtherapidspreadofquantitativeanalysisitalsoredirectedtheoreticalem-phasisfromformalinstitutionstothebehaviorandattitudesofindividu-als interactingwithinboth formal and informal institutionsLikewiserational-choiceanalysisoftenreliesongametheoryandothervarietiesof formal modeling but is defined by assumptions of methodologicalindividualismandintentionalaction
Yettheinfluenceoftheorymdashandtheimpliedinfluenceofontologymdashonmethodologicalpracticecannotbeassumedandshouldnotbeover-stated6 Theoreticalchangescananddooccurindependentlyofchangesinmethodologicalpractice(Hall2003Platt1986)Sometimesmethod-
Copyrighted Material
OvercomingChallenges bull 11
ologicalchallengesseemtodrivetheoreticalargumentsratherthantheotherwayaround(Lieberson1991318)Indeedsophisticatedmethodssometimescrowdouttheoryaltogether(Achen20022005)WearguethatmethodologicalchoicesareoftendrivenasmuchbydataavailabilityorcareerincentivesWhencareersurvivalisatstakepracticalconsider-ationscansqueezeoutconcernsaboutmatchingtheoryandmethodThelinkbetweenmethodsandcareerprospectscanhoweverbeexpectedtoinfluencethetenorofmethodologicalmdashandtheoreticalmdashdebates
SometimesmethodologicalandtheoreticaldebatestakeonexistentialovertonesWhenaparticulartheoryandassociatedmethodsbecomeex-tremelywidespreadforexampleproponentsofalternativeapproachesmayworryabouttheirownacademicsurvivalProponentsofnewtheo-riesmdashandassociatedmethodsmdashalsofaceanexistentialfightforrecogni-tionandsurvivalThedegreeof(perceived)existentialthreatdependsonthe extent to which fellowships job opportunities publishing outletsandresearchgrantsareopen(orclosed)todiversetheoriesandmethodsTherecentperestroikamovementinpoliticalscienceforexamplepre-sented itself asdefendingagainstmethodologicalhegemony conjuringimages of political scientists conspiring to control journal outlets andprofessionalassociations7ThiswasnotsimplyamethodologicalcritiquebutacalltoactionagainstpresumedtyrannyWhiletheinflammatorypublic languageassociatedwiththeperestroikamovementmaybeun-usualthelayeringofmethodologicaldebateswithvaluejudgmentsandcompetitionforprofessionalrecognitionandresourcesiscommonplace
Despite references to ldquorevolutionsrdquo and paradigm shifts new socialsciencetheoriesandmethodshavenotfullydisplacedtheirpredecessorsRathereachnewtheoryandmethodhasaddedanotherstrandConstruc-tivists institutionalists andpostmodernists coexistwithbehavioralistsandstructuralistsDespitethehistoryoftheoreticalandmethodologicalcompetitionandcritiquescholarsalsoengageincreativesynthesisThecurrentappreciationformethodologicalpluralismmaybeinterpretedasaproductofthesurvivalandadaptationofapproachesthatwereonceperceived to be under existential threat Promotion of methodologicalpluralismfavorsatheoreticaleclecticismthatshoulddecreaseconcernsaboutexistential threats toparticularapproachesandshouldthusde-creasetheintensityofmethodologicaldebates
Multiple Methods Promises and Challenges
Therearemanyreasonsforsocialscientiststowelcomemethodologicalpluralism and greater use of mixed methods No single method over-comes all challenges Case studies and small-N comparative research
Copyrighted Material
12 bull Chapter1
designs offer advantages for concept and theory development as wellasevaluationofhypothesizedcausalsequencesandmechanisms(Bates2008Batesetal1998CollierBradyandSeawright2004Coppedge1999 Lieberman 2005) Rich explanations of particular cases are of-tenvaluablesubstantivelyandtheoretically(MahoneyandGoertz2006Rogowski2004)Yetasiswidelyrecognizedsmall-Nstudiesofferanuncertainfoundationforpositingorevaluatinggeneralrelationships
Formalmethodsseektobuild logicallycoherentmodelsanddiscerntheir logical implicationsTheir emphasison logical consistency facili-tatesthedistillationofparsimoniousyetgeneralhypothesesandguidesthechoiceofstatisticaltechniques(Achen20022005Batesetal1998GranatoandScioli2004)8 Thehighlevelofabstractioninformalmod-elshoweverraisesquestionsabouttheirempiricalapplicability(GreenandShapiro1994)Thecontrolledconditionsinexperimentalresearchprovidegreaterconfidenceintheinternalvalidityofobservedrelation-shipsTheexternalvalidityofgeneralrelationshipscanbestbeevaluatedhoweverthroughanalysisofalargenumberofnonexperimentalobser-vations(Goldthorpe1997KingKeohaneandVerba1994)aswellasthroughfieldexperiments(seeCardenas2003CardenasStranlundandWillis2000Henrichetal2004List2004)
Small-Nqualitativestudiescansuggesttheplausibilityofformalmod-elsbutprovidelittleleverageinassessingthegeneralityofrelationshipsThebroad comparisons required to evaluate thegeneralityofhypoth-esizedrelationshipsdemandsomeformofquantitativeanalysisWherequantitativeanalysisoncemeantregression-basedanalysisoptions forquantitativeanalysisofempiricalsocialsciencedatanowincludeQuali-tative Comparative Analysis (QCA) and fuzzy-set Qualitative Com-parativeAnalysis(fsQCA)(Ragin19872000)aswellasprobabilisticlikelihood-basedandBayesianstatistics(Gill2004)9 Thismethodologi-calmenuincludesoptionsforscholarswhoholdvariedontologicalas-sumptionsaboutthesocialworld
MixedmethodstakeavarietyofformsAresearchermightusedifferentmethodstoaddressdifferentresearchquestionsorcontextsOrdifferentmethodsmightguidedifferentstagesofaresearchprogram(Lieberman2005)IncreasinglyscholarsstrivetousetwoormoremethodsateachstageofresearchThoseconcernedwithgeneralcausalpatternsdrawonquantitativeandqualitativemethods(Coppedge1999Lieberman2005Tarrow2004)Combinationsofformalandqualitativemethodsaddressconcernsaboutlogicalcoherenceandcausalprocessesincontingentrela-tionshipswherethereisnoexpectationofgenerality(Batesetal1998)Otherscontendthatscholarsshouldseeklogicalcoherenceandevidencefor causalprocesses and should test for thegeneralityof relationsbydrawingonformalqualitativeandquantitativemethods(Granatoand
Copyrighted Material
OvercomingChallenges bull 13
Scioli2004Laitin2003)Scholarswhodevelopagent-basedmodelsuserolegamesandexperiments tocollectdataaswellas involving stake-holdersinthevalidationoftheirmodels(BarreteauLePageandAquino2003Bousquetetal2002GurungBousquetandTreacutebuil2006)Otherscholars combine their formal models with ethnographic observations(Bharwanietal2005HuigenOvermarsanddeGroot2006)
Theuseofmultiplemethodshoweverdoesnotguaranteemethod-ologicallysuperiorsocialscienceresearchSomequestiontheextenttowhichformalqualitativeandquantitativeresearchmethodsareactuallycomplementary Several recent publications have argued that differentmethodsreflectdifferentassumptionsaboutthenatureofcausalityandhavecalledforgreatercareinmatchingmethodstoontologicalassump-tions(BennettandElman2006ClarkGilliganandGolder2006Hall2003Mahoney2003Ragin19872000)
Therearealso limits to the feasibilityofmultimethodresearchHy-potheses about complex causal relationships imply complex statisticalmodelsthatstretchthelimitsofavailabledataStatisticalanalysesoftenadd interaction termsordummyvariables tomodel contingent effectsandmultiplecausalpaths(ClarkGilliganandGolder2006Pontusson2007)but theseadditionalvariablesconsumedegreesof freedominacontextoflimiteddataavailability(Shalev2007)Othertechniquesde-velopedtoaddresscausalcomplexitysuchastheanalysisoftime-series-cross-sectional data and hierarchical models may strain the technicalskillsofboththeresearcherandtheaudience(Shalev2007)
MahoneyandGoertz(2006)contendthatinteractioneffectsdummyvariableshierarchicalmodelsandothersimilarstatisticalfixesdonotaccuratelyreflecttherelationshipspositedintheunderlyingtheoriesTheassumptionthatobservationsareindependentforexampleiscalledintoquestionbyglobalizationdiffusioneffectsandactor-centeredtheoriesthatemphasizestrategicinteractionsEvensomequantitativelyorientedscholarsquestiontheappropriatenessofstandardstatisticaltechniquesInrecentyearsnewtechniqueshavebeenproposedtoincorporateinter-dependence(Signorino1999)Bayesianstatistics(Dion1998Gill2004)andBooleanlogic(Braumoeller2003Ragin19872000)Theverdictisstilloutonwhetherthesenewtechniquesmatchunderlyingassumptionsbetterthandoesregression-basedstatistics
Too often the development of ever-more sophisticated techniquesseemstobeanendinitselfThelatesttechniquesaresometimesadoptedwithlittlereferencetotheoreticalconsiderationsorunderstandingoftheunderlyingassumptionsButmethodologicalsophisticationcannotsub-stitutefortheoryAchen(20022005)warnsthatquantitativeanalysesthat are not supported by theoretical microfoundations or careful ex-plorationofthedatayieldunreliableresultsandshouldnotbetrusted
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
10 bull Chapter1
organizationsadjusttheirresponsestosocialconditions(Alchian1950Brady2004EOstrom2000Thelen2003)Intentionaltheoriesofhu-manbehaviorassumethatadaptationoccursaspeoplestruggletosolvepuzzlesrelatedtothepursuitoftheirgoals(AlmondandGenco1977Elster1983Knight1992)While some intentional theories emphasizeroutines and heuristics there is always a possibility for creativity andinnovation (March andOlsen1984 Simon1955)Evolutionary theo-riesdonotrequire intentionalitybutdorequiresomesortofselectionmechanismsuchasmarketorelectoralcompetitiontodriveadaptationBothformsofadaptationimplythatthesamecircumstanceswillgener-atediverse responsesacrossactorsandchanges in individualbehaviorover timebut thatadaptationswill reflecthistorical trajectoriesThusthesamestimuluswillnot producethesameeffectonaverageandcon-stanteffectscannotbeassumedBothperspectivesraisequestionsaboutthesuitabilityof researchmethods thatassumeconstanteffects (Elster1998Hall2003Ragin19872000)
ThechoiceofmethodtendstosignalonersquostheoreticalperspectiveasdoesthenatureofmethodologicalcritiqueThosewhodiscountquali-tativemethodsasincapableofevaluatinggeneralrelationshipssignalabelief inbothlawlikesocialrelationsandtherelativeunimportanceoffactorssuchasagencyhistoryandinformalcontextNotsurprisinglycritiquesofquantitativemethodsoftenchargethattheydonotcapturethemost important aspects of social conditionsLikewise thosewaryofformalmodelsworryaboutthelevelofabstractionHowcanformalmodelsadequatelyrepresentthedensenetworksofformalandinformalinstitutionsandculturalunderstandingsinwhichhumanactionoccursNoneof these critiques really concerns themethodasmethod ratherthey target the theoretical assumptions as reflected in methodologicalchoicesWhatvariablesareimportantWhatistherelativeimportanceofformalinstitutionsculturesocialstructureorinformalinstitutionsHow important aremass beliefs andbehavior or individual interestsbeliefsandstrategicactionHowarethosevariablesrelatedWhilethebehavioralrevolutionduringthemid-twentiethcenturycertainlyfosteredtherapidspreadofquantitativeanalysisitalsoredirectedtheoreticalem-phasisfromformalinstitutionstothebehaviorandattitudesofindividu-als interactingwithinboth formal and informal institutionsLikewiserational-choiceanalysisoftenreliesongametheoryandothervarietiesof formal modeling but is defined by assumptions of methodologicalindividualismandintentionalaction
Yettheinfluenceoftheorymdashandtheimpliedinfluenceofontologymdashonmethodologicalpracticecannotbeassumedandshouldnotbeover-stated6 Theoreticalchangescananddooccurindependentlyofchangesinmethodologicalpractice(Hall2003Platt1986)Sometimesmethod-
Copyrighted Material
OvercomingChallenges bull 11
ologicalchallengesseemtodrivetheoreticalargumentsratherthantheotherwayaround(Lieberson1991318)Indeedsophisticatedmethodssometimescrowdouttheoryaltogether(Achen20022005)WearguethatmethodologicalchoicesareoftendrivenasmuchbydataavailabilityorcareerincentivesWhencareersurvivalisatstakepracticalconsider-ationscansqueezeoutconcernsaboutmatchingtheoryandmethodThelinkbetweenmethodsandcareerprospectscanhoweverbeexpectedtoinfluencethetenorofmethodologicalmdashandtheoreticalmdashdebates
SometimesmethodologicalandtheoreticaldebatestakeonexistentialovertonesWhenaparticulartheoryandassociatedmethodsbecomeex-tremelywidespreadforexampleproponentsofalternativeapproachesmayworryabouttheirownacademicsurvivalProponentsofnewtheo-riesmdashandassociatedmethodsmdashalsofaceanexistentialfightforrecogni-tionandsurvivalThedegreeof(perceived)existentialthreatdependsonthe extent to which fellowships job opportunities publishing outletsandresearchgrantsareopen(orclosed)todiversetheoriesandmethodsTherecentperestroikamovementinpoliticalscienceforexamplepre-sented itself asdefendingagainstmethodologicalhegemony conjuringimages of political scientists conspiring to control journal outlets andprofessionalassociations7ThiswasnotsimplyamethodologicalcritiquebutacalltoactionagainstpresumedtyrannyWhiletheinflammatorypublic languageassociatedwiththeperestroikamovementmaybeun-usualthelayeringofmethodologicaldebateswithvaluejudgmentsandcompetitionforprofessionalrecognitionandresourcesiscommonplace
Despite references to ldquorevolutionsrdquo and paradigm shifts new socialsciencetheoriesandmethodshavenotfullydisplacedtheirpredecessorsRathereachnewtheoryandmethodhasaddedanotherstrandConstruc-tivists institutionalists andpostmodernists coexistwithbehavioralistsandstructuralistsDespitethehistoryoftheoreticalandmethodologicalcompetitionandcritiquescholarsalsoengageincreativesynthesisThecurrentappreciationformethodologicalpluralismmaybeinterpretedasaproductofthesurvivalandadaptationofapproachesthatwereonceperceived to be under existential threat Promotion of methodologicalpluralismfavorsatheoreticaleclecticismthatshoulddecreaseconcernsaboutexistential threats toparticularapproachesandshouldthusde-creasetheintensityofmethodologicaldebates
Multiple Methods Promises and Challenges
Therearemanyreasonsforsocialscientiststowelcomemethodologicalpluralism and greater use of mixed methods No single method over-comes all challenges Case studies and small-N comparative research
Copyrighted Material
12 bull Chapter1
designs offer advantages for concept and theory development as wellasevaluationofhypothesizedcausalsequencesandmechanisms(Bates2008Batesetal1998CollierBradyandSeawright2004Coppedge1999 Lieberman 2005) Rich explanations of particular cases are of-tenvaluablesubstantivelyandtheoretically(MahoneyandGoertz2006Rogowski2004)Yetasiswidelyrecognizedsmall-Nstudiesofferanuncertainfoundationforpositingorevaluatinggeneralrelationships
Formalmethodsseektobuild logicallycoherentmodelsanddiscerntheir logical implicationsTheir emphasison logical consistency facili-tatesthedistillationofparsimoniousyetgeneralhypothesesandguidesthechoiceofstatisticaltechniques(Achen20022005Batesetal1998GranatoandScioli2004)8 Thehighlevelofabstractioninformalmod-elshoweverraisesquestionsabouttheirempiricalapplicability(GreenandShapiro1994)Thecontrolledconditionsinexperimentalresearchprovidegreaterconfidenceintheinternalvalidityofobservedrelation-shipsTheexternalvalidityofgeneralrelationshipscanbestbeevaluatedhoweverthroughanalysisofalargenumberofnonexperimentalobser-vations(Goldthorpe1997KingKeohaneandVerba1994)aswellasthroughfieldexperiments(seeCardenas2003CardenasStranlundandWillis2000Henrichetal2004List2004)
Small-Nqualitativestudiescansuggesttheplausibilityofformalmod-elsbutprovidelittleleverageinassessingthegeneralityofrelationshipsThebroad comparisons required to evaluate thegeneralityofhypoth-esizedrelationshipsdemandsomeformofquantitativeanalysisWherequantitativeanalysisoncemeantregression-basedanalysisoptions forquantitativeanalysisofempiricalsocialsciencedatanowincludeQuali-tative Comparative Analysis (QCA) and fuzzy-set Qualitative Com-parativeAnalysis(fsQCA)(Ragin19872000)aswellasprobabilisticlikelihood-basedandBayesianstatistics(Gill2004)9 Thismethodologi-calmenuincludesoptionsforscholarswhoholdvariedontologicalas-sumptionsaboutthesocialworld
MixedmethodstakeavarietyofformsAresearchermightusedifferentmethodstoaddressdifferentresearchquestionsorcontextsOrdifferentmethodsmightguidedifferentstagesofaresearchprogram(Lieberman2005)IncreasinglyscholarsstrivetousetwoormoremethodsateachstageofresearchThoseconcernedwithgeneralcausalpatternsdrawonquantitativeandqualitativemethods(Coppedge1999Lieberman2005Tarrow2004)Combinationsofformalandqualitativemethodsaddressconcernsaboutlogicalcoherenceandcausalprocessesincontingentrela-tionshipswherethereisnoexpectationofgenerality(Batesetal1998)Otherscontendthatscholarsshouldseeklogicalcoherenceandevidencefor causalprocesses and should test for thegeneralityof relationsbydrawingonformalqualitativeandquantitativemethods(Granatoand
Copyrighted Material
OvercomingChallenges bull 13
Scioli2004Laitin2003)Scholarswhodevelopagent-basedmodelsuserolegamesandexperiments tocollectdataaswellas involving stake-holdersinthevalidationoftheirmodels(BarreteauLePageandAquino2003Bousquetetal2002GurungBousquetandTreacutebuil2006)Otherscholars combine their formal models with ethnographic observations(Bharwanietal2005HuigenOvermarsanddeGroot2006)
Theuseofmultiplemethodshoweverdoesnotguaranteemethod-ologicallysuperiorsocialscienceresearchSomequestiontheextenttowhichformalqualitativeandquantitativeresearchmethodsareactuallycomplementary Several recent publications have argued that differentmethodsreflectdifferentassumptionsaboutthenatureofcausalityandhavecalledforgreatercareinmatchingmethodstoontologicalassump-tions(BennettandElman2006ClarkGilliganandGolder2006Hall2003Mahoney2003Ragin19872000)
Therearealso limits to the feasibilityofmultimethodresearchHy-potheses about complex causal relationships imply complex statisticalmodelsthatstretchthelimitsofavailabledataStatisticalanalysesoftenadd interaction termsordummyvariables tomodel contingent effectsandmultiplecausalpaths(ClarkGilliganandGolder2006Pontusson2007)but theseadditionalvariablesconsumedegreesof freedominacontextoflimiteddataavailability(Shalev2007)Othertechniquesde-velopedtoaddresscausalcomplexitysuchastheanalysisoftime-series-cross-sectional data and hierarchical models may strain the technicalskillsofboththeresearcherandtheaudience(Shalev2007)
MahoneyandGoertz(2006)contendthatinteractioneffectsdummyvariableshierarchicalmodelsandothersimilarstatisticalfixesdonotaccuratelyreflecttherelationshipspositedintheunderlyingtheoriesTheassumptionthatobservationsareindependentforexampleiscalledintoquestionbyglobalizationdiffusioneffectsandactor-centeredtheoriesthatemphasizestrategicinteractionsEvensomequantitativelyorientedscholarsquestiontheappropriatenessofstandardstatisticaltechniquesInrecentyearsnewtechniqueshavebeenproposedtoincorporateinter-dependence(Signorino1999)Bayesianstatistics(Dion1998Gill2004)andBooleanlogic(Braumoeller2003Ragin19872000)Theverdictisstilloutonwhetherthesenewtechniquesmatchunderlyingassumptionsbetterthandoesregression-basedstatistics
Too often the development of ever-more sophisticated techniquesseemstobeanendinitselfThelatesttechniquesaresometimesadoptedwithlittlereferencetotheoreticalconsiderationsorunderstandingoftheunderlyingassumptionsButmethodologicalsophisticationcannotsub-stitutefortheoryAchen(20022005)warnsthatquantitativeanalysesthat are not supported by theoretical microfoundations or careful ex-plorationofthedatayieldunreliableresultsandshouldnotbetrusted
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 11
ologicalchallengesseemtodrivetheoreticalargumentsratherthantheotherwayaround(Lieberson1991318)Indeedsophisticatedmethodssometimescrowdouttheoryaltogether(Achen20022005)WearguethatmethodologicalchoicesareoftendrivenasmuchbydataavailabilityorcareerincentivesWhencareersurvivalisatstakepracticalconsider-ationscansqueezeoutconcernsaboutmatchingtheoryandmethodThelinkbetweenmethodsandcareerprospectscanhoweverbeexpectedtoinfluencethetenorofmethodologicalmdashandtheoreticalmdashdebates
SometimesmethodologicalandtheoreticaldebatestakeonexistentialovertonesWhenaparticulartheoryandassociatedmethodsbecomeex-tremelywidespreadforexampleproponentsofalternativeapproachesmayworryabouttheirownacademicsurvivalProponentsofnewtheo-riesmdashandassociatedmethodsmdashalsofaceanexistentialfightforrecogni-tionandsurvivalThedegreeof(perceived)existentialthreatdependsonthe extent to which fellowships job opportunities publishing outletsandresearchgrantsareopen(orclosed)todiversetheoriesandmethodsTherecentperestroikamovementinpoliticalscienceforexamplepre-sented itself asdefendingagainstmethodologicalhegemony conjuringimages of political scientists conspiring to control journal outlets andprofessionalassociations7ThiswasnotsimplyamethodologicalcritiquebutacalltoactionagainstpresumedtyrannyWhiletheinflammatorypublic languageassociatedwiththeperestroikamovementmaybeun-usualthelayeringofmethodologicaldebateswithvaluejudgmentsandcompetitionforprofessionalrecognitionandresourcesiscommonplace
Despite references to ldquorevolutionsrdquo and paradigm shifts new socialsciencetheoriesandmethodshavenotfullydisplacedtheirpredecessorsRathereachnewtheoryandmethodhasaddedanotherstrandConstruc-tivists institutionalists andpostmodernists coexistwithbehavioralistsandstructuralistsDespitethehistoryoftheoreticalandmethodologicalcompetitionandcritiquescholarsalsoengageincreativesynthesisThecurrentappreciationformethodologicalpluralismmaybeinterpretedasaproductofthesurvivalandadaptationofapproachesthatwereonceperceived to be under existential threat Promotion of methodologicalpluralismfavorsatheoreticaleclecticismthatshoulddecreaseconcernsaboutexistential threats toparticularapproachesandshouldthusde-creasetheintensityofmethodologicaldebates
Multiple Methods Promises and Challenges
Therearemanyreasonsforsocialscientiststowelcomemethodologicalpluralism and greater use of mixed methods No single method over-comes all challenges Case studies and small-N comparative research
Copyrighted Material
12 bull Chapter1
designs offer advantages for concept and theory development as wellasevaluationofhypothesizedcausalsequencesandmechanisms(Bates2008Batesetal1998CollierBradyandSeawright2004Coppedge1999 Lieberman 2005) Rich explanations of particular cases are of-tenvaluablesubstantivelyandtheoretically(MahoneyandGoertz2006Rogowski2004)Yetasiswidelyrecognizedsmall-Nstudiesofferanuncertainfoundationforpositingorevaluatinggeneralrelationships
Formalmethodsseektobuild logicallycoherentmodelsanddiscerntheir logical implicationsTheir emphasison logical consistency facili-tatesthedistillationofparsimoniousyetgeneralhypothesesandguidesthechoiceofstatisticaltechniques(Achen20022005Batesetal1998GranatoandScioli2004)8 Thehighlevelofabstractioninformalmod-elshoweverraisesquestionsabouttheirempiricalapplicability(GreenandShapiro1994)Thecontrolledconditionsinexperimentalresearchprovidegreaterconfidenceintheinternalvalidityofobservedrelation-shipsTheexternalvalidityofgeneralrelationshipscanbestbeevaluatedhoweverthroughanalysisofalargenumberofnonexperimentalobser-vations(Goldthorpe1997KingKeohaneandVerba1994)aswellasthroughfieldexperiments(seeCardenas2003CardenasStranlundandWillis2000Henrichetal2004List2004)
Small-Nqualitativestudiescansuggesttheplausibilityofformalmod-elsbutprovidelittleleverageinassessingthegeneralityofrelationshipsThebroad comparisons required to evaluate thegeneralityofhypoth-esizedrelationshipsdemandsomeformofquantitativeanalysisWherequantitativeanalysisoncemeantregression-basedanalysisoptions forquantitativeanalysisofempiricalsocialsciencedatanowincludeQuali-tative Comparative Analysis (QCA) and fuzzy-set Qualitative Com-parativeAnalysis(fsQCA)(Ragin19872000)aswellasprobabilisticlikelihood-basedandBayesianstatistics(Gill2004)9 Thismethodologi-calmenuincludesoptionsforscholarswhoholdvariedontologicalas-sumptionsaboutthesocialworld
MixedmethodstakeavarietyofformsAresearchermightusedifferentmethodstoaddressdifferentresearchquestionsorcontextsOrdifferentmethodsmightguidedifferentstagesofaresearchprogram(Lieberman2005)IncreasinglyscholarsstrivetousetwoormoremethodsateachstageofresearchThoseconcernedwithgeneralcausalpatternsdrawonquantitativeandqualitativemethods(Coppedge1999Lieberman2005Tarrow2004)Combinationsofformalandqualitativemethodsaddressconcernsaboutlogicalcoherenceandcausalprocessesincontingentrela-tionshipswherethereisnoexpectationofgenerality(Batesetal1998)Otherscontendthatscholarsshouldseeklogicalcoherenceandevidencefor causalprocesses and should test for thegeneralityof relationsbydrawingonformalqualitativeandquantitativemethods(Granatoand
Copyrighted Material
OvercomingChallenges bull 13
Scioli2004Laitin2003)Scholarswhodevelopagent-basedmodelsuserolegamesandexperiments tocollectdataaswellas involving stake-holdersinthevalidationoftheirmodels(BarreteauLePageandAquino2003Bousquetetal2002GurungBousquetandTreacutebuil2006)Otherscholars combine their formal models with ethnographic observations(Bharwanietal2005HuigenOvermarsanddeGroot2006)
Theuseofmultiplemethodshoweverdoesnotguaranteemethod-ologicallysuperiorsocialscienceresearchSomequestiontheextenttowhichformalqualitativeandquantitativeresearchmethodsareactuallycomplementary Several recent publications have argued that differentmethodsreflectdifferentassumptionsaboutthenatureofcausalityandhavecalledforgreatercareinmatchingmethodstoontologicalassump-tions(BennettandElman2006ClarkGilliganandGolder2006Hall2003Mahoney2003Ragin19872000)
Therearealso limits to the feasibilityofmultimethodresearchHy-potheses about complex causal relationships imply complex statisticalmodelsthatstretchthelimitsofavailabledataStatisticalanalysesoftenadd interaction termsordummyvariables tomodel contingent effectsandmultiplecausalpaths(ClarkGilliganandGolder2006Pontusson2007)but theseadditionalvariablesconsumedegreesof freedominacontextoflimiteddataavailability(Shalev2007)Othertechniquesde-velopedtoaddresscausalcomplexitysuchastheanalysisoftime-series-cross-sectional data and hierarchical models may strain the technicalskillsofboththeresearcherandtheaudience(Shalev2007)
MahoneyandGoertz(2006)contendthatinteractioneffectsdummyvariableshierarchicalmodelsandothersimilarstatisticalfixesdonotaccuratelyreflecttherelationshipspositedintheunderlyingtheoriesTheassumptionthatobservationsareindependentforexampleiscalledintoquestionbyglobalizationdiffusioneffectsandactor-centeredtheoriesthatemphasizestrategicinteractionsEvensomequantitativelyorientedscholarsquestiontheappropriatenessofstandardstatisticaltechniquesInrecentyearsnewtechniqueshavebeenproposedtoincorporateinter-dependence(Signorino1999)Bayesianstatistics(Dion1998Gill2004)andBooleanlogic(Braumoeller2003Ragin19872000)Theverdictisstilloutonwhetherthesenewtechniquesmatchunderlyingassumptionsbetterthandoesregression-basedstatistics
Too often the development of ever-more sophisticated techniquesseemstobeanendinitselfThelatesttechniquesaresometimesadoptedwithlittlereferencetotheoreticalconsiderationsorunderstandingoftheunderlyingassumptionsButmethodologicalsophisticationcannotsub-stitutefortheoryAchen(20022005)warnsthatquantitativeanalysesthat are not supported by theoretical microfoundations or careful ex-plorationofthedatayieldunreliableresultsandshouldnotbetrusted
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
12 bull Chapter1
designs offer advantages for concept and theory development as wellasevaluationofhypothesizedcausalsequencesandmechanisms(Bates2008Batesetal1998CollierBradyandSeawright2004Coppedge1999 Lieberman 2005) Rich explanations of particular cases are of-tenvaluablesubstantivelyandtheoretically(MahoneyandGoertz2006Rogowski2004)Yetasiswidelyrecognizedsmall-Nstudiesofferanuncertainfoundationforpositingorevaluatinggeneralrelationships
Formalmethodsseektobuild logicallycoherentmodelsanddiscerntheir logical implicationsTheir emphasison logical consistency facili-tatesthedistillationofparsimoniousyetgeneralhypothesesandguidesthechoiceofstatisticaltechniques(Achen20022005Batesetal1998GranatoandScioli2004)8 Thehighlevelofabstractioninformalmod-elshoweverraisesquestionsabouttheirempiricalapplicability(GreenandShapiro1994)Thecontrolledconditionsinexperimentalresearchprovidegreaterconfidenceintheinternalvalidityofobservedrelation-shipsTheexternalvalidityofgeneralrelationshipscanbestbeevaluatedhoweverthroughanalysisofalargenumberofnonexperimentalobser-vations(Goldthorpe1997KingKeohaneandVerba1994)aswellasthroughfieldexperiments(seeCardenas2003CardenasStranlundandWillis2000Henrichetal2004List2004)
Small-Nqualitativestudiescansuggesttheplausibilityofformalmod-elsbutprovidelittleleverageinassessingthegeneralityofrelationshipsThebroad comparisons required to evaluate thegeneralityofhypoth-esizedrelationshipsdemandsomeformofquantitativeanalysisWherequantitativeanalysisoncemeantregression-basedanalysisoptions forquantitativeanalysisofempiricalsocialsciencedatanowincludeQuali-tative Comparative Analysis (QCA) and fuzzy-set Qualitative Com-parativeAnalysis(fsQCA)(Ragin19872000)aswellasprobabilisticlikelihood-basedandBayesianstatistics(Gill2004)9 Thismethodologi-calmenuincludesoptionsforscholarswhoholdvariedontologicalas-sumptionsaboutthesocialworld
MixedmethodstakeavarietyofformsAresearchermightusedifferentmethodstoaddressdifferentresearchquestionsorcontextsOrdifferentmethodsmightguidedifferentstagesofaresearchprogram(Lieberman2005)IncreasinglyscholarsstrivetousetwoormoremethodsateachstageofresearchThoseconcernedwithgeneralcausalpatternsdrawonquantitativeandqualitativemethods(Coppedge1999Lieberman2005Tarrow2004)Combinationsofformalandqualitativemethodsaddressconcernsaboutlogicalcoherenceandcausalprocessesincontingentrela-tionshipswherethereisnoexpectationofgenerality(Batesetal1998)Otherscontendthatscholarsshouldseeklogicalcoherenceandevidencefor causalprocesses and should test for thegeneralityof relationsbydrawingonformalqualitativeandquantitativemethods(Granatoand
Copyrighted Material
OvercomingChallenges bull 13
Scioli2004Laitin2003)Scholarswhodevelopagent-basedmodelsuserolegamesandexperiments tocollectdataaswellas involving stake-holdersinthevalidationoftheirmodels(BarreteauLePageandAquino2003Bousquetetal2002GurungBousquetandTreacutebuil2006)Otherscholars combine their formal models with ethnographic observations(Bharwanietal2005HuigenOvermarsanddeGroot2006)
Theuseofmultiplemethodshoweverdoesnotguaranteemethod-ologicallysuperiorsocialscienceresearchSomequestiontheextenttowhichformalqualitativeandquantitativeresearchmethodsareactuallycomplementary Several recent publications have argued that differentmethodsreflectdifferentassumptionsaboutthenatureofcausalityandhavecalledforgreatercareinmatchingmethodstoontologicalassump-tions(BennettandElman2006ClarkGilliganandGolder2006Hall2003Mahoney2003Ragin19872000)
Therearealso limits to the feasibilityofmultimethodresearchHy-potheses about complex causal relationships imply complex statisticalmodelsthatstretchthelimitsofavailabledataStatisticalanalysesoftenadd interaction termsordummyvariables tomodel contingent effectsandmultiplecausalpaths(ClarkGilliganandGolder2006Pontusson2007)but theseadditionalvariablesconsumedegreesof freedominacontextoflimiteddataavailability(Shalev2007)Othertechniquesde-velopedtoaddresscausalcomplexitysuchastheanalysisoftime-series-cross-sectional data and hierarchical models may strain the technicalskillsofboththeresearcherandtheaudience(Shalev2007)
MahoneyandGoertz(2006)contendthatinteractioneffectsdummyvariableshierarchicalmodelsandothersimilarstatisticalfixesdonotaccuratelyreflecttherelationshipspositedintheunderlyingtheoriesTheassumptionthatobservationsareindependentforexampleiscalledintoquestionbyglobalizationdiffusioneffectsandactor-centeredtheoriesthatemphasizestrategicinteractionsEvensomequantitativelyorientedscholarsquestiontheappropriatenessofstandardstatisticaltechniquesInrecentyearsnewtechniqueshavebeenproposedtoincorporateinter-dependence(Signorino1999)Bayesianstatistics(Dion1998Gill2004)andBooleanlogic(Braumoeller2003Ragin19872000)Theverdictisstilloutonwhetherthesenewtechniquesmatchunderlyingassumptionsbetterthandoesregression-basedstatistics
Too often the development of ever-more sophisticated techniquesseemstobeanendinitselfThelatesttechniquesaresometimesadoptedwithlittlereferencetotheoreticalconsiderationsorunderstandingoftheunderlyingassumptionsButmethodologicalsophisticationcannotsub-stitutefortheoryAchen(20022005)warnsthatquantitativeanalysesthat are not supported by theoretical microfoundations or careful ex-plorationofthedatayieldunreliableresultsandshouldnotbetrusted
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 13
Scioli2004Laitin2003)Scholarswhodevelopagent-basedmodelsuserolegamesandexperiments tocollectdataaswellas involving stake-holdersinthevalidationoftheirmodels(BarreteauLePageandAquino2003Bousquetetal2002GurungBousquetandTreacutebuil2006)Otherscholars combine their formal models with ethnographic observations(Bharwanietal2005HuigenOvermarsanddeGroot2006)
Theuseofmultiplemethodshoweverdoesnotguaranteemethod-ologicallysuperiorsocialscienceresearchSomequestiontheextenttowhichformalqualitativeandquantitativeresearchmethodsareactuallycomplementary Several recent publications have argued that differentmethodsreflectdifferentassumptionsaboutthenatureofcausalityandhavecalledforgreatercareinmatchingmethodstoontologicalassump-tions(BennettandElman2006ClarkGilliganandGolder2006Hall2003Mahoney2003Ragin19872000)
Therearealso limits to the feasibilityofmultimethodresearchHy-potheses about complex causal relationships imply complex statisticalmodelsthatstretchthelimitsofavailabledataStatisticalanalysesoftenadd interaction termsordummyvariables tomodel contingent effectsandmultiplecausalpaths(ClarkGilliganandGolder2006Pontusson2007)but theseadditionalvariablesconsumedegreesof freedominacontextoflimiteddataavailability(Shalev2007)Othertechniquesde-velopedtoaddresscausalcomplexitysuchastheanalysisoftime-series-cross-sectional data and hierarchical models may strain the technicalskillsofboththeresearcherandtheaudience(Shalev2007)
MahoneyandGoertz(2006)contendthatinteractioneffectsdummyvariableshierarchicalmodelsandothersimilarstatisticalfixesdonotaccuratelyreflecttherelationshipspositedintheunderlyingtheoriesTheassumptionthatobservationsareindependentforexampleiscalledintoquestionbyglobalizationdiffusioneffectsandactor-centeredtheoriesthatemphasizestrategicinteractionsEvensomequantitativelyorientedscholarsquestiontheappropriatenessofstandardstatisticaltechniquesInrecentyearsnewtechniqueshavebeenproposedtoincorporateinter-dependence(Signorino1999)Bayesianstatistics(Dion1998Gill2004)andBooleanlogic(Braumoeller2003Ragin19872000)Theverdictisstilloutonwhetherthesenewtechniquesmatchunderlyingassumptionsbetterthandoesregression-basedstatistics
Too often the development of ever-more sophisticated techniquesseemstobeanendinitselfThelatesttechniquesaresometimesadoptedwithlittlereferencetotheoreticalconsiderationsorunderstandingoftheunderlyingassumptionsButmethodologicalsophisticationcannotsub-stitutefortheoryAchen(20022005)warnsthatquantitativeanalysesthat are not supported by theoretical microfoundations or careful ex-plorationofthedatayieldunreliableresultsandshouldnotbetrusted
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
14 bull Chapter1
Scholarsmustdomoretodevelopexplicittheoreticalargumentsanden-surethattheirmethodsmatchtheirunderlyingassumptionsaboutcau-salityontologyandepistemology(Achen20022005BradyandCollier2004Hall2003)
Neither theorynormethodological techniques substitute for a thor-oughfamiliaritywiththedatagainedfromdiagnostictestsanddataex-plorationVisualizationtechniquessuchasgraphicalanalysisandsimplestatisticaltechniquessuchascross-tabulationsbringempiricalregulari-tiesandpatternedvariationintofocus(Achen20022005Shalev2007)Dataexplorationdrawsattentiontopotentialcausalheterogeneitynon-linearrelationshipsinteractioneffectsandotheraspectsofthedatathatareobscuredbymoresophisticatedmultivariatetechniquesThusthor-oughdataexplorationcontributestotheorytestinganddevelopmentbycomplementingmoresophisticatedformsofdataanalysisanddrawingattentiontoempiricalpatternsthatcalloutfortheoreticalexplanation(Achen20022005)
EvenifcausalepistemologicalandontologicalassumptionsposenobarrierpracticalconsiderationscomplicatemethodologicalchoiceThesepracticalchallenges largelyoverlooked in theexchangesregarding therelativemeritsofalternativeandmultiplemethodsstandatthecenterofouranalysisThegroundswellofinterestinmultiplemethodsdemandsmoreintensiveanddiversifiedformsoftechnicalskill-developmentYetindividualresearchersrarelymastermorethanacoupleofmethodolo-gies Even within a single research tradition technical language andefforts to solve technicalproblems threaten toobscureorovershadowsubstantiveissues(BeckandKatz1996GreenandShapiro1994Shalev2007) If thereare limitstothemethodsanyindividualresearchercanmasterwhatare the implications formultimethodresearchThenexttwosectionselaborateonsomeofthepracticalandcareer-relatedcon-straintsonmethodologicalpractice
Practical Challenges and Methodological Trade-Offs
Methodological debates in the social sciences have had at least threepositiveeffectsFirst steriledebatesover the superiorityofalternativemethodshavegivenwaytoanappreciationoftrade-offsandcomplemen-taritiesbetweenapproachesSecondthegoalsofqualitativeresearchandassociatedmethodsarereceivingmoreexplicitelaborationinresponsetoafeelingthattheywerewidelymisunderstood(BradyandCollier2004Coppedge 1999 Gerring 2001 2004 Goodwin and Horowitz 2002MahoneyandRueschemeyer2003)10 Thirdmoreconstructivecritiqueshave stimulated considerable innovation in techniques within specific
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 15
methodologicaltraditionsandinstrategiesforcombiningmultiplemeth-odsinresearch(Batesetal1998Braumoeller20032004Gill2004Lieberman2005Ragin19872000)
NonethelessscholarsoftenstruggletomakefullandappropriateuseofavailableresearchmethodsAseachmethodologicaltraditionbecomesmoresophisticatedthetaskofmasteringmultiplemethodsalsobecomesmorechallengingWhenresearchdemandsintensivefieldworkandsub-stantiallocalknowledgeunavoidablylargeinvestmentsindatacollectionpresent additionalobstaclesAll toooftenmethodologicaldiscussionsoverlookthesepracticalconstraintsonmethodologicalchoice
Wepromotecollaborativeresearchasawaytoexpandthepotentialforusingmultiplemethodswellintheanalysisofbroadlycomparativere-searchCollaborationcanbringscholarsfrommultipledisciplinestogetheron the same research teamwith strengths in complementary methodsincreasingconfidencethateachmethodisappliedrigorouslyLikewisecollaborationthatbringstogetherexpertiseaboutdifferentcountriescanexpandthescopeofcomparisonInthisbookwewilldiscussavarietyofstrategiesforcollaborativeresearchandanalyzeobstaclestocollabora-tiveandbroadlycomparativeresearchButfirstweoutlinesomepracti-calconstraintsonmultimethodandcollaborativeresearch
Technological Development and the Costs of Border Crossing
ContemporarysocialsciencefeaturestremendousinnovationwithineachmethodologicaltraditionInnovationindicatesvitalitybutalsoincreasesthecostsofcompetencyinaparticularmethodHigherentrycostsraisethebarrierstomethodologicalbordercrossingAndyetthebenefitsofmultimethodresearchdependoncompetentapplicationofeachmethodOtherwise theuseofmultiplemethodsweakens rather than strength-ensconfidenceintheresearchTobetterillustratethechallengesletusconsiderwhatisrequiredforaresearchertogaincompetencyinseveralmethodsformalquantitativeexperimentalandqualitative
ThetechnicaldemandsofformalmodelingwereevidentevenasthisapproachspreadacrossthesocialsciencesFormalmodelingrequiresacommandofsettheoryandmathematicallogicoptimizationandothertechniquesfromeconomicsgametheoryandcomplexitytheoryCom-putationalmodelersrequireskills inprogrammingandalgorithmicde-signFormaltheoristsdevoteconsiderableenergytothedevelopmentofnewmodelingtechniquesandsolutionconcepts
IncreasinglysimilarconditionsprevailinquantitativemethodologyAsrecentlyasthe1980smanysocialscientistsequatedquantitativeresearchwithordinaryleastsquaresregressionTheassumptionsformultivariateregressionrarelyholdforsocialphenomenahoweverandmoresuitable
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
16 bull Chapter1
statistical techniques exist ldquoStandardrdquo quantitative techniques nowencompassmaximum likelihood techniquesanalysisof cross-national-time-seriesdataandanalysesofeventhistoriesAvarietyofothertech-niquesincludingBayesianstatisticsandBoolean-basedmethodsarealsobecomingmorecommonComputationalpowerandstatisticalsoftwaremake it very easy toapplyadvanced statistical techniquesbutdonotguaranteeappropriateapplicationEachtechniqueinvolvesaparticularsetofassumptionsdiagnosticchecksandongoingdebatesabouttech-nologicalfixesAswithformalmethodsalargeinvestmentisrequiredoftheresearcherseekingtogainandmaintaincompetencyinevenasubsetofquantitativemethods
Ifresearchersaretoperformexperimentsitiscrucialthattheylearnthepracticeofexperimentaldesigninordertomeasuretherelevantattri-butesofdifferentexperimentaltreatmentsThisrequiresthedevelopmentof hypotheses related to outcomes expected from different treatmentsbased on formal models and statistical analysis on the data collectedfromtheexperimentstotestthesignificanceofdifferencesfoundacrosstreatmentsSomeoneonanexperimentalteamwillalsoneedprogram-mingskillstoenterandanalyzethedataandtoentertheexperimentalinstructions and response categories for experiments run in computerlaboratories
Themenuofqualitativemethodsofdatacollectiontechniquesincludesethnographyparticipantobservationinterviewsoralhistoriesandar-chivalresearchEachtechniqueinvolvesasetofissuesthatresearchersmustunderstandandaddresstoapplythemethodwell (egBurawoy1998Lustick1996Rocheleau1995)Manyofthesetechniquesrequirea substantial period of fieldwork keen observational skills thoroughrecord keeping and a high degree of self-awareness and ethical man-agementofsocialrelations11 Forfieldworkresearchersmusthaveap-propriatelanguageskillsandsufficientunderstandingofthelocalcontexttogainaccessrecognizeinformalinstitutionsandaccuratelyinterpretculturallycodedobservations
Fieldworkyieldsvoluminousdatabutthedatagenerallytakeformsthat are not easily processed (H Becker 1996) Thus the value of aqualitativestudyhingesondisciplineddataanalysisrelatedtotheoreti-calquestions (Campbell1975Lijphart1971)Qualitative researchershavedevelopedavarietyoftechniquestostructuredataanalysissuchascounterfactualanalysisprocesstracingstructuredcomparisonsandanalysisofdeviantcases(BennettandElman2006Fearon1991Gold-stone1997Tarrow2004)ThedevelopmentofsoftwareforComputer-AssistedQualitativeDataAnalysis(CAQDAS)expandsoptionsfordatamanagement There is considerable confusion however about whattheseprogramsdothedifferencesamongthemhowtomatchprograms
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 17
and theoretical approaches and even whether CAQDAS makes senseforaparticularstudyorapproach(MacMillanandKoenig2004)Asinquantitativeresearchtheincreaseincomputationaltoolscanfacilitaterigorousdataanalysisbutitcanalsoproducemisleadingresultsifap-pliedinappropriately
Thus each method encompasses several sophisticated techniquesWhetheramethodyieldsanalytical insightsormisleadingfindingsde-pends on competency in recognizing appropriate techniques imple-mentingthemwellandmakingsenseofthedataAlargeandongoinginvestmentisnecessaryfortheresearchertogainandmaintaincompe-tencyinagivenmethodTheinvestmentrequiredtomasteranysinglemethodisnotexcessivebut it limits thenumberofmethods inwhichanyindividualcanbeexpectedtogainandmaintaincompetencyWhilescholarsshouldutilizediversemethodsaspossibleandappropriatemeth-odologicalspecializationandmultimethodresearchdesignspresentadi-lemmaCollaborationoffersapotentialsolutionScholarswithstrengthsincomplementarymethodscanworktogetherwithincreasedconfidencethateachmethodisappliedrigorously
Availability and Accessibility of Data
DependingontheperiodcountryandscaleofanalysisdatamightbeabundantandreadilyavailableorvirtuallynonexistentDifferentmeth-ods require different kinds and quantities of data Data compiled bynational and international agencies donot addressmany issues at thesubnational levelandareoftenblindtobothinformal institutionsandnoneliteactorsEveninindustrializeddemocraciesdataavailabilityandqualityvaryconsiderablyacrossstatesprovincescitiesandothersub-nationaljurisdictionsReliableandcomprehensivedatasourcesoftendonot exist for nongovernmental organizations informal institutions orcollectiveactionInpartthelackofreadilyavailabledataoninformalinstitutionssubnationalphenomenanoneliteactorsandothersimilartopicsreflectsthedifficultyofdatacollectionInformalityandnonelitestatusimplyaneedforlocalknowledgeandtrustIntheabsenceoftrustlocalactorsmayhesitate toprovideaccurate informationabout them-selvestheirpracticesorotherinformalinstitutions
Asthecostsofdatacollectionincreasesodotherestrictionsonmeth-odologicalchoiceRecommendationsthatqualitativeresearchersshouldgathermoredata(Goldthorpe1997KingKeohaneandVerba1994Lijphart 1971) ignore the difficulty of recognizing some types of phe-nomenainfieldsettingsthecostsofcollectingqualitativedataandthevoluminousyetdifficult-to-processdatayieldedbyqualitativeresearch(HBecker1996PoteeteandOstrom2004b)Theseconditionsmakeit
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
18 bull Chapter1
moredifficulttobuildlargedatabasesforquantitativeanalysisevenifquantitativeanalysismakessenseforagiventheoreticalapproach
DataproblemsalsovaryinseverityIfdataavailabilityandaccesswereunproblematicthenscholarscouldchoosemethodologiesthatmatchedtheir causalandepistemologicalassumptionsScholarshave to choosefromasubsetoflessappropriatemethodshoweverwhendataarenotreadilyavailableAsaresultthecapacitytoengageinquantitativeanaly-sisandbroadcomparisonishigherforresearchonformalinstitutionssometypesofinternationalandnationalphenomenaandelitesBecausedataoninformalinstitutionssubnationalissuesandhistoricallydisad-vantagedpopulationsarelessreadilyavailableitisquiteachallengetoengageinbroadlycomparativeandquantitativesocialresearchonthesetopics
At least inprinciplecollaborativeresearchenhancesamoregeneralcomparativeanalysiswithoutsacrificingdataqualityCollaborativere-search offers the potential to collect larger quantities of data engagein more broadly comparative research and utilize a broader array ofmethodscompetentlyUnlikeanindividualresearcherwhoisexpectedtodoitallcollaboratorscanpooltheirdataanddrawoncomplementarymethodologicalskillsUsingformalmodelsScottPage(2007)foundthatgroupswithahigherdiversityofproblem-solvingapproachesaremoreeffectiveinovercomingdifficultproblemsThisgivesusevenmorecon-fidence in strongly recommending collaboration across methods as animportantfoundationforthefuturedevelopmentofthesocialsciences
InpracticecollaborativeresearchisitselfchallengingCollaborationisgenerallylimitedbydivergentresearchinterestsandtheoreticalorien-tations Inconsistency in conceptualization and measurement can be aproblemaswell(PoteeteandOstrom2004b)especiallyforqualitativeresearcherswhoworkhard todevelopcontextually suitablemeasuresYetthesechallengesarenotinsurmountableColleagueswithsharedin-terestsandtheoreticalperspectivescancollaborateonthefullresearchprocess from conceptualization through analysisAs discussedbelowhowever thesocialsciencesstill reward individualresearchmorethantheydo collaborative research Scholars concerned about their careersrecognize these incentivesand limit theirparticipation incollaborativeefforts
Career Incentives as Methodological Constraints
Ideallytraininginthesocialsciencesshouldencouragescholarstodevelopcompetencyinavarietyofmethodsandengageincollaborationsthatfur-
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 19
therextendtheirmethodologicalrangeUniversitiesshouldfostermulti-methodandcollaborativeresearchbyencouragingcross-appointmentsandbycreatingandsustainingthematicresearchcentersandinitiativesFundingagenciesshouldofferlonger-termgrantstosupportthelongertimeframerequiredformultimethodandcollaborativeresearchInreal-ityacademiarewardsspecializationandindividualprojectsespeciallyinearlycareerstagesAlthoughcollaborativeandmultimethodresearchcanyieldbetterknowledgeindividualaccomplishmentsdomoretoadvancecareersThetenureclockalsogeneratesmorestimulusforrapidresearchoutputthanforthedevelopmentoflonger-termresearchprogramsAndfundingagenciesrarelyprovidelong-termsupport
Training
Graduateprogramcurriculaandprogramsforintensivemethodologicaltrainingprovidean indicationofdisciplinarysupport formultimethodandcollaborativeresearchTraininginquantitativemethodshasbeenastandardcomponentofgraduateprogramsineconomicspoliticalscienceandsociologythroughoutthepostwarperiodLikewiseopportunitiestosupplementin-housecourseswithintensivetraininginmorespecializedquantitativemethodshavebeenavailablefordecadesProbablythebest-knownsourceofspecializedquantitativetrainingforsocialscientiststheInteruniversityConsortiumforPoliticalandSocialResearch(ICPSR)attheUniversityofMichiganhasofferedanannual summer institute inresearchmethodssincethe1960s
BycomparisonoptionsfortraininginqualitativemethodswererareuntilrecentlyBeforetheturnofthe(current)centurymostsocialsciencedepartmentsofferednograduatetraininginqualitativemethodsbeyondacourseinresearchdesignOpportunitiesforintensivetraininginotherqualitativemethodsandinmultimethodresearchhaveexpandedoverthepastdecadeTheConsortiumonQualitativeResearchMethodsholdsanannual intensiveInstitute inQualitativeandMulti-MethodResearch12
The (US)NationalScienceFoundationhassupportedmethodologicaltrainingprogramsforthesocialsciencesincludingmonth-longsummerinstitutesonmultimethodresearchbeginningwiththeEmpiricalImplica-tionsofTheoreticalModels(EITM)programtheSummerInstituteonResearchDesigninCulturalAnthropologyShortCoursesonResearchMethods inCulturalAnthropologyandFieldTraining inMethodsofData Collection in Cultural Anthropology13 Even with these new op-portunities social science graduate students interested inmultimethodresearch find it difficult to gain adequate training in nonquantitativemethods(Siegeletal2007)
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
20 bull Chapter1
Career Incentives and Specialization
SusanneLohmann (2007)argues forcefully that theprocedures for re-viewingmanuscriptsgrantapplicationsandapplicationsforacademicpositionsandpromotionsstronglyfavorspecializationAlloftheseformsofevaluationrelyonpeerreviewAsLohmannnotespeerreviewgener-allymeansreviewbyspecialistsTheworkofaspecialistwillbereviewedbyother specialists in the samemethodwith the sameareaexpertiseandorwiththesameorsimilarsubstantiveconcernsScholarswiththesamespecializationshareacommonunderstandingoftheirareaassumeitsvalueandarefamiliarwithpracticalchallengesfacedbytheirfavoredapproach
Scholars who engage multiple methods or disciplines on the otherhandwillmostlikelybeevaluatedbydisciplinaryspecialistsratherthanotherpractitionersofmultimethodorinterdisciplinaryresearchThere-viewersarenotlikelytofullyunderstandallofthemethodstherationaleformixingmethodsorthechallengesinvolvedinmultimethodresearchSpecialiststendtodiscounttheresultsofunfamiliarmethodsreferencestoworksinotherfieldspublicationsinjournalsoutsidetheirowndisci-plineandinterdisciplinarypublicationsThusscholarswhousemultiplemethodsanddrawonmultipledisciplines tend toget less enthusiasticandmorecontradictoryevaluationsOnlythebestscholarssurvivethisprocessAsaresultLohmannarguesasmallproportionofsocialscien-tistsaretop-notchscholarswhousediversemethodsandcrosssubfieldanddisciplinaryboundariesbut specialistsdominate thefieldnumeri-cally Despite increased interest in multimethod research hiring com-mitteesstillprefercandidateswhohaveastrongcommandofasinglemethod over candidates with more superficial competency in multiplemethods(Siegeletal2007)
Similar dynamics associated with career incentives constrain collab-orative researchHistorically as a profession the social sciences haverewarded individual innovation and individual accomplishments morethantheyhavecollaborativeresearchCommitteeschargedwithhiringand promotion typically give more weight to single-authored publica-tions than to multiauthored publications (Rothgeb and Burger 2009)Multiauthoredpublicationsareviewedwithskepticisminpartbecauseit is impossible to discern the individual contribution of each authorScholarsarewellawareoftheseissuesandrespondtothemwhenmak-ing decisions about how to pursue their research agendas Collabora-tivesocialscienceresearchhasbecomemorecommonbutpublicationsrarelyhavemore than threeauthors14 Thiscontrasts sharplywith thenaturalscienceswherepublicationsoftenincludethenamesofalloftheresearchersworkinginalaboratory
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 21
Funding opportunities and career incentives that privilege particu-larmethodsalsoprivilege researchon topics forwhich thosemethodsarepossible(Lohmann2007)Onemightimaginethatrelativescarcityof data and greater practical difficulties in collecting comparable datawould merit higher levels of funding and institutional support for so-ciallyimportanttopicsOftenhoweverthisisnotthecaseScholarswhostudydata-scarcetopicscontendwithpracticalchallengesindatacollec-tionandanalysisthatlimittheirmethodologicaloptionsbutthentheirmethodologicalchoicesoftenlimittheirabilitytocompeteforfundingandgaincriticalappreciationfortheirworkasdiscussedabove
Funding agencies encourage fieldwork collaboration and multi-methodsocial science research to someextent15 Theprevalenceof in-tensivefieldworkhaswaxedandwanedreflectingvariablefinancialandinstitutionalsupportforlanguagetrainingandextendedperiodsoffield-basedresearchaswellasfluctuatingprofessionalappreciationforsuchresearchIntheUnitedStatesfield-basedresearchwasencouragedduringtheperiodimmediatelyfollowingtheworldwarsAsfinancialsupportforareastudiesdeclinedhoweverextendedfield-basedresearchbecameless common Theoretical and methodological trends favored broadlycomparativeanalysiswhichdampenedinterest inextendedfield-basedresearchEvenwhendonorsdosupportthesortofresearchrequiredforthe studyofdata-scarce topics theyrarelyprovide long-termsupportYetforresearchontopicswheredataarerelativelyscarceanddifficulttocollectlong-termsupportmayberequiredtofullyovercomepracticalobstaclestobroadlycomparativeresearchLong-termsupportcouldalsohelpovercomecollective-actionproblemsamongscholars
Careerincentivesdiscourage broadcollaborationandmultimethodre-searchinthesocialsciencesespeciallyforjuniorfacultyandexacerbatecollective-actionproblemsTheinfluenceofcareerincentivesonmethod-ologicalchoicesappearsasaleitmotifinthisbook16 Giventheunavoid-able influence of professional incentives and the other features of theacademicworldthisbookconsidershowfundingagenciesprofessionalassociationsuniversitiesandacademicdepartmentsandprogramscouldbetterencourageinnovativeeffortstotacklepracticalchallengesthatin-fluencemethodologicalchoicesandthusinfluencesubstantiveemphasesinsocialscienceresearch
Our Substantive Focus
We illustrate the challenges advantages anddisadvantages associatedwithparticularmethodswithreferencetoresearchoncollectiveaction
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
22 bull Chapter1
for the regulation of natural resources In its contemporary form re-searchoncollectiveactionforthemanagementofnaturalresourcesre-spondstoHScottGordon(1954)andGarrettHardin(1968)bothofwhomemphasizedthedifficultyofmanagingsharednaturalresourcesOver the subsequenthalf century scholars fromacross the social andnaturalscienceshaveusedawidevarietyofresearchtechniquestoestab-lishthepossibilityofcollectiveactionfornaturalresourcemanagementidentifyconditionsassociatedwiththeemergenceanddurabilityofcol-lectiveactionandassesswhetherandwhencollectiveactioncontributestosustainablemanagementoftheresourcebase
Althoughwecouldhavedrawnonexamplesrelatedtodiverseresearchagendasfocusingonasinglewell-definedresearchstreamallowsustotracetheinteractionsbetweentheorymethodsandresultsbothintermsofhowtheoryguidesmethodologicalchoicesandhowvariousmethodscontribute to theoretical development Collective-action problems arepervasiveand importantTheyoccur in families theworkplace legis-laturesandinternationalrelationsTheyaffecttheprovisionofpublicgoodslikeinfrastructureandsocialmobilizationofgroupswithsharedpoliticalagendasProblemsofcollectiveactionhavecontributedtothecollapseoffisheriesdeforestationandclimatechangeFurthertheco-authorsof thisvolumehavethemselvesundertakenextensiveresearchusingmultiplemethodsoncollectiveactionforthemanagementofnatu-ralresourcesThuswecanspeakfromexperienceaswellasdrawingontheworkofothers
In approaching natural resource management as a question of col-lectiveactionwearemakinganumberofontologicalassumptionsInourviewtheoreticalexplanationmustidentifycausalmechanismsThetheoryofcollectiveactionassumesthatindividualbehaviorhasacriticalinfluenceoncollectiveoutcomesWearewellawareofthelimitsofratio-nalityhoweverandfavorabehavioraltheoryofindividualactionthatallowsforlimitedinformationattentionandcognitiveprocessingWealsoassumethatindividualbehaviorisstructuredbycontextFornaturalresourcesrelevantcontextualconditionsincludetheecologicalstructureoftheresourcesystemthesociopoliticalandeconomicstructureandanarrayof institutional arrangementsPerhapsmost importantly for thisvolumeweassumethatcomparisonisvaluablebutthatthereisrarelyasingleorlinearpatternWeexpectcausalheterogeneitythereismorethanoneroutetothesameoutcome
Theseassumptionsandconcernshaveinfluencedthetheoreticalpuz-zlesthatwechosetohighlighttheliteraturereviewedandthemethodsexaminedOurontologicalassumptionsaresharedbymanysocialscien-tistsbutnotallStructuralistsinterpretivistsandthosewhobelieveinlawlikesocialpatternsmayrejectoneormoreofourassumptionsWe
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 23
hopethatthesescholarswillnonethelessbenefitfromthinkingaboutthepotentialtheoreticalcontributionsofavarietyofmethodsandthepracti-calchallengesthataffectmethodologicalpractices
Theresearchtraditiononcollectiveactionfornaturalresourcemanage-mentoffersagoodpointofreferencefordiscussing(1)theinteractionsbetween methodology and theory development (2) multiple methodsandcollaborativeresearch (3)practicalconstraintsonmethodologicalchoices and (4) the influence of career incentives on methodologicalpractice
Interactions between Theory and Methods
Scholarship on collective management of natural resources draws onawidevarietyof researchmethods including innovative strategies foraddressingpracticalmethodologicalconstraintsWewillshowhowdif-ferent methodsmdashabstract formal models case studies meta-analysescross-nationalcomparisonsandlaboratoryandfieldexperimentsmdashhavecontributedatdifferentpointsandindifferentwaystothedevelopmentofthisresearchagenda
Multiple Methods and Collaborative Research
PuzzlesrelatedtocollectivemanagementofnaturalresourcesspanthesocialandnaturalsciencesandinterdisciplinaryresearchisprominentThis research tradition features several innovative efforts toovercomepracticalchallengesandenablemorebroadlycomparativequantitativeandmultimethodresearchYetaswewilldocumentinchapter5collab-orativeresearchremainsrelativelyuncommonWedrawuponourownexperiencesaswellastheliteraturetohighlightboththepossibilitiesandthechallengesofcollaborativeandmultimethodresearch17
Practical Constraints on Methodological Choices
Problems with scarce and difficult-to-access data are rampant in thisresearch tradition Collective management of many natural resourcesoccursonasubnationalscaleandoftenentailsthedevelopmentofin-formalrulesforresourceusewithlittletonogovernmentinvolvementParticipantsinthedevelopmentandenforcementofthesearrangementsmayincludelocalbutnotnecessarilynationalelitesManyexamplesofcollectiveactionfornaturalresourcemanagementmdashoritsabsencemdashin-volveordinaryorhistoricallydisadvantagedpeopleBecauseinformalin-stitutionscanbedifficultforoutsiderstorecognizedataontheseeffortsare scarce and not readily accessible Qualitative field-based research
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
24 bull Chapter1
isnecessarytosimplyidentifyrelevantcasesforanalysisThepracticalchallengesofsuchresearcharetypicalofworkontopicsforwhichdataarescarceanddifficulttoacquire
Career Incentives and Methodological Practice
Analysisofcollectivemanagementofnatural resourcesrequiresafirmunderstanding of the natural system institutional arrangements andhumanbehaviorArguablyresearchonthistopicisinherentlyinterdis-ciplinaryand requiresmultiplemethodsApatchworkofprojectsandresearchcentersprovidesinstitutionalandfinancialsupportthattosomeextent lowers the risksof interdisciplinaryandmultimethod researchbut thesedonot fullycompensate forsystemwidecareer incentives Ingeneral career incentives encourage either specialization or relativelynarrowformsofmultimethodresearchWedrawuponourownexpe-riences working with interdisciplinary and multimethod research cen-ters thatenjoyedstrong institutionalsupportaswellas inldquoordinaryrdquodiscipline-basedsettings
Outline of the Book
Wewelcometherecentturnawayfromrecurringdebatesoverthesu-periorityofparticularmethodsinthesocialsciencesWeconnectmeth-odologicaldebates todifferencesover theoryandontologyemphasizevariationinthecapacitytoengageinascendantmethodsanddrawouttheimplicationsforcompetitionforcareer-relatedresourcesforseveralperiodsofintensemethodologicalconflictacrossthesocialsciencesWeagree that theuseofmultiplemethods can improve research inmanysituationsbutalso stress that it isnotalwaysappropriateor feasibleThatpracticalconsiderationsconstrainmethodologicalchoicesisacen-tralpointEvenwhenscholarsareawareofandopentodiversemeth-odsmethodologicalchoicesareconstrainedbyspecializedtrainingdatascarcityandproblemsofdataaccessibilityCareerincentiveswithinaca-demiaunfortunatelybyencouragingspecializationanddoing little tofacilitatecollaborationmakeitmoredifficultforscholarstoovercomepracticalobstacles
InthisintroductorychapterwehavelaidoutourmethodologicalandpracticalconcernsandhaveindicatedthatwewillillustrateourpointswithreferencetoresearchoncollectiveactionandthecommonsPartsIIandIIIexamineseveralstrategiesutilizedinresearchoncollectiveactionrelatedtonaturalresourcesForeachresearchstrategy
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 25
1 weprovideabroadoverviewofthemethod2 wereviewthecontributionsofthemethodtothestudyofcollective
actiononthecommonsand3 wediscussthemethodrsquosstrengthsandweaknesseswhenamethod
isparticularlyvaluableandrefertocomplementarymethods
Wealsodiscuss(inchapter8)somerelativelynewresearchapproachesthat combine formal theoretical methods with data derived from casestudies participatory research and experimental research so as to di-rectlyassessthecapabilityoftheformalmodeltogeneratesimilarpat-ternsofoutcomes
Thus we will address some of the basic concerns related to the useofaparticularmethod includingthefollowing theassumptionsusedtheanalyticalstrategywhetherthemethodhasinternalorexternalva-lidityandcanbereplicatedthepotentialcontributionsofthismethodtotheorydevelopmentandsomepracticalconsiderations(seetable11for anoverviewof these concerns)Weprovide references to textsonparticularmethodsandmethodologicalissuesandwediscusspragmaticconsiderations that influence methodological practices but we do notoutlinehowtoapplyanyspecificmethodInotherwordsthisbookisnotaldquomethodstextbookrdquoInsteadwefocusonwhathasbeenlearnedinabroadresearchprogramthroughtheuseofadiversityofmethodsTheunderlyingissuesarediscussedingeneraltermsexamplesaredrawnpri-marilyfromworkoncollectiveactionfornaturalresourcemanagement
Part II looksmorecloselyatmethodsused inempirical researchre-latedtonaturalresourcesincludingcasestudiesmeta-analysesofcasestudiesandlarge-Nandcollaborativefield-basedempiricalresearchInchapter2wefirstprovideabriefoverviewoftheconventionaltheoryofthecommonsandthenevaluatethecontributionsofcasestudiestotheo-riesofcollectiveactionanddiscusstheirlimitationsChapter3revealsthatdespiteimportantbroadlycomparativeandsyntheticpublicationsoncollectiveactionfornaturalresourcemanagementcasestudiesandsmall-Nstudiesdominatedarticlespublishedonthistopicbetween1990and2004Most large-Nstudiespublishedduring thisperiodanalyzedsurveydatadrawnfromasinglecountryorevenasinglesubnationalre-gionConsequentlytheselarge-Nstudiesofferscantimprovementinex-ternalvalidityandarenotwellsuitedforresearchrelatedtotheprospectsforcollectiveactionAsdiscussedinchapter4meta-analysisallowsformorebroadly comparativeanalysisbymaking structuredcomparisonsbasedonalargenumberofexistingstudiesMeta-analysisisconstrainedbythebodyofexistingempiricalresearchhoweverandcannotsubsti-tute forbroadlycomparativefield-basedresearchChapter5considers
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
26 bull Chapter1
Table11Methodologicalconcernsandstrategies
Methodologicalconcerns Researchstrategies
Assumptions SingleormultiplecausalpathstooutcomeDeterministicorprobabilisticrelationshipsUniversalorcontingentrelationshipsComparabilityofcases(unithomogeneity)IndependentorinterdependentobservationsRandomassignmentorrepresentativesample
Analyticalstrategy(evidence Controlleddesignorstatisticalcontrolofcausality) Processtracing
Correlationsandanalysisofvariation
Formofvalidityand Internalorexternalvalidityreplicability Easeofreplicabilityoffindings
Potentialcontributionsto Conceptdevelopmenttheorydevelopment Deductiveorinductivetheorydevelopment
Practicalconsiderations Dataissuesaccessavailabilityoflargedatasetsgapsinsourcematerial(missingdata)dataqualitymdashconsistencyaccuracy
Coststravelmdashcostseasedifficultyofmovementfieldexpenseslabmdashavailabilitycosttorunpayoffshigh-endcomputer
Recognitionandorinterpretationofdata(egpotentialunitofcollectiveaction)
Skillslanguageskillslocal(case-specific)knowledgeanalyticalskills(includingQCAorComputer-AssistedQualitativeDataAnalysis[CAQDAS])programmingskillsstatisticalskills
Attributesofresearcherssizeofresearchteamcompositionmultipledisciplines
collaborationasastrategyforbroadlycomparativefield-basedresearchTherewasrelativelylimitedevidenceofcollaborativeresearchinarticlespublishedfrom1990to2004andcollaborativeresearchwasnotmorebroadly comparative than single-authored research was We examineseveral examples of collaboration highlighting strategies developed toovercomepracticalconstraintsandtheoreticalcontributionstothestudyofcollectiveactionfornaturalresourcemanagement
PartIIIdiscussesexperimentalresearchfieldlaboratoriesandformalmodeling approachesGame theoryhasbeen the formal approach for
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption
Copyrighted Material
OvercomingChallenges bull 27
thestudyofcollectiveactioninthepastandwillremainanimportantmethodfortheyearstocomeChapter6focusesonexperimentstostudyhowsmallgroupsmakedecisionsincollective-actionsettingsEarlyex-perimentsshowedthatpredictionsfromgametheorywerenotconfirmedinmanysocialdilemmaexperimentsLaterexperimentshavestimulatedthedevelopmentofanupdated theoryofcollectiveactionChapters7and8discuss the emerginguseofagent-basedmodeling (ABM)asanalternative formalmodelingapproach forcollectiveaction (MillerandPage2007TesfatsionandJudd2006)Thebasicpremiseofagent-basedmodeling is that the macrolevel consequences of many microlevel in-teractionscanbe investigated ItputsmoreemphasisonheterogeneityamongtheactorscognitiveconstraintsandthetopologyofinteractionThefirstgenerationofagent-basedmodelsfocusedonbigquestionsinatheoreticalperspectivesuchasldquoHowdoessegregationemergerdquoandldquoWhendoegoistscooperaterdquo(Axelrod1984Schelling1978)Althoughmostmodelsofthefirstgenerationhavebeeninspiredbyobservationofrealbiologicalandsocialsystemsthemajorityofthesemodelsarenotrigorously tested on empirical data In fact the founding agent-basedmodelingeffortsdonotgobeyondaldquoproofofconceptrdquoHoweverthisischangingsinceanincreasingnumberofscholarsarestartingtoconfronttheirmodelswithempiricalobservationinmorerigorouswaysWedis-cussthesedevelopmentsespeciallyhowtheyarecombinedwithhumansubjectexperimentsandparticipatoryprocesses
Part IVoffersa synthesisChapter9distills lessonsaboutcollectiveactionrelatedtonaturalresourcesWeprovideatheoreticalframeworkofcollectiveactionandthecommonsbasedonthefindingsoverrecentdecadesofempiricalandtheoreticalresearchInchapter10weelaborateon thepractical implications for social science researchusingmultiplemethods
This book confirms that each method can make valuable contribu-tions if appliedappropriatelybutalsounderlines the limitsof relyingonanysinglemethodManyadvantagesexisttomultimethodresearchparticularly as researchmoves through successive stagesYetpracticalchallengesaresignificantThusthefinalchapterreturnstothestructuralfeaturesofacademiathatinfluencemethodologicalchoicesWeidentifyareaswhereincentivescouldbechangedtofostermoremultimethodandcollaborative research aswell as the sort of interdisciplinary researchthatissovaluableforstudyingthemanagementofnaturalresourcesandmanyotherpolicy-relevanttopicsUltimatelythenwehopethisbookwilldrawattentiontopracticalconstraintsonresearchmethods iden-tifystrategiesforovercomingtheseconstraintsandstimulatediscussionsabouthowtoencouragetheiradoption