An Automated Content Analysis of Forestry Research: Are...

education & communication

An Automated Content Analysis of ForestryResearch: Are Socioecological ChallengesBeing Addressed?Gabriela C. Nunez-Mir, Johanna M. Desprez, Basil V. Iannone III,Teresa L. Clark, and Songlin Fei

Forests worldwide are increasingly threatened by a wide range of human-induced socioecological challenges, suchas urbanization, invasive species, and climate change. Using automated content analysis, we analyzed 14,855abstracts published in seven prominent applied forestry journals between 2000 and 2013 to determine thedegree to which these publications are providing forest managers with the information needed to address thesechallenges. We found that most articles still focus on more traditional forestry topics (e.g., silviculture and timberharvesting), with limited evidence of effort to address socioecological challenges. Although these traditional topicsare essential to forest management, framing this research within the broader context of contemporarysocioecological challenges will improve forest managers’ abilities to address these challenges and contribute tothe sustainability of forest ecosystems.

Keywords: automated content analysis, literature review, interdisciplinary, research gaps, socioecologicalchallenges, ecosystem services

F orest ecosystems provide many regu-lating, provisioning, supporting, andcultural services that are closely

linked to human well-being (Gonzalez et al.2005). Despite our reliance on forests, hu-mans have and continue to exert substantialecological strains on these systems. In thelast 200 years, land-use change has resultedin the destruction of nearly 40% of theworld’s forests (Bryant et al. 1997, Gonzalezet al. 2005). The remaining forests have ex-perienced mounting human-induced stres-sors such as fragmentation, climate change,and exotic invasion (Riitters et al. 2000,

Chornesky et al. 2005, Allen et al. 2010,Oswalt et al. 2015). Decreases in forestedareas, in conjunction with multiple human-induced stressors, may greatly diminish so-ciety’s ability to extract and use the widerange of forest-based services on which it re-lies (Gonzalez et al. 2005).

These stressors raise obvious concernsfor sustainable forest management, i.e., en-suring the long-term social, economic, andecological benefits of forests. Forestry re-search can help the forest managers to facethese challenges so long as the scope of thisresearch is broad enough to encompass the

complexity of these challenges (Winkel andJump 2014). For instance, forestry researchmust investigate not only trees but also allother components of forest communities,including the ecosystem services that forestsprovide (Burley et al. 2001). Forestry re-search must also contribute to the develop-ment of viable management and policy prac-tices to address human-induced stressors(Committee on Forestry Research 1990,Gonzalez et al. 2005, National ResearchCouncil [NRC] 2002). Finally, the diversityof resources and ecosystem services derivedfrom forests necessitates the simultaneousmanagement of multiple resources to meetthe needs of a diverse range of stakeholders(Tanz and Howard 1991, Selin and Chevez1995, Burley et al. 2001). This multiservicemanagement paradigm underscores the im-portance of interdisciplinary research for un-derstanding human-forest interactions andthe sustainability of forest resources (NRC2002, Gonzalez et al. 2005, Innes 2005,Balmford and Cowling 2006). Here we de-fine (sensu Innes (2005)) “interdisciplinary”as interactions among scientists from histor-ically distinct disciplines with the aim of ad-

Received December 18, 2015; accepted July 29, 2016; published online December 8, 2016.

Affiliations: Gabriela C. Nunez-Mir, Purdue University. Johanna M. Desprez, Purdue University. Basil V. Iannone, Purdue University. Teresa L. Clark, PurdueUniversity. Songlin Fei ([email protected]), Purdue University, Forestry and Natural Resources, West Lafayette, IN.

Acknowledgments: We thank Nathan Claus for assisting in the compilation of the text used for this analysis. The project was partially supported by the USDA NationalInstitute of Food and Agriculture (11207356 and IND011531MS), by a MacroSystems Biology Grant from the National Science Foundation (1241932), and bythe Ecological Sciences and Engineering Program at Purdue University.

RESEARCH ARTICLE

Journal of Forestry • January 2017 1

J. For. 115(1):1–9http://dx.doi.org/10.5849/jof.15-144

Copyright © 2016 Society of American Foresters

dressing a problem of common interest andas research that spans multiple research dis-ciplines.

These research directions have been andcontinue to be recognized as important tothe field of forestry (Committee on ForestryResearch 1990, Burley et al. 2001, NRC2002, Gonzalez et al. 2005) and have subse-quently been pursued by many research pro-grams. Further illustrating this need is thecontinued call from academic, government,and private forestry communities for greaterincorporation of social and economictraining into academic forestry programs(O’Hara and Salwasser 2015, Sample et al.2015). Given this importance, it is impera-tive that the findings of the interdisciplinaryresearch programs that address contempo-rary, human-induced stressors (hereafter re-ferred to as “socioecological challenges”) toforest sustainability reach forest managers.This information will help to ensure thatmanagers are equipped with the most up-to-date information when devising manage-ment and policy strategies and help to en-sure that these strategies are placed in thebroader context of the many complex, socio-ecological challenges impacting forests.

One of the primary ways by which sci-entific information is disseminated to prac-titioners is through scientific journals havingan applied focus. Many such journals exist inthe field of forestry research. These journalsaim to transfer scientific information per-taining to a wide range of forestry-relatedtopics to land managers in ways that help toguide management efforts. Evaluating thedegree to which research published in thesejournals addresses the many human-inducedstressors impacting forests or the many eco-system services that forests provide wouldhelp to determine whether forest managers

are receiving the information needed to ad-dress these challenges. Such an evaluationwould help forest researchers (i.e., scientistsand applied forestry journals) to identify re-search strengths and gaps and to determinethe actions that they may want to take toensure that managers are provided with theinformation required to manage forests sus-tainably in the face of contemporary socio-ecological challenges.

We conducted an evaluation of the de-gree to which these elements are included inthe abstracts of articles published between2000 and 2013 in seven prominent appliedforestry journals. Our main objectives wereto determine: (1) the scope of research beingconducted, (2) the prevalence of ecosystemservices-related research, (3) the extent towhich relevant socioecological challenges arebeing addressed, and (4) the presence ofpractical management and policy recom-mendations. To accomplish these, we used

automated content analysis (ACA), a ma-chine learning-based approach that is ofgreat utility for reviewing massive amountsof literature (Smith and Humphreys 2006,Blei 2012, Nunez-Mir et al. 2016). This toolhas been used extensively for reviews inother fields (Chen and Bouvain 2009,Cretchley et al. 2010, Travaglia et al. 2011),including forestry (Nunez-Mir et al. 2015).The findings of this review should help re-searchers better inform forest managementand policy.

Methods

Selection of Literature/JournalsIn March 2014, we used SCOPUS to

obtain a total of 14,855 abstracts publishedbetween 2000 and 2013 from the followingforest science journals: Annals of Forest Sci-ence (AFS), Canadian Journal of Forest Re-search (CJFR), European Journal of ForestResearch (EJFR), Forest Ecology and Manage-ment (FEM), Forest Science (FS), Journal ofForestry (JF) and Forestry (FOR). We chosethese journals based on the criteria describedbelow, with the goal of surveying the litera-ture published in journals with a high poten-tial influence on the forestry community, along history of publication, and broad self-identified aims and scopes (Table 1). Thecriteria used for journal selection were thefollowing: (1) high impact factor relative tothose of other forestry journals as designatedby International Scientific Indexing (ISI),(2) a long history of publication, (3) an ex-plicit aim to attract a diverse readershipacross the field of forestry, (4) an explicit aimto publish on the wide range of both tradi-

Management and Policy Implications

Forests are impacted by many socioecological challenges (i.e., human-induced stressors, such asurbanization, climate change, invasive species, and others) that hinder their ability to provide importantecosystem services sustainably. A key role of forest researchers is to provide forest managers with theinformation required to address these challenges, with applied forestry journals potentially serving as achannel for information transfer between these two parties. Nevertheless, we found that much of theresearch published in these journals focuses on more traditional forestry topics and provides littleinformation/guidance on how to address contemporary socioecological challenges. We recommend threeways in which forest researchers can meet this need while still reporting important traditionally basedforestry research: by placing traditionally based research into the broader context of ecosystem servicesand socioecological challenges, by promoting interdisciplinary thinking through communication of howresearch findings relate to social and economic aspects of forestry, and by making sure this informationis highlighted in the abstracts of research articles to ensure its communication to even those readers justscanning an article. These recommendations will help to communicate still-needed, traditionally basedforestry research in a way that helps forest managers to address contemporary socioecological challenges.

Table 1. List of journals (abbreviations) surveyed, their associated impact factor, year offirst publication, count of abstracts from each used in our analysis, and link to theirwebpages stating their aims and scopes.

Journal Impact factor First published Abstract count

Annals of Forest Science (AFS) 1.63 1923 1,212Canadian Journal of Forest Research (CJFR) 1.56 1971 3,220European Journal of Forest Research (EJFR) 1.96 1857 721Forest Ecology and Management (FEM) 2.77 1977 7,145Forest Science (FS) 1.09 1955 882Forestry (FOR) 1.68 1927 650Journal of Forestry (JF) 1.24 1902 1,025

The associated impact factor was from Web of Science (2014). Webpages are as follows: AFS, www.afs-journal.org/about-the-journal/aims-and-scope; CJFR, www.nrcresearchpress.com/page/cjfr/editors; EJFR, www.springer.com/life�sciences/forestry/journal/10342; FEM, www.journals.elsevier.com/forest-ecology-and-management; FS, http://www.eforester.org/Main/Library/Publications/Forest_Science/; FOR, www.oxfordjournals.org/our_journals/foresj/about.html; JF, http://www.eforester.org/Main/Library/Publications/Journal_of_Forestry/.

2 Journal of Forestry • January 2017

tional and emerging issues affecting forestry,and (5) an interest in informing forest man-agement and/or policy. Impact factor valuesaim to quantify the importance of a journalto the academic community studying agiven topic (Garfield 1994). Together withlength of publication history, journals thatsatisfy these criteria are likely to be influen-tial to the field of forestry. These criteriawere used as a method of objectively identi-fying such journals. Certainly, there areother influential applied forestry journals;however, an analysis including all journals,although interesting and potentially reveal-ing, falls beyond the scope of our review. Weincluded the breadth of readership and con-tent in our journal selection criteria as theyare likely to identify the most comprehen-sive venues for forestry research. We madethe assumption that managers use such ven-ues to quickly access knowledge on a widearray of topics. For this same reason and tonot introduce unwanted bias by inordi-nately featuring specific topics, we did notinclude journals having more narrowly fo-cused scopes (e.g., Forest Pathology or Agro-forestry Systems).

Automated Content Analysis (ACA)Before conducting the ACA, we ran-

domly selected and read a subset of 200 ar-ticles distributed equally among the sevensurveyed journals to gain a comprehensiveunderstanding of the topics being publishedin these journals. This understanding en-sured that the questions we formulated bothhelped us to meet our objectives and weregermane to the forestry literature (e.g., Fazeyet al. 2005). To answer these questions usingACA, we had to designate the categories bywhich we would classify the reviewed litera-ture. In ACA, these categories are called con-cepts (defined below). As an example, guidedby the understanding obtained from ourpreliminary review, we selected 23 concepts,each representing a distinct research areawithin forestry, to answer the question“what are the most researched fields in for-estry science?” From our preliminary review,we formulated seven questions and 35 cor-responding concepts that would provide usthe necessary information to fulfill our mainobjectives (Table 2).

We conducted the ACA on all 14,855abstracts using Leximancer (Leximancer PtyLtd., Brisbane, Australia). This program is aword mining software system that uses sta-

tistical algorithms, machine learning, andbootstrapping methods to discover and de-fine concepts and to identify relationshipsamong concepts (Smith and Humphreys2006). Unlike single words, concepts repre-sent groups of strongly interrelated wordsthat collectively represent a common themeor idea (Smith and Humphreys 2006). Bydetermining whether a concept is or is not ina body of text, ACA can provide quantitativeoutput regarding the abundance and inter-relatedness of concepts. There were two keybenefits to using ACA over a manual reviewof articles. First, it allowed us to review amuch larger body of literature than is possi-ble with a manual review. Second, it helpedus to limit issues associated with human sub-jectivity that may occur when one is tryingto determine whether a particular article ad-dresses a given topic or research question.

ACA is a three-staged process, detailedby Nunez-Mir et al. (2016), as per the fol-lowing summary:

1. Concept identification—Potential con-cepts are “identified” through the estab-lishment of concept seeds, i.e., frequentlyused single words that are likely to repre-sent major themes or ideas in the sur-veyed text (Smith 2003). These conceptseeds can be generated automatically bythe software used to conduct the ACA(i.e., unsupervised seeding) or can beprovided manually by the user (i.e.,hand-seeding).

2. Concept definition—Each concept is de-fined by a group of concept seeds. Usingthe group of concept seeds established foreach concept in the first stage, the soft-ware then constructs a thesaurus, i.e., alist of weighted single words that co-occur frequently by which each conceptis defined.

3. Text classification—The surveyed text isthen classified by concept at high resolu-tions. The text is divided into segments,each comprising a predetermined num-ber of text lines (usually 2–3 lines). Eachtext segment is then scanned for conceptseeds. If the summed weights of detectedseeds for a given concept’s thesaurussurpass the classification threshold, thetext segment is determined to containthe concept and is therefore classified un-der it.

To establish the concepts by which theACA program would classify text segments,we started the ACA process by performingunsupervised automated concept seed iden-tification. In unsupervised automatic con-cept seed identification, the seeds used toidentify the majority of concepts are dictatedby the concepts and words already predom-inantly present within the abstracts re-viewed, rather than by our own choosing.On further inspection of concept seeds iden-tified by the program for the concepts listedin Table 2, we found that the program hadnot automatically identified concept seedsfor several concepts of interest to our study,probably because these concepts were eitherdiscussed rarely or possessed relatively ab-stract definitions. We manually added con-cept seeds (i.e., hand-seeding) to identifythese missing concepts. The complete list ofconcepts and their respective concept seedsused for our analysis can be found in Sup-plemental Table S1.

Although the accuracy of ACA systemshas been confirmed (Grech et al. 2002,Smith and Humphreys 2006, Penn-Ed-wards 2010), it is still necessary to ensure thecorrect definition and classification of allconcepts, particularly those that are hand-seeded and/or that have abstract definitions(Nunez-Mir et al. 2016). To do so, we im-plemented a three-stage iterative validationprocess. First, we retrieved all text segmentsclassified within each of the 35 concepts andrandomly selected 10 text segments fromeach. Next, we manually verified the classi-fication of the text segments. Reviewers wereeach provided with a random selection ofclassified text segments and asked to deter-mine whether the segment was classifiedcorrectly (“yes”) or incorrectly (“no”) orwhether they could not decide due to eitherlack of expertise or insufficient informationin the text segment (“maybe”). Conceptswith less than 90% accuracy were flaggedand listed. Concept seeds of flagged con-cepts were inspected and adjusted accord-ingly through hand-seeding. Alternatively, ifconcept seeds appeared to be accurate de-spite the concept’s low accuracy, other clas-sification settings (e.g., the threshold for asegment to be classified for a concept) wereadjusted. This validation process was re-peated until each concept reached 90% clas-sification accuracy, resulting in 27 of the 35initially identified concepts relying on some

Supplementary data are available with this article at http://dx.doi.org/10.5849/jof.15-144.


http://dx.doi.org/10.5849/jof.15-144

degree of hand seeding to be correctly iden-tified (Table 2). Of the 299 concept seedsused to identify concepts (Supplemental Ta-ble S1), 71% were hand-seeded.

After validating the accurate identifica-tion of concepts, we determined the numberof text segments in which all concepts weredetected and used this information to assessthe proportion of the literature reviewed ex-ploring each concept over time and by jour-nal. This output revealed the scope of thereviewed articles and the degree to whichthese articles explored topics reflective ofecosystem services, socioecological chal-lenges, and practical management and pol-icy recommendations. This output also al-lowed for cross-journal comparison and forthe detection of temporal trends.

ResultsWe found seven journals that followed

the criteria described in the Methods section(Table 1). The journals selected featured im-pact factors ranging from 1.24 to 2.77. Theoldest journal selected was first published in1857, whereas the youngest was first pub-lished in 1977. From each journal, we har-vested between 650 and 7,145 articles, themajority of articles coming from FEM.

Scope of Forestry ResearchWe found that traditional technical or

production-based fields were the predomi-nant focus of the seven applied forestry jour-nals surveyed. On average, the five mostfrequently studied topics identified in ab-stracts over time were stand structure

(17.3%), silviculture (15%), soil (10.5%),modeling (10.3%), and nutrient cycling(6.2%) (Table 3; Figure 1A). With the ex-ception of fire ecology (5.9%) and harvest-ing (5.1%), all other general fields werefound in less than 5% of the reviewed text.These percentages were consistent across theyears, providing little evidence of any tem-poral shifts in the focus of research topics(Figure 1B). However, we did observe vari-ation among journals in the degree towhich they focused on each field (Figure1C). For instance, text from FS had agreater presence of modeling, text fromFOR and EJFR had a greater presence ofstand structure, and text from JF had agreater presence of silviculture.

Scope of Ecosystem Services ResearchWe found that the study of ecosystem

services has largely focused on nutrient cy-cling (average of 6%), followed by biodiver-sity and timber (averages of 2.7 and 2.1%,respectively) (Figure 2A). All other ecosys-tem service categories were addressed in lessthan 1% of the text reviewed. Research onnutrient cycling has experienced a decline ingeneral; whereas research on biodiversity hasincreased slightly; research on all other eco-system services has been temporally consis-tent (Figure 2B). Noteworthy differences

Table 2. Objectives, questions, and concepts used in our ACA to determine the status offorestry research.

Objectives Question Concept

Research scope 1. What are the most researched fields inforestry science?

Biomass/BiofuelsBiometricsCommunity EcologyEconomicsEntomologyFire EcologyForestry TechnologyGenetics1

Harvesting1

HydrologyLandscape EcologyModelingNon-Traditional Forest ProductsNutrient CyclingPathologyRemote Sensing/GISRestoration EcologySilvicultureSoil1

Stand Structure and DynamicsSustainability1

Tree PhysiologyWildlife

Scope of ecosystemservices research

2. What is the scope and extent of ecosystemservices research in forestry science?

BiodiversityNutrient CyclingTimber1

Water ServicesManagement1

Recreation1

Soil ServicesSocioecological challenges 3. To what extent has research in forestry

science addressed different aspects of/challenges to forest health?

InvasionFragmentation1

Biodiversity4. To what extent has research in forestry

science addressed urbanization in thecontext of forestry science?

Urban

5. To what extent has research in forestryscience addressed climate change?

Climate Change

6. Has the interdisciplinary barrier beencrossed in forestry science?

Interdisciplinary

Practicalrecommendations

7. Are practical management and policyrecommendations being provided?

Management1

Policy

1 These concepts were automatically identified and defined by ACA software. The others were hand-seeded; i.e., we provided thewords used to identify and define the concepts in the reviewed text, as described in the Methods section.

Table 3. Average percentage of textsegments from seven reviewed forestryjournals (Table 1) in which the top fieldsof forestry research were detected from2000 to 2013.

Field studied Percentage (%)

Stand structure 17.3Silviculture 15.0Soil 10.5Modeling 10.3Nutrient cycling 6.2Fire ecology 5.9Harvesting 5.1Landscape ecology 4.9Biomass/biofuel 4.9Community ecology 4.1Tree physiology 3.0Entomology 2.3Wildlife 2.2Genetics 2.1Hydrology 1.9Biometrics 1.7Pathology 1.6Economics 1.5Restoration ecology 1.3Sustainability 1.3Forestry technology 1.1Remote sensing/geographic

information system (GIS)1.0

Nontraditional forest products 0.1


among journals existed regarding the focusand extent of ecosystem services research(Figure 2C). For instance, the presence ofthe concept of biodiversity was approxi-mately three times greater in FEM than inother journals. Similarly, the presence of theconcept of recreation was greater in JF thanin all other journals combined.

Extent to Which SocioecologicalChallenges Have Been Addressed

The presence of concepts reflectingsocioecological challenges (e.g., climatechange, urbanization, and forest health) waslow (Figure 3A). For example, the conceptsof climate change and urbanization werepresent in less than 0.5% of the reviewedtext on average each year. In the case of as-pects and challenges to forest health, wefound biodiversity to be the most commonconcept, being present in 2.7% of the re-

viewed text each year. Invasion and frag-mentation were found in less than 1% of thereviewed text. Our results also show thatmany studies are not utilizing interdisciplin-ary approaches to address socioecologicalchallenges, as evidenced by the percentage oftext containing the concept interdisciplinaryfalling under 0.05% on average for eachyear. Despite being present in a very smallproportion of the reviewed text (less than0.5%), the concept of climate change seemsto be increasing, albeit slowly, with its pres-ence increasing from 0.3 to 0.8% from 2000to 2010 (Figure 3B). We found similar in-creases in the concepts of biodiversity (from1.7 to 3.8%) and biological invasions (from0.4 to 1.5%). Although the journals re-viewed performed comparably for these con-cepts, a few exceptions are noteworthy (Fig-ure 3C). The concepts of urbanization and

interdisciplinary were present in text fromJF up to 5 times more frequently than in thetext of other journals. Reviewed text fromFEM contained the concept of biodiversitytwice as much as did the text from otherjournals.

Policy and ManagementRecommendations

The proportion of the reviewed textclassified under policy was less than 0.7%(Figure 4A), with little annual change (Fig-ure 4B). In contrast, we found that an aver-age of 11.5% of the reviewed text from eachyear contained the concept of management(Figure 4B). This pattern was found to beconsistent over time and across journals,with the notable exception of JF, in whichthe concept of management was found in40% of the reviewed text, almost 4 times theproportions found in the other journals

Figure 1. Assessment of the scope of forestry research reviewed as revealed by an ACA of all abstracts published from 2000 to 2013 inseven prominent forestry journals (Table 1). A. Bars represent the percentage of abstract text segments classified under each concept acrossall journals. B. Lines represent percentage of abstract text segments classified under each concept over time. C. Stacked bars represent thepercentage of abstract text segments classified under each concept out of the total number of text segments analyzed for each journal (doesnot add up to 100%).

Figure 2. Assessment of the scope and extent of ecosystem services research in the forestry literature reviewed as revealed by an ACA ofall abstracts published from 2000 to 2013 in seven prominent forestry journals (Table 1). A. Bars represent the percentage of abstract textsegments classified under each concept across all journals. B. Lines represent percentage of abstract text segments classified under eachconcept over time. C. Stacked bars represent the percentage of abstract text segments classified under each concept out of the total numberof text segments analyzed for each journal (does not add up to 100%).


(Figure 4C). In addition, text from JF andFS also showed slightly higher percentagescontaining the concept of policy in compar-ison with other journals (3.5 and 1.3%,respectively).

Discussion

Overall FindingsWe found that the majority of the re-

viewed text from the 14,855 abstracts har-vested focused on more traditionally basedforestry topics, such as stand structure, silvi-culture, soil, and harvesting (i.e., on the pro-cess of growing and harvesting trees). Thisfinding is not surprising, given that thesetopics are the cornerstone of forestry. Nev-ertheless, despite the many socioecologicalchallenges currently impacting forests andforest sustainability and the interest of re-viewed journals in addressing these chal-lenges, we found little evidence that the ar-

ticles to which the abstracts we reviewedbelonged to provide such information.

Only a small proportion of abstract textexplored the topic of ecosystem services. Theecosystem services that were mentionedagain seemed more attuned to traditionalforestry research (e.g., nutrient cycling andtimber). However, biodiversity was found tobe a relatively highly explored ecosystem ser-vice, perhaps highlighting an increased rec-ognition by forest scientists of the role thatbiodiversity plays in maintaining forests orin the importance of conservation. None-theless, the overall percentage of reviewedtext focusing on biodiversity was still quitelow.

Similarly, socioecological challengeswere minimally explored in the majority ofthe abstracts reviewed despite the vastnumber of socioecological challenges im-pacting remaining forests (e.g., Riitters et al.

2000, Allen et al. 2010, Oswalt et al. 2015).This finding is perhaps surprising, given thepotential negative effects of these challengeson the more traditional aspects and servicesof forests (e.g., silviculture and timber pro-duction). Despite a heightened awareness inthe public and scientific community of thepotential threat of climate change and in-vasive species to forest resources, includ-ing timber production (Sohngen andMendelsohn 1998, Holmes et al. 2009),their mention remained low and only ex-hibited a slight increase over time. Thelow frequency with which forest fragmen-tation and urbanization are discussed de-spite their known negative effect on forestsresources (Aizen and Feinsinger 1994,Faulkner 2004, Nowak and Walton 2005,Melo et al. 2010) is another example.Likewise, there was a near absence of theconcept of interdisciplinary despite the

Figure 3. Assessment of the extent of research related to socioecological challenges in the forestry literature reviewed as revealed by anACA of all abstracts published from 2000 to 2013 in seven prominent forestry journals (Table 1). A. Bars represent the percentage ofabstract text segments classified under each concept across all journals. B. Lines represent percentage of abstract text segments classifiedunder each concept over time. C. Stacked bars represent the percentage of abstract text segments classified under each concept out of thetotal number of text segments analyzed for each journal (does not add up to 100%).

Figure 4. Assessment of the extent to which reviewed article abstracts attempt to provide practical recommendation for forests managementand policy as revealed by an ACA of all abstracts published from 2000 to 2013 in seven prominent forestry journals (Table 1). A. Barsrepresent the percentage of abstract text segments classified under each concept across all journals. B. lines represent percentage ofabstract text segments classified under each concept over time. C. Stacked bars represent the percentage of abstract text segments classifiedunder each concept out of the total number of text segments analyzed for each journal (does not add up to 100%).


recognized need for interdisciplinary re-search when complex socioecologicalissues of applied ecological importance areaddressed (Innes 2005, Balmford andCowling 2006).

On the other hand, our results revealedrelatively higher proportions of the analyzedabstract text addressing practical recommen-dations but with some limitations. Manage-ment was mentioned across all journals atrelatively higher proportions (�11%) com-pared with the other concepts analyzed, sug-gesting that the seven targeted journals are infact providing guidance on how to best man-age forests. Nevertheless, given the appliednature of forestry science, one might expectthis number to be even higher, as it was, forexample, in JF (40%). In contrast to man-agement, we found very little evidence ofmention of policy recommendations, reveal-ing a potential new direction for journals—the application of research findings to policydevelopment and decisionmaking.

Our findings suggest that forest re-searchers could be doing more to communi-cate the information needed to make in-formed decisions on how to address themany complex socioecological challengesfacing forests and in doing so help to meetthe stated goals of applied forestry journals.As pointed out by Pullin et al. (2004) in thehighly applied field of conservation biology,practitioners do not often access primary re-search because it is too time-consuming tolocate, access, and read. The forestry jour-nals reviewed here not only have broaderscopes and applied relevance but also specif-ically target these individuals. Therefore,these journals could serve as gateways to in-tegrate socioecological challenges into man-agement practices and policy development.Barriers to the effective communication be-tween researchers and professional forestersand policymakers can greatly hinder the ap-plication of scientific knowledge to real-world land management (Wright 2007) andby doing so greatly affect the sustainabilityof forests. By placing research findings in thecontext of key socioecological challenges im-pacting forests and the resolution of thesechallenges, applied forestry journals and thescientists who publish in these journals havethe opportunity to facilitate this integration.

Other Factors Contributing to FindingsWhere forest researchers choose to pub-

lish may be affecting our findings. Forinstance, some may choose to publish inves-tigations on particular socioecological chal-

lenges, such as climate change, invasive spe-cies, and land-use change, in journals morefocused on these topics (e.g., Global ClimateChange, Biological Invasions, and LandscapeEcology), whereas others may be attemptingto reach broader audiences by publishing inmore general journals (e.g., Conservation Bi-ology, Ecology, and Ecological Applications).There are also journals more focused on so-cial and policy aspects (e.g., Ecology and So-ciety and Environmental Science and Policy).The high percentage of abstracts containingthe word “forest” or “forests” in the abstractsof some of these journals support this con-jecture (Supplemental Table S2). In addi-tion, because of the nature by which re-searchers are evaluated, researchers maychoose to publish in journals having higherimpact factors than those of most appliedforestry journals. Finally, what some jour-nals chose to publish may impact findings.For instance, FEM directs authors to submitmanuscripts having social, economic, pol-icy, or urban focuses to publish in alternativejournals (Table 1).

However, we have evidence that ourfindings are not solely the result of whereresearchers choose to publish or what jour-nals choose to publish. We conducted a fol-low-up ACA to determine the degree towhich the abstracts from the 2014 Interna-tional Union of Forest Research Organiza-tions (IUFRO) World Congress revealedfindings similar to those of this investigation(Supplement 3). Given the broad focus ofthe IUFRO Congress, one would expectpatterns to differ between published andIUFRO abstracts if publication choice was aleading contributor to the low amount ofresearch focused on ecosystem services andsocioecological challenges. However, the re-sults of the follow-up ACA echoed those ob-tained in the original ACA. Technical orproduction-based research was also predom-inantly featured in the IUFRO World Con-gress abstracts, whereas only a small percent-age of abstracts discussed ecosystem servicesor socioecological challenges. So althoughforestry research on ecosystem services andcontemporary socioecological challenges iscertainly being conducted, the frequencywith which it is conducted appears to remainlow.

Two other factors related to scientificcommunication may also be affecting ourfindings. First, forest researchers might notbe emphasizing the connections betweentheir research and key socioecologicalchallenges and management and policy im-

plications. For example, literature on sil-viculture, although being of direct rele-vance to forest management and policyand to ecosystem services, might simplynot be discussed within these contexts.This lack of discussion may reflect re-searchers’ limited recognition of the po-tential links between traditional researchand socioecological challenges (NRC2002). Alternatively, it may indicate dis-interest in these links to socioecologicalchallenges. Second, choices on what to in-clude in an abstract could be contributingto our findings. That is, socioecologicalchallenges and ecosystem services may bediscussed in the body of the article, butnot in the abstract.

RecommendationsEach of the journals that we reviewed

expressed in their statements of scope of in-terest in linking the findings of broadly fo-cused forestry research, including on the so-cioecological challenges addressed here, toreal-world forest management (Table 1).Achieving this goal is obviously challengingand requires that research papers communi-cate important information to forest manag-ers regarding not only the socioecologicalchallenges that may hinder forests from pro-viding important ecosystem services but alsothe interdisciplinary aspects of these chal-lenges and the potential management andpolicy recommendations for addressingthem. Certain journals are already makingstrides in this direction. For instance, JF re-quires authors to include a “Managementand Policy Implications” section after theabstract in which implications to forestersand policymakers are described in practicalterms. On the other hand, FS classifies arti-cles focused on applied research with practi-cal implications for forestry professionalsunder the “Applied Research articles” desig-nation (as opposed to “Fundamental Re-search articles”). Ideally, forest managerswould be able to access a broad array of jour-nals, including those that specialize or aremore likely venues for research on socioeco-logical challenges. However, because of thelimited time and resources forest managersmay be prone to utilizing mostly or solelyapplied forestry journals. Therefore, we sug-gest some approaches by which forest re-searchers, when publishing in applied for-estry journals, may better communicate thisinformation. In turn, these recommenda-tions may help to expand readership beyondmore traditionally focused individuals of the


forest management and research communi-ties, for instance, to include social scientistsinterested in studying the political or cul-tural aspects of forest management practices.Such an expansion in readership may facili-tate the interdisciplinary research needed toaddress these complex challenges (Innes2005, Balmford and Cowling 2006, Winkeland Jump 2014). These recommendationsare not new, as their need has been notedrepeatedly by others (Committee on For-estry Research 1990, Burley et al. 2001,NRC 2002, Gonzalez et al. 2005), includingwithin the scopes and aims of reviewed jour-nals (Table 1). These recommendations arenot meant to prompt shifts in research agen-das but rather a reframing of ongoing re-search programs, making these more achiev-able. We recommend the following:

1. Place research into the broader context ofecosystem services and the socioecologicalchallenges that may hinder these services.The fact that we found the majority ofreviewed text to be focused on more tra-ditionally based forestry topics, such asstand structure, silviculture, soil, andtimber harvesting is not at all surprisinggiven that these topics are important cor-nerstones of forestry. Nevertheless, thesetopics are directly related to ecosystemservices and to the socioecological chal-lenges that may hinder these services.Making this link more apparent when-ever possible may greatly benefit the de-velopment of future forest managementand policy practices.

2. Communicate how research findings relateto social and economic aspects of forestry,i.e., promote interdisciplinary thinking.The multifaceted nature of human-induced, socioecological challenges man-dates interdisciplinary approaches to re-solve these challenges. Earlier studies inforestry, as well as studies in the similarlyapplied field of conservation biology,have recognized this need and advocatedthe crossover of knowledge among disci-plines, especially with the social sciences(e.g., Innes 2005, Balmford and Cowling2006). Acknowledging these links mayhelp forest practitioners to better reflecton the social and economic effects oftheir management and policy decisions.

3. Highlight socioecological challenges, ecosys-tem services, interdisciplinary efforts, andmanagement and policy implications in ab-stracts. For better or worse, abstracts areof tent imes the firs t— or only—

encounter of a reader with a scientific ar-ticle. This may be particularly true forland managers, having to prioritize actualland management efforts over literaturereviews. Therefore, it is vital that the ab-stract of a research article highlights themost salient findings of an investigationaccurately to ensure the greatest real-world utility of these findings. This util-ity will probably be strengthened evenfurther if authors highlight the broaderimplications of research findings. Jour-nals can assist with this effort by increas-ing abstract word limits or requestingthis information in similarly structuredsections (e.g., the Management and Pol-icy Implications section of the JF).

ConclusionForest researchers have the opportunity

to provide forest managers and policymakerswith the knowledge needed to address socio-ecological challenges. Applied forestry jour-nals provide an important link betweenforestry researchers and forest managersthrough which knowledge can be trans-ferred. Nevertheless, our review of article ab-stracts suggests that the majority of paperspublished in these journals display limitedeffort to address these socioecological chal-lenges. The work published in these journalsfocuses on traditional topics that, althoughof great importance to forestry, are not beingpresented in a manner that facilitates the useof this knowledge by managers to addressthese challenges. Therefore, despite the factthat forest researchers are interested andlikely to be pursuing interdisciplinary workand topics related to socioecological chal-lenges, it would be greatly beneficial if re-searchers and applied forestry journalsplaced greater emphasis on communicatingresearch findings within the context of thesecomplex socioecological challenges. In thisway, the forestry research community mayhelp managers and policymakers to addressthese challenges and contribute to the sus-tainability of forests and forest resources.

Literature CitedAIZEN, M.A., AND P. FEINSINGER. 1994. Forest

fragmentation, pollination, and plant repro-duction in a Chaco dry forest, Argentina. Ecol-ogy 75(2):330–351.

ALLEN, C.D., A.K. MACALADY, H. CHENCHOUNI,D. BACHELET, N. MCDOWELL, M. VENNETIER,T. KITZBERGER, A. RIGLING, D.D. BRESHEARS,E.H. HOGG, P. GONZALEZ, R. FENSHAM, Z.ZHANG, J. CASTRO, N. DEMIDOVA, J.-H. LIM,G. ALLARD, S.W. RUNNING, A. SEMERCI, AND

N. COBB. 2010. A global overview of droughtand heat-induced tree mortality reveals emerg-ing climate change risks for forests. For. Ecol.Manage. 259(4):660–684.

BALMFORD, A., AND R.M. COWLING. 2006. Fu-sion or failure? The future of conservation bi-ology. Conserv. Biol. 20(3):692–695.

BLEI, D.M. 2012. Probabilistic topic models.Commun. ACM 55(4):77–84.

BRYANT, D., D. NEILSON, AND L. TANGLEY. 1997.The last frontier forests: Ecosystems and econo-mies on the edge. World Resources Institute,Washington, DC.

BURLEY, J., R. SEPPALA, H. EL-LAKANY, J. SAYER,AND M. KROTT. 2001. Voicing interests andconcerns: Challenges for forest research. For.Policy Econ. 2(1):79–88.

CHEN, S., AND P. BOUVAIN. 2009. Is corporateresponsibility converging? A comparison ofcorporate responsibility reporting in the USA,UK, Australia, and Germany. J. Bus. Ethics87(1):299–317.

CHORNESKY, E.A., A.M. BARTUSKA, G.H. APLET,K.O. BRITTON, J. CUMMINGS-CARLSON, F.W.DAVIS, J. ESKOW, D.R. GORDON, K.W.GOTTSCHALK, AND R.A. HAACK. 2005. Sciencepriorities for reducing the threat of invasivespecies to sustainable forestry. Bioscience 55(4):335–348.

COMMITTEE ON FORESTRY RESEARCH. 1990. For-est research: A mandate for change. NationalAcademy Press, Washington, DC.

CRETCHLEY, J., D. ROONEY, AND C. GALLOIS.2010. Mapping a 40-year history with Lexi-mancer: Themes and concepts in the journal ofcross-cultural psychology. J. Cross-CulturalPsychol. 41(3):318–328.

FAULKNER, S. 2004. Urbanization impacts on thestructure and function of forested wetlands.Urban Ecosyst. 7(2):89–106.

FAZEY, I., J. FISCHER, AND D.B. LINDENMAYER.2005. What do conservation biologists pub-lish? Biol. Conserv. 124(1):63–73.

GARFIELD, E. 1994. The impact factor. CurrentContents, 25:3–4.

GONZALEZ, P., R. HASSON, P. LAKYDA, I. MC-CALLUM, S. NILSSON, J. PULHIN, B. VAN

ROSENBERG, B. SCHOLES, A. SHVIDENKO, C.V.BARBER, AND R. PERSSON. 2005. Forest andwoodland systems. P. 585–621 in Millenniumecosystem assessment: Ecosystems and humanwell-being: Current state & trends assessment,Hassan, R., R. Scholes, and N. Ash (eds.). Is-land Press, Washington, DC.

GRECH, M.R., T. HORBERRY, AND A. SMITH.2002. Human error in maritime operations:Analyses of accident reports using the Lexi-mancer tool. Proc. Hum. Factors Ergonom. Soc.Annu. Meet. 46(19):1718–1721.

HOLMES, T.P., J.E. AUKEMA, B. VON HOLLE, A.LIEBHOLD, AND E. SILLS. 2009. Economic im-pacts of invasive species in forests. Ann. N.Y.Acad. Sci. 1162(1):18–38.

INNES, J.L. 2005. Multidisciplinarity, interdisci-plinarity and training in forestry and forest re-search. For. Chron. 81(3):324–329.

MELO, F.P.L., E. MARTÍNEZ-SALAS, J. BENÍTEZ-MALVIDO, AND G. CEBALLOS. 2010. Forestfragmentation reduces recruitment of large-


seeded tree species in a semi-deciduous tropicalforest of southern Mexico. J. Trop. Ecol.26(01):35–43.

COMMITTEE ON NATIONAL CAPACITY IN FOREST

RESEARCH, BOARD ON AGRICULTURE AND NAT-URAL RESOURCES, DIVISION ON EARTH AND

LIFE STUDIES, NATIONAL RESEARCH COUNCIL.2002. National capacity in forestry research. Na-tional Academies Press, Washington, DC.162 p.

NOWAK, D.J., AND J.T. WALTON. 2005. Pro-jected urban growth (2000–2050) and its es-timated impact on the US forest resource. J.For. 103(8):383–389.

NUNEZ-MIR, G., B. IANNONE III, K. CURTIS, AND

S. FEI. 2015. Evaluating the evolution of forestrestoration research in a changing world: A“big literature” review. New For. 46:669–682.

NUNEZ-MIR, G.C., B.V. IANNONE, B.C. PI-JANOWSKI, N. KONG, AND S. FEI. 2016. Auto-mated content analysis: Addressing the big lit-erature challenge in ecology and evolution.Methods Ecol. Evol. 7(11):1262–1272.

O’HARA, K.L., AND H. SALWASSER. 2015. Forestscience education in research universities. J.For. 113(6):581–584.

OSWALT, C., S. FEI, Q. GUO, B.V. IANNONE III,S. OSWALT, B. PIJANOWSKI, AND K. POTTER.2015. A subcontinental view of forest plantinvasions. NeoBiota 24:49–54.

PENN-EDWARDS, S. 2010. Computer aided phe-nomenography: The role of Leximancer com-puter software in phenomenographic investi-gation. Qual. Rep. 15(2):252–267.

PULLIN, A.S., T.M. KNIGHT, D.A. STONE, AND K.CHARMAN. 2004. Do conservation managersuse scientific evidence to support their deci-sion-making? Biol. Conserv. 119(2):245–252.

RIITTERS, K., J. WICKHAM, R. O’NEILL, B. JONES,AND E. SMITH. 2000. Global-scale patterns offorest fragmentation. Conserv. Ecol. 4(2):3.

SAMPLE, V.A., R.P. BIXLER, M.H. MCDONOUGH,S.H. BULLARD, AND M.M. SNIECKUS. 2015.The promise and performance of forestry edu-cation in the United States: Results of a surveyof forestry employers, graduates, and educa-tors. J. For. 113(6):528–537.

SELIN, S., AND D. CHEVEZ. 1995. Developing acollaborative model for environmental plan-ning and management. Environ. Manage.19(2):189–195.

SMITH, A.E. 2003. Automatic extraction of se-mantic networks from text using Leximancer.P. 23–24 in Proc. of the 2003 Conference of theNorth American Chapter of the Association forComputational Linguistics on Human LanguageTechnology: Demonstrations, Vol. 4. Associa-tion for Computational Linguistics, Edmon-ton, AB, Canada.

SMITH, A.E., AND M.S. HUMPHREYS. 2006. Eval-uation of unsupervised semantic mapping ofnatural language with Leximancer conceptmapping. Behav. Res. Methods 38(2):262–279.

SOHNGEN, B., AND R. MENDELSOHN. 1998. Valu-ing the impact of large-scale ecological changein a market: The effect of climate change onUS timber. Am. Econ. Rev. 88(4):686–710.

TANZ, J.S., AND A.F. HOWARD. 1991. Meaning-ful public participation in the planning andmanagement of publicly owned forests. For.Chron. 67(2):125–130.

TRAVAGLIA, J.F., D. DEBONO, A.D. SPIGELMAN,AND J. BRAITHWAITE. 2011. Clinical gover-nance: A review of key concepts in the litera-ture. Clin. Govern. Int. J. 16(1):62–77.

WINKEL, G., AND A. JUMP. 2014. Perspectives onforest conservation: Building evidence at thefrontier between policy and conservation sci-ence. Biodivers. Conserv. 23(14):3359–3372.

WRIGHT, V. 2007. Communication barriers toapplying federal research in support of landmanagement in the United States. P. 55 inProc.: International conference on transfer of for-est science knowledge and technology. USDAFor. Serv., Gen. Tech. Rep. PNW-GTR-726,Pacific Northwest Research Station, Portland,OR.


An Automated Content Analysis of Forestry Research: Are...

Documents

Transcript of An Automated Content Analysis of Forestry Research: Are...