Leahey & Moody (2014) - Sociological Innovation through Subfield Integration

7/23/2019 Leahey & Moody (2014) - Sociological Innovation through Subfield Integration

1/29

Social Currents

2014, Vol. 1(3) 228256 The Southern Sociological Society 2014Reprints and permissions:

sagepub.com/journalsPermissions.navDOI: 10.1177/2329496514540131

scu.sagepub.com

How is new and useful knowledge produced?

Classic and recent literature suggests that

spanning boundaries and pooling diverse

information from distinct knowledge domains

is essential. Adam Smith ([1766] 1982:539)

argued that when the mind is employed about

a variety of objects it is somehow expandedand enlarged. Katz and Lazarsfeld (1955:345)

noted that people who see and act on differ-

ences across groups, and bridge them, have an

advantage in detecting and developing reward-

ing opportunities. More recent scholarship

suggests that information pooled from dispa-

rate sources provides a (if not the) foundation

from which new combinations and ideas spring

(Abbott 2001; Fleming, Mingo, and Chen

2007; Hargadon 2002). The close link between

domain spanning and idea generation is cap-

tured in the term recombinant innovation

(Weitzman 1998).

Yet scholars are beginning to question the

benefits of domain-spanning in academe and

to document associated challenges. Jacobs and

Frickel (2009) show that the benefits of domain

spanning are largely unsubstantiated, so enthu-

siasm may be premature. In fact, domain-span-

ning ideas face numerous challenges. On theproducer side, it is difficult to search unfamil-

iar topics (Fleming 2001; Schilling and Green

2011), to master and adequately represent lit-

erature from distinct subfields, to

accommodate the research mores of multiple

540131 SCUXXX10.1177/2329496514540131Social CurrentsLeahey and Moodyresearch-article2014

1University of Arizona, Tucson, USA2Duke University, Durham, NC, USA3King Abdulaziz University, Jeddah, Saudi Arabia

Corresponding Author:

Erin Leahey, Department of Sociology, University ofArizona, P.O. Box 210027, Tucson, AZ 85721-0027,

USA.

Email: [email protected]

Sociological Innovationthrough Subfield Integration

Erin Leahey1and James Moody2,3

Abstract

Is domain-spanning beneficial? Can it promote innovation? Classic research on recombinantinnovation suggests that domain-spanning fosters the accumulation of diverse information and

can thus be a springboard for fresh ideasmost of which emanate from the merger of extant

ideas from distinct realms. But domain-spanning is also challenging to produce and to evaluate.Here, the domains of interest are subfields. We focus on subfield spanning in sociology, a

topically diverse field whose distinct subfields are still reasonably permeable. To do so, weintroduce two measures of subfield integration, one of which uniquely accounts for the novelty ofsubfield combinations. We find (within the limits of observable data) the costs to be minimal but

the rewards substantial: Once published, sociology articles that integrate subfields (especiallyrarely spanned subfields) garner more citations. We discuss how these results illuminate trendsin the discipline of sociology and inform theories of recombinant innovation.

Keywords

science knowledge, boundary spanning, higher education, networks, multilevel models,innovation

by guest on June 23, 2015scu.sagepub.comDownloaded from
mailto:[email protected]://scu.sagepub.com/http://scu.sagepub.com/http://scu.sagepub.com/http://scu.sagepub.com/mailto:[email protected]


2/29

Leahey and Moody 229

specialty areas (Lamont, Mallard, and

Guetzkow 2006), and to produce output that is

standard in form and content. On the audience

side, reviewers drawn from the representative

domains may evaluate the merits of a work

very differently (Lamont 2009; Lamont et al.2006; Mansilla 2006), leading to low inter-

reviewer reliability that, in turn, increases the

likelihood of rejection.

The goal of this article is to critically apply

these ideas to academic scholarship. How do

domain-spanning academic articles fare? Do

they benefit from the diverse perspective,

tools, and theories that different domains offer,

as the recombinant innovation literature sug-

gests? Or does their integrative character makethem challenging to produce and evaluate?

These questions are important to answer

because domain spanning is ubiquitous in aca-

deme; for example, interdisciplinary scholar-

ship is on the rise (Brint 2005; Jacobs and

Frickel 2009), perhaps because most research

problems lie at the intersection of established

ideas (Braun and Schubert 2003). To address

these questions, we focus on the field of soci-

ology, a diverse discipline integrating many

topics (Clemens et al. 1995) with robust yet

permeable subfields1(Moody 2004). Our study

of inter-subdisciplinary scholarship elucidates

the process of (knowledge) product legitima-

tion and increases our understanding of how

scientific knowledge evolves (Mulkay

1974:228). More immediately, our research

speaks to ongoing academic debates about the

state of social science disciplines, especially

concerns about increasing specialization and

fragmentation in sociology.This article makes contributions in terms of

theory, measurement, and the substantive

topic.Theoretically, we introduce the concept

of subfield integration and demonstrate its rel-

evance to scholarship on recombinant innova-

tion and on the evaluation of domain-spanning

work. We then develop two measures of such

scholarly domain spanning: a binary one that

measures whether domain-crossing occurs

(which we call nominal integration) and amore refined one that incorporates the distinc-

tiveness of the pairing (which we call novel

integration). Use of alternate measures allows

us to assess two mechanisms that may expli-

cate the aforementioned benefits of integra-

tion: Is it merely access to a broader audience

(i.e., members of two subfields rather than

one) that enhances value, or is the novelty of

the research implicated? While we cannotmeasure innovation directly, we assess whether

novelty is one reason why subfield integration

accrues benefits. Thus, like Fleming et al.

(2007), we distinguish between novelty (a

mechanism we tap by juxtaposing two mea-

sures of subfield integration) and usefulness

(the outcome of interest). Our results show that

there are consistent, strong, and meaningful

positive returns to subfield integration.

The Concept of Integration

Integration across disciplinesinter-discipli-

narityhas received much scholarly attention

(Abbott 2001:131; National Academies of

Science 2005; Rhoten and Parker 2006), but

integration within disciplineswhat we call

subfield integrationis much less studied.

While this likely reflects the topical homoge-

neity of many disciplines (Moody and Light

2006), it is an obvious lacuna in broad, diverse

fields like sociology where the internal driver

of new ideas might well come from integrating

subfields. This imbalance, which we begin to

rectify in this article, is surprising given that

processes that occur both between and within

disciplines are critical to adequately capturing

the the substantive heart of the academic sys-

tem (Abbott 2001:148). Like fields, subfields

produce insights, provide structure to the aca-

demic labor market (where advertisements forpositions are distinguished by both field and

subfields), and prevent knowledge from

becoming too abstract and overwhelming for

scholars (Abbott 2001).

An influential working definition of inter-

disciplinary research is adaptable to the kind of

integrative research that interests us: that

which spans or bridges subdisciplines. The

National Academies report (National

Academies of Science 2005:188) defines inter-disciplinary research as a mode of research . . .

that integrates perspectives, information, data,

techniques, tools, perspectives, concepts, and/

http://scu.sagepub.com/http://scu.sagepub.com/http://scu.sagepub.com/http://scu.sagepub.com/


3/29

230 Social Currents 1(3)

or theories from two or more disciplines or

bodies of specialized knowledge. By analogy,

we define subfield integration as the borrow-

ing or mixing of insights from two or more

subfields within a discipline. While perhaps

less stable than disciplines, subfields are the

building blocks of science (Small and Griffith

1974). Whereas disciplines form the teaching

domain of science, smaller intellectual units,nestled within and between disciplines con-

stitute the research domain of science (Chubin

1976) and thus are most relevant to a study of

scholarly innovation. Subfields are the micro-

environments for research (Hagstrom

1970:93) that provide potential interconnec-

tions (Ennis 1992:260). According to our

definition, an article on student performance

might fall cleanly within the subfield sociol-

ogy of education, but an article that examinesgender differences in student performance

might be drawing fromand speaking back

tothe research literature in two subfields,

sociology of education as well as gender,

and thus be considered integrative.

Internal divisions like subfields may be par-

ticularly important in low consensus fields

(Shwed and Bearman 2010) like sociology.

Subfields are largely distinguished by their

substantive focus of inquiry (more so thanmethodological or theoretical approaches,

where there is overlap) and typically have their

own American Sociological Association

(ASA) section, journal(s), and conferences. As

these subfields grow in number (see Figure 1),

so do the opportunities for combination and

cross-fertilizationwhat we call subfield inte-

gration. Numerous studies in the 1990s used

data on ASA section comemberships or cocita-

tion patterns to identify sociologys subdisci-

plinary structure (Cappell and Gutterbock

1992; Crane 1972; Ennis 1992). These studiesshow that while subfields are clearly distinct,

some hang together more than others, forming

a loosely coupled system. This is also apparent

in our data (for the period 19852004) as

depicted in Figure 2. Proximate fields possess

cognitive overlaps that are based largely on

ideas and subject matter (Cappell and

Gutterbock 1992; Edge and Mulkay 1975). It

is near the edges of subfields, where they over-

lap with other subfields, that innovations aremost likely to emerge (Chubin 1976).

While most scholars simply assess whether

domain spanning has occurred (Clemens et al.

1995; Jacobs and Frickel 2009), consideration of

the novelty of the pairing is important for clari-

fying innovation processes. Only a few attempts

have been made to assess the uniqueness, or rar-

ity, of combinations (Braun and Schubert 2003;

Carnabuci and Bruggeman 2009; Rosenkopf

and Almeida 2003; Schilling and Green 2011;Uzzi et al. 2013). This is ideal because it is new

connections that characterize originality

(Guetzkow, Lamont, and Mallard 2004). The

Figure 1. Growth in number of American Sociological Association sections.



4/29


most fertile creative products are drawn from

domains that are far apart (Poincare [1908]

1952) and the best conceptual metaphors are

those that create ties across great distances

(Knorr Cetina 1980). Put simply, integrative

work can be more or less innovative, depending

on the relationship between the integrated enti-

ties (Carnabuci and Bruggerman 2009:608).

Thus, for our analysis of subfields, it is possible

to examine not only whether two subfields are

spanned (which we call nominal integration)

but also how rarely they are spanned, which

gives us an indication of the novelty of the pair-

ing (which we call novel integration).2

The scholarship on domain-spanning also

tends to neglect time (see Carnabuci and

Bruggeman 2009 for an exception). When con-

sidering the effects of subfield integration, it

may be critical to assess the trajectory of each

combination. In sociology, the combination of

poverty and culture was on the wane 1985

2004, but the combination of poverty and

urban sociology was on the rise. A rare-and-

getting-rarer combination may reflect some-

thing of a dead endreflecting a combination

that used to be of interest but is no longer,

while a rare-but-increasingly common combi-

nation reflects a growth area that might signal

disciplinary excitement. Bridges do decay

(Burt 2002; Ryall and Olav 2007), and theadvantages of bridging distinct domains,

which we discuss below, decline when

Figure 2. Observed subfield integration, 19852004.Note.Edges present if subfields share more than 1.75 (Expected value).



5/29


everyone strives for them (Buskens and van de

Rijt 2008).

The Effect of Integration

Scholarship on the effect of domain-spanningis divided. This is likely because innovation is

a risky endeavor that is challenging but poten-

tially beneficial. We tailor these ideas to the

context of academe (where domain-spanning

is positively valued) and the case of sociology

(where robust subfields are clearly distincta

majority of articles fall in a single subfield

but spanning is common enough to allow

exploration) (Clemens et al. 1995; Moody

2004). As elaborated below, we expect thatalthough challenges may be confronted in the

short term, benefits of subfield spanning likely

emerge in the long term.

Challenges

Scholarship on academia has documented a

number of challenges associated with domain-

spanning research (Cummings and Kiesler

2005). On the producer side, it is difficult for

scholars to accommodate the research mores

and concepts of multiple specialty areas

(Lamont et al. 2006), to master and adequately

represent literature from distinct subfields, and

to produce output that is standard in form and

content (Bauer 1990). On the audience side,

experts in different subfields may disagree on

the merits of an article (Lamont 2009), and this

conflict may be particularly evident during the

peer-review process (Mansilla 2006).

Birnbaum (1981) found that research that doesnot fit neatly within the substantive bounds of

normal science is typically received by jour-

nal editors and reviewers with irritation, con-

fusion, and misunderstanding. This makes the

road to journal publication challenging (Ritzer

1998). Former sociology journal editors rein-

forced this point. Sociological Forum editor

(19931995) Stephen Cole (1993:337)

believed that if an author writes an article on a

relatively narrow subject . . . the chances of thearticle being accepted are significantly greater

than if the author is more ambitious.American

Sociological Review(ASR) editor (19781980)

Rita Simon (1994:34) noted that works of

imagination, innovation, and iconoclasm may

fail to receive positive appraisals from review-

ers who are good at catching errors and omis-

sions but might miss a gem, or at least the

unusual, the provocative, the outside the main-stream submission.

These challenges are likely tempered, or

partly off-set, in academic sociology, the disci-

pline we study here. First, academe (especially

of late) operates under the general presumption

that domain spanning is beneficial to science

and perhaps scientists as well (Jacobs and

Frickel 2009). This is evident from the recent

enthusiasm for interdisciplinarity (National

Academies of Science 2005), synthesis (Parkerand Hackett 2012), as well as cross-cutting

funding initiatives like National Science

Foundations (NSF) Creative Research Awards

for Transformative Interdisciplinary Ventures

(CREATIV).3Second, the rise of collaboration

in academic science, including sociology

(Hunter and Leahey 2008; Moody 2004), com-

bined with a division of labor based largely on

subfield expertise (Leahey and Reikowsky

2008), suggests that integrative sociology arti-

cles may be becoming easier to produce. Third,

unlike behavioral sciences like psychology

(which has a limited number of distinct and

highly autonomous subfields), sociology has a

plethora of subfields but comparatively weak

boundaries dividing them (Moody 2004).

To the extent that challenges remain under

these more welcoming circumstances, inte-

grative research entails risk. Given the diffi-

culties that integrative work experiences in

the review process, it likely gets rejectedrepeatedly and ends up in a low-tier journal.

Authors, ever aware of the advantages of top-

tier publication for promotion and scholarly

impact, typically submit an article to the most

prestigious journal they think it can be pub-

lished in and turn to less prestigious journals

if it is rejected (Calcagno et al. 2012). But

there is also a (perhaps small) chance that

integrative work will strike big and get

accepted in a top journal. In sociology, thethree top journals are generalist in nature:

They publish articles on any sociological

topic and strive to publish articles of interest

http://scu.sagepub.com/http://scu.sagepub.com/http://scu.sagepub.com/


6/29


to many sociologists. One top journal explic-

itly aims to publish integrative work: ASR

asks reviewers to assess whether a submitted

manuscript speaks to two or more subfields,

and if not, how it could be revised to do so

effectively.4The top journals also aim to publishthe very best, most innovative research. Thus,

we expect integration to have a curvilinear,

U-shaped effect on journal prestige. Integration

will generally have a negative effect on jour-

nal prestige, except for the most innovative

integrative articles that may be more likely to

appear in top journals.5

Of course, the biggest penalty an article

can incur is relegation to the notorious file

drawer: It never gets accepted for publication.Like almost all research on science, we rely

on archival data that is subject to selection

bias (Fleming et al. 2007:464): Unpublished

articles are not observed and may be more

integrative than published articles. We thus

cannot know whether some subset of integra-

tive articles is so difficult to frame success-

fully, and so challenging to review, that its

authors give up trying to publish it (or never

try at all). We alleviate this concern some-

what by our sampling technique, described

below, which captures elite as well as low-tier

journals; much research only investigates the

former (Clemens et al. 1995; Karides et al.

2001). We also perform sensitivity analyses

to mimic the effect that selection bias might

have on our results so we can gauge the mag-

nitude and direction of bias.

BenefitsResearch on creativity and innovation demon-

strates that drawing on ideas from diverse

domains is advantageous. Actors and organiza-

tions that span domains are exposed to diverse,

unrelated ideas that can be recombined in new

ways (Carnabuci and Bruggerman 2009). Such

new combinations produce good ideas (Burt

2004), higher quality output (Singh 2008), and

serve as a foundation for innovation (Hargadon

2002; Schumpeter 1939; Weick 1979;Weitzman 1998). As Uzzi and Spiro (2005:447)

summarize, We know that creativity is spurred

when diverse ideas are united or when creative

material in one domain inspires or forces fresh

thinking in another.

When applied to academic science, the

domains of interest are typically disciplines. A

recent study conducted by the National

Academies of Science (2005) argues thatbreakthroughs in research will increasingly

occur at the interstices of disciplines, where

substantive areas of inquiry are blended.

According to Burt (2004), scientists are stimu-

lated most by people outside their own disci-

pline; the shock of the interface is what is

interesting and sparks creativity. Bringing

together two new things and seeing their cor-

respondencewhat Knorr Cetina (1980)

refers to as making conceptual metaphorspromotes the extension of ideas and knowl-

edge (Weisberg 2006). Fleming et al. (2007)

find that working with diverse scientists and

having diverse work experiences facilitates the

generation of new ideas.

The benefits of domain spanning should

also extend to subfields. Within academe, there

exists a tight correspondence between fields

and subfields. Processes that occur both

between and within disciplines are critical to

adequately capturing the the substantive heart

of the academic system (Abbott 2001:148),

though subfields better reflect the realm of

research (Chubin 1976). Like fields, subfields

produce the same insights (albeit through dif-

ferent routes), emerge via similar processes,

provide a structure to the academic labor mar-

ket (where advertisements for positions are

distinguished by both field and subfields), and

help academics by preventing knowledge from

becoming too abstract and overwhelming(Abbott 2001). Because subfields of a single

discipline are not as diverse as a set of disci-

plines, the benefits of subfield spanning may

not be as great as the benefits of field span-

ning. However, the sheer number and variety

of subfields, especially in a discipline as

diverse and loosely coupled as sociology, sug-

gest that many opportunities for recombinant

innovation exist. By linking areas of research,

we gain tentative bridges between localknowledges and a highly creative product

(Abbott 2001). Thus, scholarship that pools

ideas from diverse subfields should generate



7/29


new and valued ideas. Arguably, the greatest

benefit in reputational work organizations

(Whitley 2000) like academe is having ones

research recognized and highly regarded by

others. While this may be difficult for integra-

tive, unconventional articles to achieve in thereview stage, it may be more likely once they

have been legitimated by publication in a peer-

reviewed journal. Thus, in the academic con-

text we study here, we expect integrative

research to be more highly valued than non-

(or less novel) integrative research, at least

after publication. Even if challenges emerge in

the review stage, integrative research may

begin to accrue benefits after it is published.

One intuitive mechanism driving such bene-fits has been uncovered by previous research; in

this article, we propose and test another. Extant

research suggests that one route to scholarly

influence is to widen ones prospective audi-

ence by appealing to multiple communities. For

example, Lamont (1987) found that Derridas

ability to speak to several publics, to make his

scholarship adaptable, and to publish in various

outlets contributed to his influence both within

and outside of France. Scholars able to master

multiple genres are more likely to gain entry to

multiple conversations (Clemens et al. 1995);

indeed, the number of different journals a scien-

tist publishes in is the largest predictor of subse-

quent H-index (Acuna, Allesina, and Kording

2012). Here, we test an additional mechanism:

Compared with non- (or less novel) integrative

research, integrative research appeals to schol-

ars because of its novelty. We are able to access

this black box by first controlling for subfield

productivity (a proxy for audience size) andthen assessing the relative impact of our two

measures of integration: If it is not merely the

spanning of subfields (nominal integration) but

also the rarity of the combination (novel inte-

gration) that makes research articles notewor-

thy, then we can conclude that our proposed

mechanism is operating.

Scrutinizing the nature of the audience

helps reconcile our two hypotheses. Those

who decide whether an article gets publishedin a given journal (editors and reviewers) and

those who cite published work (researchers)

are members of the same scholarly community

who fulfill multiple roles. Indeed, the manu-

script reviewer in stage 1 (who may find inte-

grative work challenging to review) may also

be the scholar in stage 2 (who finds integrative

work valuable and worth citing). How can we

argue that they may penalize integrative workat one stage and value it at the next? The

answer relies on different role expectations.

Peer reviewers are solicited for their expertise,

so in this capacity, scholars provide a critical

evaluation of an article, emphasizing revisable

quality over inherent quality (Ellison 2002).

Outside their reviewer role, researchers choose

the scholarship they read, and they read to

engage with it and to glean its relevance to

their own research.The overarching goals of this article are to

develop and measure the concept of subfield

integration, and to assess whether the chal-

lenges of integrative research are manifested in

the review stage (i.e., by appearing in low-

prestige journals) and whether benefits accrue

after publication (i.e., by being cited heavily

by peers). If we find that all but the most novel

integrative articles tend to be published in low-

tier journals, then we will feel confident that

the challenges of domain spanning extend to

academic sociology. If we find that not only

nominal integration but also novel integration

(articles that span rarely spanned, cognitively

distant subfields) is valued more highly by the

scientific community, then we will feel confi-

dent that we have empirically tapped a process

that is critical to scientific innovation.

Data and Method

Sample

To investigate the potential benefits of inte-

grating subfields, we examine journal articles

written by a 20 percent probability sample of

tenured and tenure-track faculty members

located in sociology departments at Extensive

Research Universities6in spring 2004. Details

are provided in Appendix A. The sampling

frame was constructed using faculty lists ondepartment Web sites, which were more up-

to-date than the ASA Guide to Graduate

Departments. We limit scholars articles to



8/29


those published on or after the first occurrence

of integrative research in Sociological

Abstracts(SA) in 1985 and to those published

on or before 2004, to give recent articles some

time to accumulate citations. This results in a

sample of 1,785 articles by 180 sociologists,housed in 99 universities. Our statistical

approach capitalizes on this multistage sam-

pling procedure, but because most scholars

produce both integrative and non- (or less

novel) integrative work,7 the research article

serves as the unit of observation.

This sampling strategy is appropriate for

the task at hand. The vast majority of peer-

reviewed research takes place at research uni-

versities (Levin and Stephan 1989). Byselecting only Extensive Research Universities,

we control for resource-based influences on

the outcomes of interest, such as productivity

expectations, and time and money for research

(Fox 1992). By selecting individuals and their

articles, we gain enough articles per person to

evaluate person- and department-level effects

and to account for unmeasured characteristics

at these levels.

Like many previous studies of inequality in

science (Keith et al. 2002; Long 1992; Reskin

1977), we study a single discipline. We do this

because disciplines differ in terms of their

stratification processes (Fox and Stephan

2001; Levin and Stephan 1989; Prpic 2002),

their degree of receptivity to boundary-cross-

ing research, and the degree to which their

work is cited in other fields (Pierce 1999). We

study sociology because the social sciences

have generally been neglected by the sociol-

ogy of science and knowledge (Guetzkow etal. 2004). Moreover, sociology sits at the

crossroads of several different disciplines; it is

embedded in multiple fields, potentially

speaking to many audiences (Clemens et al.

1995; Moody and Light 2006) making an

investigation of integrative research in this dis-

cipline particularly informative. The permea-

bility of sociologys subfields (Moody 2004)

provides a conservative test of the effects of

domain spanning: If we find effects in sociol-ogy, they are likely more pronounced in fields

like psychology and economics where sub-

fields are more insular.

This study, like most studies of academic

rewards (Allison and Long 1987; Ferber and

Loeb 1973; Fox and Faver 1985; Moody 2004;

Reskin 1978; Ward and Grant 1995; Xie and

Shauman 1998), is based only on journal arti-

cles and does not include other forms of schol-arly publication, most notably books. This is a

limitation imposed by our reliance on SA to

obtain keywords (discussed below), which we

use to measure integration. However, previous

studies have found that article productivity

correlates strongly with total productivity that

includes books, book reviews, and contribu-

tions to edited volumes (Clemens et al. 1995;

Leahey 2007; Reskin 1977, 1978). Articles and

books differ largely in terms of their methodand evidence, not substantive subfields

(Clemens et al. 1995), which serve as the basis

of the integration measure we use here.

Data Sources

Most of the data needed to construct our key

explanatory variable, subfield integration, and

other article-level variables (all described in

the next section) were obtained from the elec-

tronic database Sociological Abstracts (SA).

By entering sampled sociologists names, we

accessed all of their refereed journal articles.8

For each article, we collected classification

codes (keyword descriptors indicating disci-

plinary subfieldssee the entire list in

Appendix B), which are assigned by staff at

Cambridge Scientific Abstracts (CSAs), the

umbrella organization that manages the data-

base.9SA applies at least one and sometimes

two classification codes to each article. Whileother aspects of articles (e.g., abstracts, text,

bibliography) also indicate its content, key-

words give a good indication of each articles

substantive topic, map easily onto ASA sec-

tions that demarcate substantive areas of study,

and help structure professional identity within

the field. Because there is a fixed set of classi-

fication codes assigned by information science

experts at SA, they are easier to work with ana-

lytically than an open-ended list generated byauthors or a Web of Science algorithm (e.g.,

there is no need to construct a thesaurus for

similar terms). Classification codes have been



9/29


used extensively in previous work on socio-

logical subfields (Leahey 2006, 2007; Moody

2004) and serve as the basis for our key

explanatory variable (subfield integration) as

well as two control variables: subfield prestige

and subfield productivity. Later, we show thatour results are robust to the use of an alternate

classification scheme.

We rely on the Thomson Reuters Web of

Science to construct two outcome variables. To

assess whether integrative research ends up in

lower-tier journals, we collected data on jour-

nal prestige from Journal Citation Reports, a

comprehensive and unique resource tool that

allows you to evaluate and compare journals

using citation data.

10

To assess whether inte-grative research is more highly valued, we col-

lected data on each articles citation counts as

of 2010, from the Web of Science.

Additional data about scholars, their respec-

tive departments, and their subfields were

obtained from department Web pages, profes-

sional association directories, and curriculum

vitae(CVs), which provide data comparable to

other sources of career histories (Dietz et al.

2000; Heinsler and Rosenfeld 1987). We

obtained the prestige rating of each scholars

department from the National Research

Council (NRC) (Goldberger, Maher, and

Flattau 1995).

Measures

Key explanatory variables. We use two mea-

sures of the key explanatory variable: subfield

integration. Both measures are based on extant

keywords from SA; they are not based on theauthors own classifications of their articles.

The less precise measure is binary and cap-

tures what we call nominal integration. It

indicates whether the article was assigned key-

words that come from more than one keyword

family. A keyword family comprises a par-

ent code and several child codes. In SAs clas-

sification system (Appendix B), there are 29

parent codes containing 94 child codes. For

example, the parent code complex organiza-tion contains several child codes, including

bureaucratic structure and jobs and work

organization. If an article was assigned one

code belonging to the social control family

and one code from the complex organization

family, then it would be considered integra-

tive. If an article received two codes from the

same family, then it would not be considered

integrative. Just over one-quarter (470/1785)of the sociologists articles were classified as

integrative according to this binary measure. A

validity check suggests that the measure we

use is quite good at distinguishing integrative

from nonintegrative articles.11

The more refined measure of integration is

continuous and captures what we call novel

integration. The measure captures not only

whether two subfields are combined but also

the rarity (and thus novelty) of the combina-tion. Like similar measures used by others

(Schilling and Green 2011; Uzzi et al. 2013),

we control for chance occurrence based on the

size of each subfield by using the expected

number of integrative articles. For each non-

zero cell, we calculate novel integration as 1

[observed/expected], and where the expected

number of cross-subfield publications is calcu-

lated in the standard manner: Eij = p

i p

j T,

wherepi

is the proportion of articles in subfield

i, and T is the total number of articles pub-

lished.12

Thus, as the number of observed

cross-subfield articles approaches the number

expected by chance, the value approaches

zero, and the article is thus not very integra-

tive. When we see more cross-subfield publi-

cations than expected by chance, the article has

a low score on the novel integration scale.

When we see fewer cross-subfield publica-

tions than expected by chance, the article has a

high score on the novel integration scale.13Uniquely, our calculation of novel integration

is based on a time-sensitive co-classification

matrix, akin to the one presented in Appendix

Cwhich shows frequencies for all combina-

tions during the entire time period under study

(19852004)but restricted to the five-year

window prior to each articles publication date.

Assuming that any cross-subfield publication

is more integrative than a publication that falls

only within one category, we hard-code allsingle-category (i.e., nonintegrative) publica-

tions to just below the observed minimum

(7).14

Finally, for ease of interpretation, we



10/29


rescale the score, using percentiles, so that it

ranges from 1 to 100.

To illustrate, consider articles with two con-

trasting combinations, both published in 1995:

One is associated with the codes for Gender

(29) and Organizations (06), the other is asso-ciated with the codes for Political Sociology

(09) and Family (19). (Appendix C provides

the prevalence of each combination among all

integrative articles published between 1985

and 2004not just the articles in our sample.)

When we restrict it to the five years prior to

publication (19901995), we find that 267 arti-

cles are classified as both Gender and

Organizations. While these are both large sub-

fields, with 2,119 Gender articles and 1,431Organizations articles (out of a total of 24,521

articles), we would only expect to find 123.6

integrative articles [(2,119 1,431) / 24,521],

so the novel integration score is 1(267 /

123.6)=1.16 (84th percentile), indicating that

this is a fairly common (and not so novel) com-

bination. On the contrary, we find 33 articles

classified as both Political Sociology and

Family (containing 1,284 and 2,200 articles,

respectively), we would expect to find 115.2

articles by chance [(1,284 2,200) / 24,521],

and so the 33 observed articles in the 1990

1995 window results in a novel integration

score of 1(33 / 115.2) = 0.71 (which would

put it in the 100th percentile).

With data covering a span of almost 20 years,

and an interest in distinguishing short- and long-

term effects, it is important to account for time.

We do this in three ways. First, we construct and

use a time-sensitive co-classification matrix

(i.e., a five-year window prior to each articlespublication date) to measure novel integration,

as described above. Second, we control for year

of publication in the analyses. Third, we control

for the trend in each combinations popularity.

Two articles published in the same year could

have the same novel integration score, but their

respective combinations could be on very dif-

ferent trajectories. For example, the combina-

tion of poverty and culture was on the wane

19852004 (suggesting perhaps a dead end),but the combination of poverty and urban soci-

ology was on the rise (indicating disciplinary

excitement). To ensure that slopes are scaled

similarly across categories, we first standardize

the data to have a global mean of 0 and standard

deviation of 1, and then calculate slopes for

each category combination. We use a period-

specific slope, using two periods (before or after

1995), as a way to capture the general trend rel-evant to each articles publication date.

15

Outcome variables. To assess potential chal-

lenges of integrative work, we examine jour-

nal prestige. This is captured by the Web of

Sciences journal impact factor (JIF), calcu-

lated as the number of citations to recent

(within the past few years) articles in the jour-

nal divided by the number of articles recently

published, and thus is the average number ofcitations per article. We assign a value of

0.005 to journals that are not indexed by the

Web of Science (which indexes 1,000 social

science journals) and thus omitted from the

Journal Citation Reports.16

Impact factors are

available only from 1997 onward; because

they shift somewhat from year to year, we use

the average of the factors from 1997, 2000,

and 2003. The JIFs correlate very highly with

sociologists general perceptions, and also

with Allens (1990) assessment of journal

influence,17

which we cannot use because it is

only available for a small fraction of the jour-

nals represented in our sample.

We use a cumulative citation count to test

our second hypothesis: that integrative research

is viewed as more innovative and thus will be

more noteworthy to other scholars. Specifically,

the cumulative number of citations that each

article has received as of fall 2010 captures the

extent to which the work was useful toandvalued byother scholars.

18 Although self-

citations are not eliminated from this count,

Clemens et al. (1995:455) found these to be

rare, and there are few differences in motiva-

tion for citation to self and to others (Bonzi and

Snyder 1991). Articles that have never been

cited are given a value of 0.005, and the entire

variable is then log transformed.19

Both outcome measures rely on citations,

whichdespite various criticismsare gener-ally accepted as an indicator of an articles

impact. Certainly, citation counts may also be

reflecting the authors visibility, disciplinary



11/29


alliances, attempts to flatter potential reviewers,

or even the controversial nature of an article

(Baldi 1998; Ferber 1986; Latour 1987).

Authors may cite previous work in a casual

way, or rely on it heavily. They may think of it

highly, or dismiss it as flawed. Despite this vari-ation in what a citation signals (van Dalen and

Henkens 2005), citation counts reflect the useful-

ness of an article because it contributed, in some

way, to a subsequent work. This is largely accepted

by the academic community, which continues to

rely on citation counts when making decisions

about merit raises, promotion, and tenure

(Diamond 1986; Sauer 1988; Leahey et al. 2010).

Control variables. In addition to the trend in eachcombinations popularity, other characteristics

of subfields likely influence the impact of arti-

cles written in those areas. For example, even a

nonintegrative article may garner a fair number

of citations if the single subfield it addresses is

highly productive. There are also sharp differ-

ences in the frequency with which different sub-

fields are cited (Clemens et al. 1995:472). For

these reasons, we control for the productivity

and prestige of the subfield(s) corresponding to

each article. Subfield productivityis captured by

counting the number of articles (in thepopula-

tionof articles appearing in SA 19852004) that

were assigned each possible keyword descrip-

tor, which also helps alleviate concerns about

fluctuations in small-topic areas. For articles

that spanned more than one subfield (i.e., had

multiple keyword descriptors), we summed the

counts from the respective subfields. A rough

proxy for subfield prestige is the number of

times each subfield (keyword descriptor)appeared in (or was applied to an article appear-

ing in) American Journal of Sociology (AJS),

American Sociological Review (ASR), or Social

Forces(SF) in the year 2003.

We also control for other variables that may

confound the relationships of interest. Because

coauthored articles tend to be cited more

(Wuchty, Jones, and Uzzi 2007), we include a

control variable for sole-authoredarticles. We

also control for whether the article can be con-sidered theoretical, methodological, or both

(i.e., receiving both the theory and the method

keyword). Because disciplinary lore suggests

that qualitative work might be more difficult to

publish in top-tier journals, and that quantita-

tive work may squeeze out creativity and per-

haps integrative impulses (Blalock 1991; Ritzer

1998), we control for whether the article uses

quantitative methods. Instead of relying, asMoody (2004) did, on the number of tables pre-

sented in the article as a proxy for quantitative

methods, we use a more conservative measure,

which only classifies articles as quantitative if

they have a quantitative methods keyword

assigned to it by SA. We control for year of

publication because more recently published

articles are disadvantaged in terms of exposure

(i.e., opportunity to be cited), and also because

almost all research is cited less frequently overtime (Dogan and Pahre 1990). Last, when the

outcome of interest is an articles citation count,

we also control for JIF.

We also account for several author-level

attributes and one department-level character-

istic. Because newcomers to a field are more

likely to instigate paradigm shift (Collins

1968:136) and more established professors are

most likely to seek involvement outside their

home field (Klein 1996; Pierce 1999), we con-

trol for authors professional age (years since

PhD) and its square. We also control for

authors productivity and visibility in the field,

as these factors influence access to top journals

as well as citations. An authorsproductivityis

captured by a cumulative, time-varying count

of published journal articles. An authors visi-

bilityis captured by a binary variable indicating

whether a scholar has published inASRorAJS

to date. We also control for each scholars

department prestige using 1995 ratings fromthe NRC; we convert the scores to deciles

(ranging from 1 to 10, where higher values

indicate greater prestige) and use regression-

based imputation for unrated departments (pre-

dictors include average journal impact, average

citations, percent of faculty publishing in top

journals, and percent of faculty publishing in

JCR-rated journals).

Statistical Approach

Given the structure of our data, we use multi-

level modeling to derive estimates. The data



12/29


are inherently clusteredarticles within schol-

ars, and scholars within departmentsand

thus suitable for multilevel (i.e., hierarchical)

modeling (Raudenbush and Bryk 2002).

Specifically, the level 1 unit is the article, the

level 2 unit is the scholar, and the level 3 unitis the department. This modeling strategy

accounts for clustering by incorporating both

fixed and random effects.20

Our hypotheses

concern main effects, and we explore two

same-level interactions, but we do not specify

cross-level interactions. The models we esti-

mate (using PROC MIXED in SAS 9.3) can be

expressed as a combination of submodels, one

representing each level:

Level 1 articles ln ARTIC( ) ( ) = + +: Y eijk jk jk ijk 0

Level 2 authors AUTH( ) = + +: 0 0jk k k jkr

Level 3 departments DEPT( ) = +: 0 0k

where there are i articles belonging to j

authors in k departments. The intercept from

the article-level equation (0jk

) is not specified

as a fixed effect. Rather, it is allowed to vary

across authors (a random effect), and that vari-ation is then explained using author-level char-

acteristics, such as gender or productivity, in

the author-level (level 2) equation. Similarly,

the intercept from the author-level equation

(0k

) is allowed to vary across departments (a

random effect), and that variation is then

explained using department-level characteris-

tics, such as prestige. To estimate the model,

we combine these three equations and rear-

range terms to distinguish the fixed and ran-dom components:

ln ARTIC+ AUTH + DEPT

int

Y e rijk jk k ijk jk ( )= + + +=

0

eercept all fixed effects random effects.+ +

The combined equation looks very similar

to single-level regression model, with the

exception of the last term, which is a random

effect associated with authors. This can be

thought of as an error term, or unexplained

variance, associated with the author level. A

random effect at the department level was not

specified.

Results

Among the 1,785 articles of interest, there is

wide variation in both outcomes of interest:

JIF and the number of cumulative citations as

of 2010 (see Table 1). The JIF ranges from 0(for articles not indexed in the Web of Science)

to 23.87 (the highest impact journal is

Science), and the citation count ranges from 0

to 340. Three-quarters of the articles appear in

a journal that is rated by the Web of Sciences

JCR, and just over a quarter are integrative

according to our binary measure. Authors of

these articles are predominantly men (36 per-

cent are women), in early- to mid-career stage,

with nine published articles; over one-thirdhave experience publishing in the disciplines

top journals (ASRand AJS). Authors are dis-

proportionately employed at prestigious uni-

versities. Within this elite sample, however,

lies some significant variation between inte-

grative and nonintegrative articles (results not

shown). First, integrative pieces are written by

authors who are professionally younger than

authors of nonintegrative work. Second,

integrative articles are more likely to be pub-

lished in large and prestigious subfields.Importantly, this may buffer them from the

penalties typically associated with domain-

spanning work.

Contrary to expectations, subfield integra-

tion does not have a curvilinear U-shaped

effect on journal prestige. In fact, novel inte-

gration has no significant effect on JIF (see

Table 2), indicating that integrative research is

no less likely to appear in prestigious journals

than nonintegrative research. This suggeststhat, despite theoretical reasons to expect dif-

ficulty publishing integrative work, there

appears to be no publication-prestige pen-

alty.21The trend in the popularity of the under-

lying category or combination, however,

negatively affects impact factor, suggesting

that as topics become more popular, articles

on such topics appear less frequently in top

journals. We also find that sole-authored arti-

cles and theory articles tend to be published inlower-tier journals. Few other variables are

significantly associated with JIF. Exceptions



13/29


include department prestige and author expe-rience publishing in top journals likeASRand

AJS, demonstrating some effective path

dependency. These results are robust to the

substitution of the binary measure of nominal

integration: The sign and significance of the

coefficient remain the same, and model fit is

comparable.

Consistent with expectations, we find that

integrative research garners a greater number

of citations than non- (and less novel) integra-

tive work. The models presented in Table 3

suggest that even after controlling for

characteristics of authors, departments, andsubfields, articles that are more integrative

receive a greater number of citations. In the

model with all controls (model 3c), the coef-

ficient for novel integration is positive and

significant (+0.005**) and, considering the

scale of the novel integration measure (1 to

100), represents a reasonably sized effect. For

a 10 percent increase in novel integration, an

articles citations increase by 5 percent, and

for a 50 percent increase in novel integration,

an articles citations increase by 25 percent.

Controlling for year of publication and the

Table 1. Descriptive Statistics of 1,785 Articles Published by 180 Sociologists in 99 Departments.

Mean SD Percent 0s Minimum Maximum

Outcomes variables

Journal impact factor (mean of 1997,

2000, and 2003 scores)

0.86 1.02 23.8 0 23.87

Cumulative citation count as of 2010 19.34 30.68 18.3 0 340

Article attributes

Novel integration score (percentiles) 24.69 39.7 73.7a 1 100

Nominal integration score (binarymeasure)

0.26 73.7 0 1

Year of publication 1995 5.36 1985 2004

Trend 0.006 0.09 0.15 1.26

Sole authored 0.23 0.42 77.1 0 1

Theory 0.07 0.26 92 0 1

Method 0.06 0.25 93 0 1

Theory and method 0.002 0.05 99.7 0 1 Quantitative methods topic 0.03 97.5 0 1

Subfield(s) productivity 13,055 5,399 1,149 21,293

Subfield(s) prestige 0.68 31.7 0 1

Author attributesfor last year in data-file

Professional age (2004-PhD year) 8.91 9.65 9 39

Productivity (number of articlespublished)

8.79 9.75 1 69

Ever published inASRorAJS(yes = 1, no = 0)

0.37 63 0 1

Gender (female = 1, male = 0) 0.36 64 0 1

Department attributes NRC rating of department prestige 5.85 2.66 12c 1 10

aThis is the percent of observations that are nonintegrative, so have a score of 1.bThis is the percent of observations with the imputed score of 7, equivalent to percentile score of 1.cThis is the percent of observations whose department prestige score was predicted based on their average journalimpact, citations, percent of faculty publishing in top journals and in JCR-rated journals. This predicted value wasimputed.ASR =American Sociological Review; AJS =American Journal of Sociology; NRC = National Research Council



14/29


Table 2. Assessing Risks: Effects on Journal Impact Factor (Multilevel Model Estimates).

Model 2a Model 2b Model 2c

Coefficient SE Coefficient SE Coefficient SE

Fixed effects Article attributes

Novel integration scorea,b a,b 0.0008 0.0006 0.0008 0.0006 0.0009 0.0006

Year of publication 0.009 0.0004 0.007 0.005 0.006 0.005

Trend 0.56* 0.26 0.47**** 0.26 0.47**** 0.26

Sole authored 0.18** 0.06 0.17** 0.06 0.21*** 0.06

Theory 0.21* 0.09 0.26** 0.09 0.28** 0.09

Method 0.08 0.12 0.06 0.13 0.05 0.12

Theory and method 0.02 0.45 0.03 0.45 0.001 0.44

Quantitative methods topic 0.12 0.19 0.13 0.18 0.13 0.18

Subfield(s) productivity 0.008**** 0.005 0.0009* 0.0004 0.0009* 0.0005

Subfield(s) prestige 0.07 0.05 0.06 0.05 0.06 0.05 Author attributes

Professional age (years since PhD)b 0.02* 0.008 0.02* 0.008

Professional age2 0.0005* 0.0002 0.0004**** 0.0002

Productivity (number of articles)b 0.009* 0.004 0.009* 0.004

Ever published inASRorAJSb 0.72*** 0.08 0.62*** 0.08

Gender (female = 1, male = 0) 0.03 0.09 0.05 0.09

Department attributes

Department prestige (NRC rating) 0.07*** 0.01

Intercept 19.0* 9.17 14.36 11.22 13.09 10.77

Random effects

Variance of level 1 random effect 0.83*** 0.02 0.82*** 0.03 0.82*** 0.03 Variance of level 2 random effect 0.27*** 0.04 0.18*** 0.04 0.14*** 0.03

Sample size 1,785 1,785 1,785

2 Res log pseudo-likelihood 5,019.4 4,977.7 4,965.8

BIC 5,029.7 4,988.1 4,976.2

Note.ASR =American Sociological Review; AJS =American Journal of Sociology; BIC = Bayesian Information Criterion ;NRC = National Research Council.aOne-tailed tests (two-tailed otherwise).bTime-varying variable.*p< 0.05. **p< 0.01. ***p< 0.001. ****p0.10.

prestige and productivity of subfields, we find

that articles on topics that are increasing in

popularity actually receive fewer citations

(the coefficient for the trend variable is 2.22

and statistically significant). These effects

hold even when we control for journal pres-

tige, arguably the largest determinant of an

articles citation count. To provide a sense of

the size of the effects in model 3c, we providepredicted value plots in Figure 3 that stratify

integration (xaxis) by selected other variables

(different lines spanning data range). For ease

of comparison, we have equated the y axis

range across all panels.

The size of integrations effect is even

more apparent when we substitute in the

binary measure of nominal integration. In

model 3d, the coefficient for nominal integra-

tion (+0.41**) demonstrates that integrativearticles receive almost 41 percent more



15/29

242

Table3.

Assessing

Benef

its:

Effectson

Log

gedCumulat

ive

Citat

ions

(Multileve

lMo

delEst

imates

).

Mo

del3a

Mo

del3b

Mo

del3c

Mo

del3d

Mo

del3e

Coef

ficient

SE

Coe

fficient

SE

Coef

ficient

S

E

Coef

ficient

SE

Coef

ficient

SE

Fixe

def

fects

Art

icleattr

ibutes

Novelintegrationscore

(percent

iles)

a,b

0.005**

0.002

0.00

5**

0.002

0.005**

0.0

02

0.0005

0.002

Nominal

integrat

ionscore

a

0.41**

0.16

Yearof

publicat

ion

0.06***

0.01

0.05

***

0.02

0.05***

0.0

2

0.05***

0.02

0.06***

0.02

Trend

2.29**

0.74

2.21

**

0.74

2.22**

0.7

4

2.27**

0.75

6.91***

1.38

Soleauthore

d

0.18

0.18

0.15

0.17

0.12

0.1

8

0.12

0.17

0.12

0.18

Theory

0.96

0.27

0.99

***

0.27

1.01***

0.2

7

1.00***

0.27

1.08***

0.27

Met

hod

0.16

0.35

0.12

0.35

0.12

0.3

5

0.12

0.35

0.14

0.35

Theory

andmet

hod

2.18****

1.27

2.09

****

0.27

2.11****

1.2

7

2.08****

1.26

1.94

1.26

Quantitativemet

hodstopic

0.04

0.53

0.12

0.53

0.17

0.4

3

0.13

0.53

0.07

0.53

Subfield

(s)pro

duct

ivity

0.00004**

0.00001

0.00

03**

0.00001

0.00004*

0.0

0001

0.00004*

0.00001

0.00004*

0.00001

Subfield

(s)prestige

0.12

0.16

0.11

0.16

.11

0.1

6

0.11

0.16

0.19

0.16

Journal

prestige

(impact

factor)

1.28***

0.06

1.25

***

0.07

1.24***

0.0

7

1.24***

0.07

1.14***

0.07

Aut

horattr

ibutes

Pro

fessiona

lage

(yearssince

PhD)b

0.04

**

0.01

0.04**

0.0

1

0.04**

0.01

0.04**

0.01

Pro

fessiona

lage

2 (yearssince

PhD)b,c

Pro

ductivity

(#articlespu

blished)

b

0.01

0.01

0.02

0.0

1

0.02

0.01

0.012

0.010

Everpu

blishedinASRorAJS(yes=

1,no=

0)b

0.49

*

0.22

0.47*

0.2

2

0.47*

0.23

0.36****

0.22

Gen

der

(female=

1,male=

0)

0.09

0.26

0.19

0.2

6

0.19

0.26

0.04

0.25

Departme

ntAttri

butes

Departmentprestige

(NRCrating

)

0.02

0.0

3

0.02

0.03

0.06

0.04

Interactionterms


Journa

lprest

ige

0.005**

0.001


Trend

0.08***

0.02

Intercept

133.8***

25.4

109.1*

**

30.4

108.6***

30.3

7

108.37***

30.3

6

114.56***

30.0

Ran

domeffe

cts

Var

ianceo

flevel

1randomef

fect

(res

idua

l)

6.601***

0.23

6.62

***

0.23

6.601***

0.2

3

6.62***

0.23

6.56***

0.23

Var

ianceo

flevel

2randomef

fect

1.3***

0.24

1.17

***

0.23

1.3***

0.2

4

1.15***

0.23

1.01***

0.22

Samplesize

1,785

1,785

1,785

1,785

1,785

2Res

logp

seudo-li

kelihoo

d

8,640.4

8,644.5

8,647.3

8,638.5

8,644.4

BIC

8,650.7

8,654.9

8,657.7

8,648.9

8,650.7

Note.B

IC=

Bayes

ian

Informat

ion

Criterion

aOne-t

ailedtests

(two-t

ailedotherw

ise).

bTime-varyin

gvariab

le.

cNotsign

ificant,remove

d.

*p

Leahey & Moody (2014) - Sociological Innovation through Subfield Integration

Documents

Transcript of Leahey & Moody (2014) - Sociological Innovation through Subfield Integration