Balancing Quality with Output: Increasing Proposal Productivity (Iron Mountain)
Team Output and Individual Productivity
Transcript of Team Output and Individual Productivity
![Page 1: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/1.jpg)
Decoding Teams: Team Output and Individual Productivity
Mohammad Ahmadpoor and Benjamin F. Jones Northwestern University
May 22nd , 2018 Science of Team Science Conference
![Page 2: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/2.jpg)
The Rise of Teams
Ref: Wuchty, Jones, Uzzi, The increasing dominance of teams in production of knowledge, 2007
MA1
![Page 3: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/3.jpg)
Slide 2
MA1 Mohammad Ahmadpoor, 5/22/2018
![Page 4: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/4.jpg)
Research QuestionsTeam Output & Credit Assignment• How does a set of individual qualities , , … map into the team’s
output, ?
• At one extreme, team output might be a “max” process, meaning max , , … )• At other extreme, team output might be a “min” process, meaning m , , … )
• How can individual productivities be inferred when people work in teams?
• Given series of observed outcomes how can we infer the individual productivities ?• High stakes, but current individual metrics are either “team blind” or use ad hoc sharing rules
(e.g., Hirsch 2005, Ellison 2013; NAS 2015; Perry and Reny 2016, Waltman 2016, Merton 1968)
![Page 5: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/5.jpg)
Research QuestionsTeam Assembly• How should teams be organized? That is, what is the efficient match between
individuals and does this occur in practice?
• For example, consider a vs a process. These have fundamentally opposing implications for how you would organize teams and organizations (e.g., Becker 1973, Kremer 1993, Grossman and Maggi 2000, Topkis 2011)
![Page 6: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/6.jpg)
Method: Key Idea
• The Hölder Mean• is a measure of impact, • is productivity of individual , • and is team size
• Special cases• max ( → ∞)• min ( → ∞) • arithmetic mean ( 1) • geometric mean ( 0)• harmonic mean ( 1)
1
Intuition: The lower , the greater influence of lower productivity team member
![Page 7: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/7.jpg)
Potential Team Advantage,
• captures impact benefit associated with team of size , incl. advantages of aggregating effort, skill, marketing, or disadvantages via coordination costs (Wuchty et al. 2007; NAS 2015).
• Normalize by setting 1 for solo-authored work. • Thus for solo-authored work => individual productivity measured on
the scale of outcome metric. • interpreted as the impact advantage of teamwork over solo-work for
individuals that share a common individual productivity level
![Page 8: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/8.jpg)
Estimation
• Use non-linear least squares to estimate , ,
• Run separately for each field of science, technological class of patents• All 185 science & engineering and social science fields with 500 papers• All 384 tech classes with 500 papers• Restrict to papers/patents with <= 8 team members
• 24 million journal articles, 13 million authors (WOS)• 3.9 million patents, 2.6 million inventors (USPTO)
• Use citations received within 8 years for
1
![Page 9: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/9.jpg)
Result Set #1: Max or Min?
The Team Production Function
![Page 10: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/10.jpg)
The Distribution of across Fields
• All fields of science, engineering, and social science, and all technology classes of patenting
• Centers between geometric and harmonic averages
• Implication: weak links. Greater influence of lower-productivity team members
![Page 11: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/11.jpg)
The Distribution of across FieldsPapers Patents
1 for all WOS fields and 94% of patenting fields Implication: team advantage seen in prior literature not simply about higher-quality people working in
teams; exists conditional on quality of individual team members.
![Page 12: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/12.jpg)
Result Set #2: Inferring Individual Productivity
![Page 13: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/13.jpg)
Individual Productivity Estimates
log 1 Distribution for Patent Classes and WOS Fields
• Determined in light of the team-production parameters, and through for each field of science and patenting
• Implications: (a) Credit shared in non-linear ways ( 1); (b) outcome more informative for lower productivity team members ( 1)
• By construction, recall that measured on same metric as outcome metric,
• Interpretation: If everyone only did solo work, then would be arithmetic mean of an individual’s citations.
![Page 14: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/14.jpg)
Accuracy of Individual Productivity Estimates• Consider capacity to predict outcomes for out-of-sample papers and patents
• Compare predictive capacity against alternative, commonly-used measures of individual productivity (Waltman 2016):i. mean citations (no adjustment for team size);ii. mean citations per collaborator (dividing citations to each work by number of
collaborators);iii. mean citations for individual’s solo works only.
• Run regressions by field, take as success metric
![Page 15: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/15.jpg)
Prediction ResultsPapers Patents
Solo
Dual
![Page 16: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/16.jpg)
Result Set #3: Team Assembly
![Page 17: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/17.jpg)
Team Assembly
• A key organizational implication of 1 is positive assortative matching in team assembly• Do we see this kind of matching?
• Method:• Measure individual productivity purely using solo-authored work, giving
individual productivity estimates that are independent of coauthors’ productivity estimates
• Then ask who works with whom• Present ratio of (a) observed frequency of two-person pairings to (b) frequency
expected by chance.
![Page 18: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/18.jpg)
Matching ResultsPatents
Papers
![Page 19: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/19.jpg)
Summary
• Computationally-feasible method for analyzing team production, deployed across large paper and patent datasets.
• Universal regularities about team production• 1: Individuals typically combine between geometric and harmonic
average of individual productivities• 1: Large team advantage when members have similar individual
productivity• Positive assortative matching
• Tool for estimating individual productivity in team contexts
![Page 20: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/20.jpg)
![Page 21: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/21.jpg)
Extension / Future Directions• New metric for individual productivity. In follow-on project, comparing against multitude
of metrics (Waltman 2016) that have been proposed to evaluate individuals in science
• Richer team production functions (where there is data)• Multi-dimensional skills• Hierarchical teams• Effort
• Team production in other contexts• Arts/Music• Sports• Manufacturing
![Page 22: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/22.jpg)
Further Intuition & Visualization of Model Fit • Consider two-person teams and normalized measure
• Model: ∑ ,
• Raw data version: Actual citations received instead of y Mean citations received to solo work instead of
12
![Page 23: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/23.jpg)
Visualization: Model Fit and Intuition (Papers)
![Page 24: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/24.jpg)
Visualization: Model Fit and Intuition (Patents)
![Page 25: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/25.jpg)
Matching, Distribution across Fields
Calculate mean along diagonal for each field. Plot distribution of these means across fields.
Positive assortative matching in all fields
![Page 26: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/26.jpg)
Team Assembly Implications?
Consistent with 1, teams tend to avoid weak links
Implications for stratification in science? Shift toward teams may ↑ stratification c.f., Jones et al. 2008
Implications for Matthew Effect? 1 runs against M.E. to extent that the top member of
the team receives less credit But PAM consistent with signaling aspect of M.E.: co-
authorship itself is a positive signal
![Page 27: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/27.jpg)
Fitting Productivity Estimation Dist.
![Page 28: Team Output and Individual Productivity](https://reader030.fdocuments.net/reader030/viewer/2022032706/623ad62bd3cdd863e25fd8b6/html5/thumbnails/28.jpg)