Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics...

40
Astrophysicists on Twitter and other social media metrics research Stefanie Haustein & Vincent Larivière Canada Research Chair on the Transformations of Scholarly Communication École de bibliothéconomie et des sciences de l’information

description

Presentation at the Harvard-Smithsonian Center for Astrophysics, February 7, 2014, 3pm, Phillips Auditorium

Transcript of Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics...

Page 1: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Astrophysicists on Twitter and other social media metrics research

Stefanie Haustein & Vincent Larivière

Canada Research Chair on the Transformations of Scholarly Communication École de bibliothéconomie et des sciences de l’information

Page 2: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Background: bibliometrics •  publication and citations used as proxy for research

productivity and impact •  based on studies to understand structure and norms

of science •  sociological research

•  publications and scientific/academic capital •  reasons to cite

•  bibliometric research •  disciplinary differences in publication and citation behavior •  delay and obsolescence patterns

Ø  theoretical framework and legitimation for citation analysis

Page 3: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Background: altmetrics •  social media metrics as alternatives or complements to

citation analysis •  similar but more timely than citations

Ø  predicting scientific impact? •  different, broader impact than citations

Ø  measuring societal impact? •  including all research “products”

Page 4: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Background: altmetrics •  similar to bibliometrics in 1960s, little known about

meaning of social media metrics •  altmetrics are “representing very different things”

(Lin & Fenner, 2013)

•  unclear what exactly they measure: •  scientific impact? •  social impact? •  “buzz”? •  all of the above?

Page 5: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Altmetrics: increasing use •  social media activity around scholarly articles grows

5% to 10% per month (Adie & Roe, 2013) •  Mendeley and Twitter largest sources for mentions of

scholarly documents

Mendeley statistics based on monthly user counts from 10/2010 to 01/2014 on the Mendeley website accessed through the Internet Archive

Mendeley •  521 million bookmarks •  2.7 million users •  32% increase of users

from 09/2012 to 09/2013

Page 6: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Altmetrics: increasing use •  increase of Twitter use

•  230 million active users, 500 million tweets per day •  39% increase of users from 09/2012 to 09/2013 •  16% of US, 3% of world population in 2013

•  uptake by researchers •  1 in 40 university faculty member in US and UK

have Twitter account (Priem, Costello, & Dzuba, 2011)

•  9% of researchers use Twitter for work (Rowlands et al., 2011)

•  80% of Digital Humanities scholars consider Twitter relevant source of information (Bowman et al., 2013)

Twitter statistics calculated based on data from: http://www.sec.gov/Archives/edgar/data/1418091/000119312513400028/d564001ds1a.htm and http://www.census.gov/population/international/data/

Page 7: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Background

Page 8: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Background

Page 9: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Background

Page 10: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Background

Page 11: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Research questions Ø  What kind of impact do Mendeley readers and tweets reflect? •  What is the relationship between social media activity around a

document and the bibliometric variables of these documents? •  Which topics receive the most attention on Mendeley and Twitter? •  How and to what extent do researchers use social media? •  Who is engaging with scholarly material on social media sites?

What are the motivations behind this use?

Results of two case studies: •  Study I: in-depth analysis of astrophysicists on Twitter •  Study II: large-scale analysis of tweets and Mendeley readers of

biomedical papers

Page 12: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Aim of this study •  in-depth analysis of astrophysicists on Twitter

•  number of tweets, followers, retweets •  characteristics of tweets: RTs, @messages,

#hashtags, URLs •  relationship with scientific output

•  publications •  citations

•  comparison of tweet and publication content •  identify different types of conversations Ø  provide evidence of use for scholarly communication

Study I: Astrophysicists on Twitter

Haustein, S., Bowman, T.D., Holmberg, K., Peters, I., Larivière, V. (in press). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. ASLIB Proceedings.

Page 13: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Data sets & methods •  37 astrophysicists on Twitter identified by

Holmberg & Thelwall (2013) •  focus on astrophysics professors and researchers •  often bloggers, science communicators

Study I: Astrophysicists on Twitter

Page 14: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Data sets & methods •  collection of Twitter account information

Ø  heterogeneous group of Twitter users •  collection and analysis of 68,232 of 289,368 tweets

•  number of RTs per tweet •  % of tweets that are RTs •  % of tweets containing #hashtags, @usernames, URLs

•  web searches to identify person behind account •  publications in WoS journals

•  publication years: 2008-2012 •  manual author disambiguation Ø  heterogeneous group of authors

Study I: Astrophysicists on Twitter

Page 15: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Data sets & methods •  grouping astrophysicists according to tweeting and

publication behavior •  analyzing differences of tweeting characteristics

between user groups Selected astrophysicists (N=37)  

tweet rarely (0.0-0.1 tweets per day)  

tweet occasionally (0.1-0.9)  

tweet regularly (1.2-2.9)  

tweet frequently (3.7-58.2)  

total (publishing activity)  

do not publish (0 publications 2008-2012)   --   --   1   5   6  publish occasionally (1-9)   4   3   4   2   13  publish regularly (14-37)   --   5   5   3   13  publish frequently (46-112)   1   3   1   --   5  total (tweeting activity)   5   11   11   10   37  

Study I: Astrophysicists on Twitter

Page 16: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Data sets & methods •  comparison of tweet and publication content

•  limited to 18 most frequently publishing astrophysicists to ensure certain number of abstracts

•  extraction of noun phrases from abstracts and tweets with part-of-speech tagger

•  analyzing overlap of character strings •  calculating similarity with cosine per person and overall

Selected astrophysicists (N=37)  

tweet rarely (0.0-0.1 tweets per day)  

tweet occasionally (0.1-0.9)  

tweet regularly (1.2-2.9)  

tweet frequently (3.7-58.2)  

total (publishing activity)  

publish regularly (14-37)   --   5   5   3   13  publish frequently (46-112)   1   3   1   --   5  total (tweeting activity)   1   8   6   3   18  

Study I: Astrophysicists on Twitter

Page 17: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Data sets & methods •  social network analysis of conversational networks

•  56,415/15,420 connections between 11,252 users •  limited to users mentioned ≥20 times:

518 users including 32 selected astrophysicists •  coding users by type

•  visualization and analysis with Gephi •  OpenOrd layout •  community detection

•  analyzing clusters •  user types •  hashtags and noun phrases •  visualizing term co-occurrence with VOSviewer

Study I: Astrophysicists on Twitter

Page 18: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: correlations •  comparison of Twitter and publication activity and impact

Study I: Astrophysicists on Twitter

Page 19: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: characteristics Mean share of retweets and tweets containing at least one hashtag per person per group

Study I: Astrophysicists on Twitter

Page 20: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: characteristics Mean share of tweets containing at least one user name or URL per person per group

Study I: Astrophysicists on Twitter

Page 21: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: content similarity •  overall similarity between abstracts and tweets low

•  cos=0.081 •  4.1% of 50,854 tweet NPs in abstracts •  16.0% of 12,970 abstract NPs in tweets

•  high Twitter coverage of most frequent abstract terms •  97,1% of 104 most frequent noun phrases on Twitter

Study I: Astrophysicists on Twitter

Page 22: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: content similarity •  similarity varies between cos=0.096 and cos=0.037 per user

cos=0.096 P=46 Tcol=2,832

cos=0.050 P=112 Tcol=423

cos=0.060 P=49 Tcol=1,236

Page 23: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: conversational network

Cluster 7

Cluster 1 #fb #twinkletweet #dotastro #AAS221 #cs17

python twinkletweet code van astrobetter

Cluster 2 #AstroFact #astro101 #clickers #clickers2012 #scio13 steelykid pip San Diego kiddo top stories today

Cluster 3 #gzconf #FGM #hugs #NHS #ff

hug hahahaha cake revision tea

Cluster 4 #stfc #scipolicy #rcuk #scienceisvital #scicuts

cut programme stfc item deadline

Cluster 5 #AAS218 #PS1 #NucATown #ff #astrojc

printculture post atel detection bond montreal

Cluster 6 #math #JWST #NASA #Hubble #mathed

buff fyi europa lilah brilliant blunder

Page 24: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: conversational network Study I: Astrophysicists on Twitter

n=68   n=88   n=40   n=180   n=30   n=109   n=3  

Page 25: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: conversation network Study I: Astrophysicists on Twitter

Cluster 1

Cluster 4 Cluster content •  large overlap of noun phrases •  most frequent terms appear in

all clusters today day time thank year

earth person science planet thing

star way sun talk life

paper moon world week lot

grey nodes appear in >2 of 6 clusters

Page 26: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: conversational network

scientific careers and funding

personal; time and places

astronomy; observation

meetings and conferences; traveling

planets; observation

telescopes; observation; places

Cluster 4

Page 27: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Conclusions •  Twitter and publication activity are negatively correlated •  user groups show different tweeting behavior regarding use of

hashtags, usernames, URLs and retweeting •  low similarity between abstracts and tweets Ø  Twitter activity does not reflect publication activity

•  conversations mainly with science communicators and other astrophysicists, hardly teachers, students or amateurs Ø  communication with general public through "middlemen"

•  conversational clusters vary by user type but topics overlap Ø  astrophysicists are involved in various discussions

Study I: Astrophysicists on Twitter

Page 28: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Outlook •  study of Facebook group •  analysis of arXiv papers on Twitter •  survey of astrophysicists on Twitter Survey on Twitter use by our colleague Kim Holmberg

Please participate: http://goo.gl/S7s6e6 or https://survey.abo.fi/lomakkeet/4445/lomake.html

Study I: Astrophysicists on Twitter

Page 29: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Aim of the study •  large-scale analysis of tweets and Mendeley readers

•  Twitter and Mendeley coverage •  Twitter and Mendeley user rates •  correlation with citations

•  discovering differences between: •  documents •  journals •  disciplines & specialties

Ø  providing empirical framework to understand use of biomedical papers on Twitter and Mendeley

Study II: Biomedical papers on Twitter and Mendeley

Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (2014). Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature. Journal of the Association for Information Sciences and Technology. doi: 10.1002/asi.23101 Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (submitted). Tweets vs. Mendeley readers: How do these two social media metrics differ? IT-Information Technology.

Page 30: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Data sets & methods •  1.4 million PubMed papers covered by WoS

•  publication years: 2010-2012 •  document types: articles & reviews •  matching of WoS and PubMed

•  tweet counts collected by Altmetric.com •  collection based on PMID, DOI, URL •  matching WoS via PMID

•  Mendeley readership data collected via API •  matching title and author names

•  journal-based matching of NSF classification

Study II: Biomedical papers on Twitter and Mendeley

Page 31: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Data sets & methods Current biases influencing correlation coefficients

Ø  compare documents of similar age Ø  normalize for age differences

Study II: Biomedical papers on Twitter and Mendeley

Page 32: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Data sets & methods Study II: Biomedical papers on Twitter and Mendeley

Framework x-axis coverage of specialty on platform compared to mean coverage

y-axis correlation between social media counts and citations

bubble size intensity of use based on mean social media count rate

Page 33: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: documents

Article Journal C T

Hess et al. (2011). Gain of chromosome band 7q11 in papillary thyroid carcinomas of young patients is associated with exposure to low-dose irradiation PNAS 9 963

Yasunari et al. (2011). Cesium-137 deposition and contamination of Japanese soils due to the Fukushima nuclear accident PNAS 30 639

Sparrow et al. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips Science 11 558

Onuma et al. (2011). Rebirth of a Dead Belousov–Zhabotinsky Oscillator Journal of Physical Chemistry A -- 549

Silverberg (2012). Whey protein precipitating moderate to severe acne flares in 5 teenaged athletes Cutis -- 477

Wen et al. (2011). Minimum amount of physical activity for reduced mortality and extended life expectancy: a prospective cohort study Lancet 51 419

Kramer (2011). Penile Fracture Seems More Likely During Sex Under Stressful Situations Journal of Sexual Medicine -- 392

Newman & Feldman (2011). Copyright and Open Access at the Bedside New England Journal of Medicine 3 332

Reaves et al. (2012). Absence of Detectable Arsenate in DNA from Arsenate-Grown GFAJ-1 Cells Science 5 323

Bravo et al. (2011). Ingestion of Lactobacillus strain regulates emotional behavior and central GABA receptor expression in a mouse via the vagus nerve PNAS 31 297

Top 10 tweeted documents: catastrophe & topical / web & social media / curious story scientific discovery / health implication / scholarly community

Study II: Biomedical papers on Twitter and Mendeley

Page 34: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: correlations

Spearman correlations between citations (C), Mendeley readers (R) and tweets (T) for all papers published in 2011 (A, n=586,600), for papers with respectively at least one citation (B, n=410,722), one Mendeley reader (C, n=390,190) or one tweet (D, n=63,800), one Mendeley reader and one tweet (E, n=45,229) and one citation, one Mendeley reader and one tweet (F, n=36,068). All results are significant at the 0.01 level (two-tailed).

PubMed papers covered by Web of Science (PY=2011)

Study II: Biomedical papers on Twitter and Mendeley

Page 35: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: disciplines PubMed papers covered by Web of Science 2010-2012

Study II: Biomedical papers on Twitter and Mendeley

Page 36: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Results: specialties

x-axis coverage of specialty on platform

y-axis correlation between social media counts and citations

bubble size intensity of use based on mean social media count rate

Page 37: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Conclusions •  uptake, usage intensity and correlation differ between

disciplines and specialties Ø  social media counts from different fields not directly

comparable •  citations, Mendeley readers and tweets reflect different kind

of impact on different social groups •  Mendeley seems to mirror use of broader but still academic

audience, largely students and postdocs •  Twitter seems to reflect popularity among general public and

represents mix of societal impact, scientific discussion and buzz Ø  the number of Mendeley readers and tweets are two distinct

social media metrics

Study II: Biomedical papers on Twitter and Mendeley

Page 38: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Outlook •  before applying social media counts in information

retrieval and research evaluation further research is needed: Ø  identifying different factors influencing popularity of

scholarly documents on social media Ø analyzing uptake and usage intensity in various

disciplines Ø differentiating between audiences and engagements Ø determine roles of social media in scholarly

communication

Page 39: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

Stefanie Haustein

Thank you for your attention! Questions?

Vincent Larivière [email protected] @lariviev

[email protected] @stefhaustein

Survey on Twitter use by our colleague Kim Holmberg

Please participate: http://goo.gl/S7s6e6 or https://survey.abo.fi/lomakkeet/4445/lomake.html

Page 40: Stefanie Haustein & Vincent Larivière: Astrophysicists on Twitter and other social media metrics research

References Adie, E. & Roe, W. (2013). Altmetric: Enriching Scholarly Content with Article-level Discussion and Metrics. Learned Publishing, 26(1), 11-17. Bowman, T.D., Demarest, B., Weingart, S.B., Simpson, G.L., Lariviere, V., Thelwall, M., Sugimoto, C.R. (2013). Mapping DH through heterogeneous communicative practices. Paper presented at Digital Humanities 2013, Lincoln, Nebraska. Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I., (in press). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. Aslib Proceedings. Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (submitted). Tweets vs. Mendeley readers: How do these two social media metrics differ? IT-Information Technology. Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (2014). Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature. Journal of the Association for Information Sciences and Technology. doi: 10.1002/asi.23101 Holmberg, K., & Thelwall, M. (2013). Disciplinary differences in Twitter scholarly communication. Proceedings of ISSI 2013 – 14th International Conference of the International Society for Scientometrics and Informetrics, Vienna, Austria (Vol. 1, pp. 567-582). Lin, J. & Fenner, M. (2013). Altmetrics in evolution: Defining and redefining the ontology of article-level metrics. Information Standards Quarterly, 25(2), 20-26. Priem, J., & Costello, K. L. (2010). How and why scholars cite on Twitter. Proceedings of the 73th Annual Meeting of the American Society for Information Science and Technology, Pittsburgh, USA. Rowlands, I., Nicholas, D., Russell, B., Canty, N., & Watkinson, A. (2011). Social media use in the research workflow. Learned Publishing, 24, 183–195.