Copyright 2011 Digital Enterprise Research Institute. All rights reserved.
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge
Twitter and research impact
Marie Boran
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge2
A review of: Eysenbach, G., 2011. Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact. Journal of Medical Internet Research, 13(4), p.e12.
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge3
A little background…
Impact Factor as a measure of scientific impact:The Good, the Bad and the Ugly.
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge4
Imperial College London researcher Stephen Curry: ‘the stupid, it burns.”
http://occamstypewriter.org/scurry/2012/08/13/sick-of-impact-factors/
“dependency on a valuation system that is
grounded in falsity.”
“we need to find ways to attach to each piece of work the value that the scientific community places on it though use and citation.”
Sick of Impact Factor?
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge5
Alternative web-based social metrics Scientometrics from online social activity centred around
scholar’s work Self-publishing: blogging, uploading, tweeting, sharing Impact measured via: articles viewed, shared, downloaded,
‘retweeted’, ‘liked’, etc.
What are altmetrics?
From: altmetrics.org/manifesto
“Scholars are moving their everyday work to
the web. Online reference managers Zotero and Mendeley each claim to store over 40 million articles (making them substantially larger than PubMed); as many as a third of scholars are on Twitter, and a growing number tend scholarly blogs. These new forms reflect and transmit scholarly impact […] That hallway conversation about a recent finding has moved to blogs and social networks–now, we can listen in.
- Altmetrics.org manifestoA”
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge6
Study objectives:
Feasibility of measuring social impact/public attention to scholarly articles through social media
Relation between dynamics, timing of tweets about a scholarly article (aka tweetations) and journal citations
Evaluating accuracy of resulting metrics in predicting highly cited articles
Eysenbach (2011)
Journal of Medical Internet Research top articles, ranked by tweets
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge7
Journal of Medical Internet Research Highly-cited, open access journal Articles published between issues 3/2009 and 2/2010 Thomson Reuters 3-year impact factor of 4.7 Citation counts (SCOPUS and Google Scholar) Twitter citations or ‘tweetation” – must mention journal article
URL Only tweets with URLs linking directly to the journal article are
captured. Does not count links to blogs or newspaper articles mentioning research.
Note: Eysenbach is the editor-in-chief and publisher of JMIR
Methods
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge8
Tweets captured: all sent and archived by JMIR between July 24, 2008 and November 20, 2011
Classification: “highly-cited” articles - top 25th percentile of each issue (by citation counts)
“highly-tweeted” - top 25th percentile (ranked by tweetations)
Adjusted for increasing popularity of Twitter over time & older articles have higher citations.
Methods (cont’d)
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge9
55 articles 4208 tweetations Average 14 tweetations per article Majority of tweets published on or day
after article published (see graph) First 30 days: “network propagation
phase” 30+: “sporadic tweetation phase” Observed 80/20 rule (Pareto principle) Highly tweeted articles 11 times more
likely to be highly cited than less-tweeted articles
75% of highly tweeted articles were highly cited in comparison to 7% of less-tweeted articles
Results
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge10
Citation and tweetation patterns Scopus and Google Scholar citations tested for agreement Eysenbach observed “distribution […] typically observed for citations”
Results (cont’d)
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge11
First systematic, prospective, longitudinal article and journal-level investigation of how mention (citations or tweetations) of scholarly articles in social media accumulate over time
First study correlating altmetrics to citations Online buzz around articles is measurable Tweets are “surprisingly accurate” predictors of future journal
citations
Findings
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge12
Complementary, *not* a replacement for Impact Factor “Tweetations” as buzz, attentiveness, social impact
Limitations
Via Scienceblogs.com
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge13
Proposes “twimpact factor” (twn) as metric of impact in social media, where n is cumulative number of tweetations within n days after publication
“The cumulative number of tweetations by day 7 (perhaps as early as day 3), could be used as a diagnostic test to predict highly cited articles.”
Tweetations as proxies for social impact of scientific research Can be applied to other social media and non-scholarly articles to measure
issue impact on social media user population
Twitter + metrics = wider perspective on research impact
Conclusions
+ =
Digital Enterprise Research Institute www.deri.ie
Enabling networked knowledge14
Related research
• Priem, J. & Costello, K.L., 2010. How and why scholars cite on Twitter. Proceedings of the 73rd ASIST Annual Meeting, 47(1), p.1-4.
• Priem, J. & Hemminger, B.M., 2010. Scientometrics 2.0: Toward new metrics of scholarly impact on the social Web. First Monday, 15(7)
Top Related