Altmetrics: Listening & Giving Voice to Ideas with Social Media Data

Anatoliy Gruzd, PhDCanada Research Chair and Associate Professor Director of Research, Social Media LabRyerson University, Toronto, Canada @GruzdGruzd@Ryerson.ca

Should scholarly use of social media be considered towards tenure and/or promotion?

Gruzd, A., Staves, K., and Wilk, A. (2011). Tenure and Promotion in the Age of Online Social Media.

Proceedings of the American Society for Information Science and Technology (ASIS&T) Conference.

Back in 2011 …

This is what academics say about Altmetrics on

Twitter

6 years later…

This is what academics say about Altmetrics on

Twitter

6 years later…

How did we get here?

Evolution in Scholarly Communication Channels

Letters

Emails

Mailing lists

Social Media

Scholarly Communication: Then and NowLetters of Edwin Gilpin, a mining engineer, government official & author (1850-1907)

Tweets of a contemporary scientist in the domain of Earth Sciences (2014)

MacDonald, B., Duggan, L., Gruzd, A, & Collins, V., Scientific Communication: Testing Historical & Present-Day Communication Networks with Social Network Analysis. Working paper.

9 months | 1300 letters | people=616 | ties=1277 1 month | 1302 tweets | people=756 | ties=1578

Popular Social Media Sites among Academics

Frequent Use

Non-academic soc.networks Blogs

Online document

management

Media repositories Wikis

Occasional Use

Presentation sharing sites

Video/teleconference Blog Wikis Academic

soc.networks

Gruzd, A., & Goertzen, M. (2013). Wired Academia: Why Social Science Scholars Are Using Social Media. The 46th Hawaii International Conference on System Sciences (HICSS): 3332-3341, DOI: 10.1109/HICSS.2013.614

Benefits of Using Social Media

0% 10% 20% 30% 40% 50% 60%

Discovering new fundingGarnering mass media attention

Publishing findingsMaintaining professional image

Soliciting advice from peersCollaborating with other researchers

Making new research contactsPromoting current work/research

Discovering new ideas or publicationsFollowing other researchers' work

Keeping up to date with topics

Gruzd, A., & Goertzen, M. (2013). Wired Academia: Why Social Science Scholars Are Using Social Media. The 46th Hawaii International Conference on System Sciences (HICSS): 3332-3341, DOI: 10.1109/HICSS.2013.614

Related benefits of social media use based on the factor analysis

Social & Info Dissemination

Information Gathering

Collaboration explains 24% of the total variance

explains 16% of the total variance

Who talks about research on social media?

• Not just academics! But also • institutions • journalists• librarians• policy makers • other groups

Unexpected Receptor Communities

As more people talk about research online, social ‘signals’ are becoming more valuable for …

• Academics – discover what peers are discussing

• Institutions & Funders –assess research impact

• Publishers - ↑readership, feature most-discussed research, discover popular topics for future calls

• ATP Committees – evaluate scholarly output / service-component

Example: Libraries & Museums Making Biodiversity Heritage Library (BHL) collections more “social”!

Google Trends for “Altmetrics” and “Altmetric”

Altmetrics is …

A set of “metrics proposed as an alternative to the widely used journal impact factor and personal citation

indices, like the h-index” (Wikipedia)

“Study and use of scholarly impact measures based on activity in online

tools and environments” (Priem, 2014)

“The creation and study of new metrics based on the Social Web for analyzing

and informing scholarship” (Adie & Roe, 2013)

Research on Altmetrics is growing… but still very young

Top 10 most prolific scholars in this area

Source: Web of Science, Sep 2017

Altmetrics: Research Topics

Common research questions:

• To what extent articles published in a journal are discussed on social media (coverage)?

• Is there a relationship between altmetricsand more traditional impact factors (correlation studies)?

Ex: among altmetrics, blog count is the strongest predictor of increased citations:

• “One more blog post discussing a publication increases the chance of more citations by 4.7%” (Hassan et al., 2017)

• Very discipline specific

• Recent review paper: Sugimoto et al., 2017

Altmetrics: Data Providers, Aggregators and Metrics

Data Providers

Aggregators Metrics

Altmetrics: Data Providers, Aggregators and Metrics

Data Providers

Aggregators Metrics

Altmetrics: Data Providers

Twitter68%

Facebook17%

Blogs7%

News6%

Google+2%

% COVERAGE OF PUBLICATIONS IN SOCIAL MEDIA

* Based on ~1M articles published between2011-2015 (indexed by Scopus) and that have atleast one citation & one social media mention(captured up until Feb 2017)(Hassan et al., 2017)

Altmetrics: Data ProvidersLack of APIs for some prominent SN platforms

Researchgate.net Academia.edu

Altmetrics: Data ProvidersLack of attention to some other SN platforms

Reddit

(Kumar et al., 2018)

Content Type n=1,227 posts (100%)

Explanation 592 (48%)Information Seeking 274 (22%)Providing Resources 260 (21%)Socializing with Positive Intent 204 (17%)Explanation with Disagreement 71 (6%)Subreddit Rules and Norms 66 (5%)Explanation with Agreement 45 (4%)Socializing with Negative Intent 4 (0%)

Altmetrics: Data Providers, Aggregators and Metrics

Data Providers

Aggregators Metrics

Altmetrics: Data Aggregators (Melero, 2015)

NISO Alternative Assessment Projecthttp://www.niso.org/apps/group_public/document.php?document_id=17090

transparency

replicability

accuracy

Altmetrics: Data Aggregators

NISO Alternative Assessment Projecthttp://www.niso.org/apps/group_public/document.php?document_id=17090

Altmetrics: Data Aggregators

Altmetrics: Data Providers, Aggregators and Metrics

Data Providers

Aggregators Metrics

Altmetrics: Metrics Examples based on a case study of measuring impact of a drug safety article published by the Canadian Network for Observational Drug Effect Studies (CNODES)with Gamble, Traynor, Gruzd, Mai, Dormuth, Sketris

Basic Indicators

Altmetrics: Metrics Example

Who tweeted about the CNODES paper?

Twitter user type # users % users

Members of the public 22 84%

Practitioners (doctors, other healthcare professionals) 3 11%

Science communicators (journalists, bloggers, editors) 1 3%

Account Type Twitter Account

Organization @bcdpicOrganization @action_designerOrganization @e24BusinessIndividual @social_club_Individual @SrinjoyOrganization @connectcontactsOrganization @StartupPortalIndividual @kekesimotOrganization @youngentre

Source: Altrmetric.com

Altmetrics: Developing Metrics based on Social Network Analysis (SNA)

Nodes = Social Media Users

Ties (lines) = Interactions

• ~10% of the 3,005 blogs analyzed cite at least 1 article from the dataset of 2,246 articles. • The most influential blogs, as measured by in-links, are written by diabetes patients and tend not to cite biomedical literature.

Gruzd, A., Black, F.A., Le, Y., Amos, K. (2012). Investigating Biomedical Research Literature in the Blogosphere: A Case Study of Diabetes and HbA1c. Journal of the Medical Library Association 100(1): 34-42. DOI: 10.3163/1536-5050.100.1.007

The Rise of Social Bots• Who are we studying:

Humans or Bots?

Social Bot – software designed to act on the Internet with some level of autonomy

Altmetrics: Metrics - Challenges

Different Types of Bots

Free music, games, books,

downloads

Jewelery, electronics,

vehicles

Contest, gambling,

prizes

Finance, loans, realty

Increase Twitter

following

DietAdult

(Grier et al, 2010)

Detecting Bots…

Detecting Bots…

Phot

o • Color & Edge histograms

• Color & Edge Directivity Descriptor (CEDD)

• Image Similarity

Mes

sage • Sensitive

words• URL• Duplicates • #hashtags• @replies

Post

er • Username• Engagement

level• Creation

date

Soci

al N

etw

ork • # Friends

• # Following• In/out degree

centrality • Clustering

(Yardi et al, ‘09; Grier et al, ‘10; Wang, ‘10; Jin et al, ’11; Varol et al, ‘17)

How to introduce these emerging techniques to altmetrics researchers and developers who are relying on social media as their go-to data source!

The challenge is ...

© Chris Allen licensed under Creative Commons

Altmetrics: Challenges & Opportunities!

• Lack of access to some data providers

• Mostly tracking social mentions based on DOIs/unique identifiers

• Reliance on different data providers

• Measuring different things

• Need for transparency, replicability & accuracy

• Noisy data and social bots

Altmetrics: Listening & Giving Voice to Ideas with Social Media Data

Anatoliy Gruzd, PhDCanada Research Chair and Associate Professor Director of Research, Social Media LabRyerson University, Toronto, Canada @GruzdGruzd@Ryerson.ca

Slides available at http://bit.ly/4amkey

References• Grier, C., Thomas, K., Paxson, V., & Zhang, M. (2010). @spam: the underground on 140 characters or less (p. 27). ACM Press.

http://doi.org/10.1145/1866307.1866311• Gruzd, A., Black, F.A., Le, Y., Amos, K. (2012). Investigating Biomedical Research Literature in the Blogosphere: A Case Study of Diabetes and

HbA1c. Journal of the Medical Library Association 100(1): 34-42. DOI: 10.3163/1536-5050.100.1.007• Gruzd, A., Staves, K., and Wilk, A. (2011). Tenure and Promotion in the Age of Online Social Media. Proceedings of the American Society for

Information Science and Technology (ASIS&T) Conference.• Gurajala, S., White, J. S., Hudson, B., Voter, B. R., & Matthews, J. N. (2016). Profile characteristics of fake Twitter accounts. Big Data &

Society, 3(2), 2053951716674236.• Hassan, S. U., Imran, M., Gillani, U., Aljohani, N. R., Bowman, T. D., & Didegah, F. (2017). Measuring social media activity of scientific

literature: an exhaustive comparison of scopus and novel altmetrics big data. Scientometrics, 1-21.• Kumar, P., Gruzd, A., Haythornthwaite, C., Gilbert, S., Esteve Del Valle, M., Paulin, D. (2018). Social Media in Educational Practice: Faculty

Present and Future Use of Social Media in Teaching. In Proceedings of the 51st Hawaii International Conference on System Sciences.• MacDonald, B., Duggan, L., Gruzd, A, & Collins, V., Scientific Communication: Testing Historical & Present-Day Communication Networks

with Social Network Analysis. Working paper. • Melero, R. (2015). Altmetrics–a complement to conventional metrics. Biochemia medica, 25(2), 152-160.• Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: a review of the literature. Journal

of the Association for Information Science and Technology, 68(9), 2037-2062.• Varol, O., Ferrara, E., Davis, C. A., Menczer, F., & Flammini, A. (2017). Online human-bot interactions: Detection, estimation, and

characterization. arXiv preprint arXiv:1703.03107. • Wang, A. H. (2010). Don’t follow me: Spam detection in Twitter. In Proceedings of the 2010 International Conference on Security and

Cryptography (SECRYPT) (pp. 1–10). IEEE.