Alternative metrics to evaluate scientific impact

Prof. Dra. Nanci OddonePrograma de Pós-Graduação em Biblioteconomia/UNIRIO

With Iara Vidal & Andréa do Nascimento

Simpósio “Biased Science and Alternatives for the Publication System”15 de setembro de 2014 ● UFRJ, Rio de Janeiro, Brasil

ALTMETRICS

“The creation and study of new metrics based on the Social Web for analyzing and informing scholarship”

ORIGINS

ORIGINS• Altmetrics: a manifesto / Jason

Priem, Dario Taraborelli, Paul Groth, Cameron Neylon (2010)

• Scholarship’s main filters for importance – peer review, citation counts, JIF – are failing. Altmetrics offers possible solutions

ALTERNATIVE WAYS• Alternative Metrics: shares, tags, notes,

comments, bookmarks, social mentions...

• Alternative Products: blogs, raw data, code, grey literature...

• Alternative Audiences: organizations, doctors, patients, educators, students, citizen scientists...

MULTIPLE DIMENSIONS

IMPACT

USAGEDownloadsViews

PEER REVIEW

Expert opinion

CITATIONSALTMETRICS

StorageLinks

BookmarksConversations

Adapted from altmetrics.org

ALTMETRIC INDICATORS

Extraído de: KONKIEL, S. Altmetrics: A 21st-Century Solution to Determining Research Quality. Online searcher, v. 37, n. 4, p. 10-15, 2013.

Dado Descrição Exemplos

Shares / Mentions A public post to share news about an article or research output

Twitter, Topsy, Facebook, reddit, news articles, blog posts, Google+, YouTube, Figshare, Mendeley

Saves / CapturesSocial bookmarks, favorites, online reference managers

Mendeley, CiteULike, Delicious, Github, Twitter, and Slideshare

Reviews / Social Media Discusion, including additional commentary

Faculty of 1000 (F1000), blog posts, comentários de artigos, comentários no Facebook

AdaptationsCreation of derivative work using an article or other research output Github

Views / UsageDownloads or views in web services and social media sites

Figshare, Slideshare, Dryad, Facebook, YouTube, LinkedIn

Cites Citations in academic literature Pubmed, Scopus, Web of Science

Adaptado de: KONKIEL, S. Altmetrics: A 21st-Century Solution to Determining Research Quality. Online searcher, v. 37, n. 4, p. 10-15, 2013; e TANAMBAUM, G. Article-Level metrics: a SPARC primer. 2013.

ADVANTAGES• Diversity: diverse metrics, for diverse

products, capturing multiple dimensions of scientific impact

• Complementing traditional metrics (San Francisco Declaration on Research Assessment – DORA, 2012)

• Speed: almost real-time data

• Openness: open data and tools

DISADVANTAGES• Gaming

• Lack of credibility: a measure of impact or popularity?

• Altmetric sources and indicators are heterogeneous and ephemerous

• CONTEXT IS KEY!

ARTICLE-LEVEL METRICS (ALMS)

• The combination of traditional and alternative data to capture the impact of a particular article

• A clearer picture of how articles are being discussed, shared, and used

ALMS VS. ALTMETRICS

ALMs = attempts to

measure impact at the article-

level

Altmetrics = refers to data sources (social media tools)

IMPACT MEASURES

From TANAMBAUM, G. Article-Level metrics: a SPARC primer. 2013.

PUBLISHERS & ALTMETRICS•Elsevier•EBSCO•PLOS•BioMed Central•Nature•Springer

•Wiley•Emerald•High Wire Press•OJS/PKP•Rede SciELO•(…)

TOOLSFour Main Providers:•Altmetric•ImpactStory•Plum Analytics•PLOS ALM

CHALLENGES• Need for standards/best practices to

guide data collection and treatment• Altmetrics Initiative, National

Information Standards Organization (NISO)

• Need for ALM/altmetrics studies in different contexts – including peripheric countries.

THANK YOU

Iara Vidal@iaravps / iaravidalps@gmail.com

Andréa do Nascimento@deiafix / aandreafg@gmail.com

Prof. Dra. Nanci Oddoneneoddone@gmail.com