Altmetrics: a primerWhere does the data come from?

Can it be gamed?Buy in or build your own?

Mike TaylorResearch Specialist

http://orcid.org/0000-0002-8534-5985mi.taylor@elsevier.com

• Alternative metrics, altmetrics, article data, usage data, assessments, metrics, impact, understanding, attention, reach…

• If this seems confusing…• Altmetrics is at the big bang stage – this

universe has not yet cooled down and coalesced

Some words on terms

• A set of altmetric data is about a common document and represents usage, recommendation, shares, re-usage

• Identified by DOI, URL, shortened URL, other ID (eg Arxiv, Pubmed)

• It does not show common intent: a tweet is not the same as a Mendeley share is not the same as a Data Dryad data download is not the same as mass media coverage or a blog

What is the data?

• Altmetric.com• Kudos• Plum Analytics• PLOS / PLOS code• Impactstory.org• Altmetrics is not Altmetric.com

Each has strengths and weaknesses,no canonical source

Various providers…

• Scopus donut• Sciencedirect• Scival and Pure• We’re looking to improve the ‘donut’• “Altmetric.com score” can probably be more

useful

Elsevier uses Altmetric.com

• Altmetrics isn’t one thing, so attempting to express it as one thing will fail.

• We favour intelligent clusters of data: social activity, mass media, scholarly activity, scholarly comment, re-use

• Elsevier believes that more research is needed, and that best indicators are scholarly activity and scholarly comment

Bringing together sources…

The more data we can see

• The bigger the picture

scholarly activity (mean)

scholarly comment (mean)

log(social activity (mean))

mas media (mean)

• If people take this data seriously, will they cheat?• Eg, Brazilian citation scandal, strategies used by people

to increase IF of journals• Expertise in detecting fraudulent downloads (eg, SSRN),

self-tweeting – when is ‘normal’ corrupt?• One thing to buy 1000 tweets, another to buy 10 blogs,

or mass media coverage• Do those twitter accounts have scholarly followers? • Pattern analysis, usage analysis, network analysis• Public data = public analysis = public response

Gaming / cheating

• Biggest criticisms are when people try and conflate all the data into a single thing

• Easy point of attack – tweets are all about “sex and drugs and rock ‘n’ roll papers”*

• Using clusters is more intelligible to academic community – eg, re-use, scholarly activity, scholarly comment (blogs, reviews, discussions)

• * this isn’t true anyway

Other criticisms

• Altmetrics has got where it is today on the basis of standards

• Without ISSNs and DOIs, the world is a harder place x 1000

• Elsevier is supporting research to discover scholarly impact in areas that don’t use DOIs

• (Other standards exist: PubMed IDs, Arxiv IDs)

Making altmetrics work

• Increasingly, we’re seeing altmetrics being used to describe objects other than articles, but also institutions, books, journals, data and people

• For institutions, Snowball Metrics has recently adopted the same formulation for grouping altmetrics as Elsevier (www.snowballmetrics.com)

Expanding views of altmetrics

• More funders are insisting on open data• And the way to understand whether its being

used … is data metrics – combining altmetrics and traditional (web-o-)metrics

• Downloads, citations, shares, re-uses…• Downside: data repository is fragmented, 600+

repositories registered at databib.org• Upside: Datacite, ODIN, ORCID, DOI, RDA,

Draft Declaration of Data Citation Principles

Making data count

• Governments don’t operate like scholars• Rhetoric, argument, polemics• Personal reputation is important• Laws don’t contain citations• The relationship is fuzzy – less a chain of

evidence, more a miasma of influence• Elsevier is sponsoring work to understand this

relationship

Measuring the effect of research on society

• Being deployed across our platforms• Engaging with thought leadership• Promoting open metrics• Acceptance research• Academic research• Sponsoring research in cutting-edge

technology• LibraryConnect, www.researchtrends.com

Elsevier and alternative metrics