altmetrics (medical library perspective)

ALTMETRICS | Tara Brigham | 1

ALTMETRICS NNLM Southeastern/Atlantic Region webinar by Tara Brigham June 17th 2015 Presentation Outline 1. ALTMETRICS ALTERNATIVE TO TRADITIONAL METRICS “Altmetrics is the study and use of scholarly impact measures based on activity in online tools and environments. The term has also been used to describe the metrics themselves – one could propose in plural a ‘set of new altmetrics.’ Altmetrics is in most cases a subset of both scientometrics and webometrics; it is a subset of the latter in that it focuses more narrowly on scholarly influence as measured in online tools and environments, rather than on the Web more generally.”

Priem, J., P. Groth, and D. Taraborelli. “The Altmetrics Collection.” PLoS ONE 7, no.11: e48753. doi:10.1371/journal.pone.0048753.

Photo by Evil Erin 2. "Altmetrics are measures of scholarly impact mined from activity in online tools and environments.” Jason Priem, author of “Altmetrics: a manifesto” Mr. Priem also is acknowledged as the person that first coined the term ‘altmetrics’, in which he mentioned in a 2010 tweet. Priem, J. (jasonpriem). “I like the term #articlelevelmetrics, but it fails to imply *diversity* of measures. Lately, I’m liking #altmetrics.” September 28, 2010, 7:28pm, https://twitter.com/jasonpriem/status/25844968813. 3. WHAT CAN BE MEASURED? Articles, Books, Posters Plus non‐traditional scholarly work or research ‘products’ Examples: lab notes, data sets or software Article‐level metrics + Includes research ‘products’ like lab notes, data sets or software. Photo by thisisbossi 4. SOURCES OF ALTMETRICS

PubMed

Mendeley

Twitter

Facebook

research blogs

& many others


* Depends on aggregator or altmetric tool (More to come later in the presentation). Photo by mkhmarketing 5. SLIDE 5 http://altmetrics.org/manifesto/ 6. TRADITIONAL METRICS The best way for us to explore the new field of altmetrics is to identify how altmetrics differ from traditional metrics:

number of peer reviewed article published by an author

number of citations accumulated from those published articles

ranking of the journals in which those articles were published (journal impact factor) 7. Number of peer reviewed articles published You can search for the number of peer‐reviewed articles a clinician or researcher has published by looking at their CV, or more reliably might be searching for the articles in PubMed, Scopus, Web of Science (or a respective database for their particular discipline). Interestingly, some reviewers of journal articles are now being credited for their work as a reviewer.

See this article: http://www.nature.com/news/the‐scientists‐who‐get‐credit‐for‐peer‐review‐1.16102

Publons: https://publons.com/ Photo by Jason A. Howie 8. Number of citations accumulated from those published articles Can be found on some journal websites, in the Scopus database, Web of Science database & GoogleScholar.

Scopus: http://www.elsevier.com/solutions/scopus/features

Web of Science: http://interest.science.thomsonreuters.com/content/WOKUserTips‐201011‐IN

GoogleScholar: http://scholar.google.com/intl/en‐US/scholar/citations.html Photo by Wiertz Sébastien 9. Ranking of the journals in which those articles were published (journal impact factor) http://wokinfo.com/essays/impact‐factor/ https://en.wikipedia.org/wiki/Impact_factor


Alberts, B. (May 17, 2013). "Impact Factor Distortions". Science 340 (6134): 787. doi:10.1126/science.1240319. ‐ http://www.sciencemag.org/content/340/6134/787.full Archambault, Éric; Vincent Larivière (2009). "History of the journal impact factor: Contingencies and consequences" (PDF). Scientometrics 79 (3): 635–649. doi:10.1007/s11192‐007‐2036‐x Simons, K. (2008). "The Misused Impact Factor". Science 322 (5899): 165. doi:10.1126/science.1165316. PMID 18845714 Photo by taberandrew 10. EVOLUTION So traditional metrics have been the status quo ‐ why not stick to this model? Well ‐ as with everything ‐ the scholarly environment is changing and constantly evolving. 11. Explosion of social media (and the ability to track it)

https://en.wikipedia.org/wiki/Social_media

http://www.pewinternet.org/data‐trend/social‐media/social‐media‐use‐by‐age‐group/

http://www.pewinternet.org/fact‐sheets/social‐networking‐fact‐sheet/ http://altmetrics.org/altmetrics12/priem/

Photo by kdonovangaddy 12. The San Francisco Declaration on Research Assessment (DORA) 2012 Eliminate the use of journal‐based metrics, such as Journal Impact Factors, in funding, appointment, and promotion considerations. http://am.ascb.org/dora/ http://www.ascb.org/dora‐old/files/SFDeclarationFINAL.pdf Initiated by the American Society for Cell Biology (ASCB) together with a group of editors and publishers of scholarly journals. A number of themes run through these recommendations:

the need to eliminate the use of journal‐based metrics, such as Journal Impact Factors, in funding, appointment, and promotion considerations;

the need to assess research on its own merits rather than on the basis of the journal in which the research is published; and

the need to capitalize on the opportunities provided by online publication (such as relaxing unnecessary limits on the number of words, figures, and references in articles, and exploring new indicators of significance and impact).

Photo by Keoki Seu 13. NIH Grant Applications


Whereas before 2014, the NIH only wanted to see what peer‐reviewed publications you had produced to establish whether or not you were a qualified candidate, now they are allowing investigators to include "non‐publication research products". Non‐publication research products include:

audio or video products

patents

data and research materials

databases

educational aids or curricula

instruments or equipment

models

protocols

software or netware

The biosketch directions now state: "Briefly describe up to five of your most significant contributions to science. For each of these contributions, reference up to four peer‐reviewed publications or other non‐publication research products." https://nexus.od.nih.gov/all/2014/05/22/changes‐to‐the‐biosketch/ http://grants.nih.gov/grants/funding/424/index.htm http://grants.nih.gov/grants/funding/424/SF424R‐R_biosketchsample_VerC.docx This change first happened for National Science Foundation grant applications: National Science Foundation. NSF 13‐1, Grant Proposal Guide Summary of Changes. January 2013. http://www.nsf.gov/pubs/policydocs/pappguide/nsf13001/gpg_sigchanges.jsp 14. DIFFERENCES/ BENEFITS This article, from one of the co‐founders of ImpactStory, lists many of the benefits of altmetrics: Piwowar, H. “Altmetrics: What, Why and Where? Introduction.” ASIS&T Bulletin 39, no. 4 (April/May 2013): 8‐9. http://asis.org/Bulletin/Apr‐13/ 15. BROADNESS BEYOND JUST ARTICLES

Altmetrics can measure the impact of scholarly products other than articles.

Reflect some of the impact of web‐native scholarly products like blog posts, software, data sets, videos and much more. http://www.slideshare.net/eservice/altmetrics‐aishe‐seminar‐may‐2015

Photo by NASA's Marshall Space Flight Center 16. IMPACT FLAVORS CAPTURES NUANCES IN HOW ARTICLE/ PRODUCT IS BEING USED

altmetrics can provide a greater understanding of how a publication or product is being used.


They can disclose which scholarly products are being read, discussed, saved, and recommended as well as cited.

Tracking the variance in usage has led to the concept in altmetric literature called “impact flavors.”

Impact flavors are a way to “understand the distinctive patterns in the diverse impacts of individual products.”

The impact flavor of a scholarly or research product featured in the mainstream media is going to be much different than one heavily saved in an online citation manager or cited in a research paper.

Piwowar, H. “Altmetrics: What, Why and Where? Introduction.” ASIS&T Bulletin 39, no. 4 (April/May 2013): 8‐9. Lapinski, S., H. Piwowar, and J. Priem. “Riding the Crest of the Altmetrics Wave.” C&RL News 74, no. 6 (June 2013): 292‐300.

Photo by brianjmatis 17. SPEED QUICKER TURN AROUND TIME TO SEE INFLUENCE

altmetrics aggregators are pulling data daily or weekly, so they can produce a quicker turn‐around time to reflect the influence an article or research product is exerting on a particular field.

altmetrics permit impact to be measured shortly after the publication of a paper or the completion of other products.

Photo by medically_irrelevant 18. DIVERSITY REACHING A WIDER AUDIENCE

"Indications of impacts on diverse audiences including scholars but also practitioners, clinicians, patients, educators and the general public."

For scholars and researchers who would like to show how far their research stretches into the mainstream, altmetrics can be that link. ‐ As a rule, it is easy for most to obtain altmetrics data.

Piwowar, H. “Altmetrics: What, Why and Where?” ASIS&T Bulletin 39, no. 4 (April/May 2013): 8‐9.

Photo by KellyCDB 19. CAVEATS As with traditional citation metrics, altmetrics can succumb to the similar problems. And these issues have caused many scholars and researchers to be hesitant to use altmetrics in grant applications or promotion material.


20. Open to manipulation or gaming

Open to manipulation and gaming.

Jeffrey Beall made this argument that altmetrics could be easily gamed.

Beall is well‐known for his blog, which he uses to expose predatory journals and publishers that abuse Open Access publishing.

However, this is true for traditional metrics as well.

Many of the altmetrics aggregators are working on this issue and making it harder to do so example, Altmetric noticing fake Twitter accounts (new, never cited scholarly items before). Beall blog posting about issues with altmetrics:

Beall’s blog post: http://scholarlyoa.com/2013/08/01/article‐level‐metrics/ Rebuttal to Beall's blog post: http://blog.impactstory.org/beall‐altmetrics/ Photo by elPadawan 21. Author disambiguation

True for traditional metrics as well.

ORCID exists, but still somewhat difficult to get individuals / researchers to sign up since it is voluntary.

Attended a presentation session at MLA 2015 where libraries were working with other areas to get their users signed up for ORCID.

* Encourage our patrons to get an ORCID http://orcid.org/ http://blog.impactstory.org/ten‐things‐you‐need‐to‐know‐about‐orcid‐right‐now/ What about at conferences? For posters? http://betterposters.blogspot.com/2013/12/identifying‐poster‐authors‐conference.html

Photo by db Photography | Demi‐Brooke 22. Users choose different online tools, tools which come and go

Use of online tools differ by individual researcher, discipline, and can change over time.

Whereas one discipline has adopted Twitter as a platform for discussing new and relevant information, another might have a stronger following on Mendeley.

Data sources come and go: ‐ The popularity evolution among the social media platforms can also be a limitation of

altmetrics. ‐ Social media tools like MySpace and Friendster were much more active eight years ago,

but have since diminished in popularity and have been replaced with other tools like Twitter and Facebook.

Photo by Ludwig Gatzke


23. Lack of authority or regulations Lack of standards and oversight.

Unlike the regulated computations of scholarly influence such as the journal impact factor, it is unclear how altmetrics would distinguish between a poorly written article receiving a high number of mentions in the mainstream media and an article of higher quality discussed among leaders in that particular field.

“[A] new paper that receives Facebook likes from a dozen Nobel laureates may be of higher impact than another paper that receives 50 likes from undergraduates at the local junior college.”

Tananbaum, G. “Article‐Level Metrics: A SPARC Primer.” SPARC. April 2013.

Important distinctions in the quality of feedback and recognition are currently lacking in most altmetric calculations.

Many of the altmetrics tools have incorporated a ranking system that makes comparisons between other articles within the same journal, written during the same time frame, and across the whole database.

Photo by Nicolas Hoizey 24. Popularity or attention ≠ quality

Popularity (attention) does not always equal quality of research.

For example, an article or other artifact might receive more attention because it’s poorly constructed, scandalous, or controversial.

Photo by Siebbi 25. ALTMETRICS TOOLS What does altmetrics measure? API will capture all types of data:

Usage, they are working with the publishers (how many downloads)

Captures, the basis for captures is DOI (bookmarks, shares)

Mentions (blogs, news, Wikipedia) ‐ Social Media (Facebook likes, tweets)

Citations 26. SLIDE 26 Major product vendors for altmetric data are Altmetric, Impactstory, and Plum Analytics. 27. ALTMETRIC.COM http://www.altmetric.com/


28. ALTMETRIC Track anything with a digital object identifier (DOI) Trademark Altmetric “donuts” Altmetric is a fee‐based, subscription tool Geared toward publishers and institutions

It can find mentions of papers, books, datasets in social media sites (e.g., Twitter, Facebook, Pinterest, Google+), science blogs, many mainstream media outlets (including the NY Times, The Guardian, and special interest publications like Scientific American and New Scientist).

‐ Now close to 80,000 news outlets, government policy documents, and reference managers for mentions of academic papers are tracked.

Altmetric “donuts” are a very recognizable feature of this service. ‐ The colors that make up the donut reflect the mix of sources mentioning the Altmetric

score: blue for Twitter, yellow for blogs, red for mainstream media sources, and so on. http://support.altmetric.com/knowledgebase/articles/83335‐which‐data‐sources‐does‐altmetric‐track.

The Altmetric score is an aggregate metric that is created by using the count from all the relevant mentions from a set of online sources, along with the relative influence of each source. http://support.altmetric.com/knowledgebase/articles/83337‐how‐is‐the‐altmetric‐score‐calculated.

Altmetric is a fee‐based, subscription tool. It is geared toward publishers and institutions with various plans and pricing.

29. Screen shot of Nature article.

Nature Publishing Group (nature.com) and Scopus are two examples that have incorporated Altmetric’s information into their website.

Note the Altmetric 'donut'. 30. Screen shot of article in Scopus database.

Nature Publishing Group (nature.com) and Scopus are two examples that have incorporated Altmetric’s information into their website.

Note the Altmetric 'donut'. 31. ALTMETRIC‐ FREE TOOLS Bookmarklet Altmetric Explorer (for academic librarians & repository managers) Embeddable badges for all institutional repositories

Anyone can use a free Bookmarklet to instantly view the altmetrics of certain papers. There are a few restrictions to using the Altmetric Bookmarklet, but it can produce altmetric figures if used in PubMed or any web page with a DOI.

Altmetric offers free embeddable badges for all institutional repositories.


Librarians can get free access to Altmetric Explorer.

I have an account and while limited in capabilities, it has helped me answer some research questions.

‐ For example, there was an anesthesiologist that was giving a presentation titled: "Best critical care papers of the past 12 months that you may have missed.” The audience was going to be a group of critical care physicians who are also anesthesiologists. The physician wanted to know if it was possible to come up with a list of the most viewed articles that fit these criteria. He wanted a list of about 100 papers from the past year that fit the criteria and were the most READ (not cited).

32. Screen shot of Altmetric Explorer. 33. IMPACTSTORY https://impactstory.org/ 34. IMPACTSTORY Co‐founded by Heather Piwowar and Jason Priem Metrics are computed based on a variety of sources Geared toward individuals 1 month free, $60/ month

Impactstory.org is a non‐profit.

Originally called “total‐impact” and was a hackathon project developed at the UK Beyond Impact workshop in 2011.

Impactstory tracks the impact of articles, data sets, software, videos, presentations, posters, and lab websites by monitoring a wide variety of data sources.

At present, metrics are computed based on the following sources: ‐ Altmetric.com (which includes, blog posts, Facebook public posts, Google+ posts,

Twitter tweets) ‐ arXiv.org ‐ CiteULike ‐ crossref ‐ delicious ‐ Dryad ‐ figshare ‐ GitHub ‐ Mendeley ‐ PLoS search and Article‐Level Metrics ‐ PubMed ‐ Scopus ‐ SlideShare ‐ Vimeo ‐ Web pages ‐ Wikipedia


‐ YouTube 35. Screen shot of my Impactstory profile.

Scholars or researchers can create a collection by uploading articles or products using Google Scholar or Open Researcher and Contributor ID (ORCID), or by providing PubMed IDs, DOI, or URL identifiers.

Impactstory then gathers and provides information on the author’s profile that conveys the altmetrics of each article or research product, detailing the influence within the context of the year it was published or created, and based on engagement type and audience.

Over all Impactstory is useful because you can get some context. i.e. You’ve reached 5,000 views which is better than 75% of all slide decks in SlideShare.

36. Screen shot of my Impactstory profile with world map.

Impactstory also provides context on the reach your research has had globally. 37. Screen shot of my Impactstory profile. 38. Screen shot of an Impactstory Advisor profile. An example of a 'super user' of Impactstory. 39. PLUM ANALYTICS http://www.plumanalytics.com/ 40. PLUM ANALYTICS Founded in 2011, bought by EBSCO in 2014 PlumX is an analysis tool which tracks about 20 different types of what it calls ”artifacts” Collects impact metrics in five categories: usage, captures, mentions, social media, and citations

Plum Analytics was founded in 2011 by former Microsoft librarian, Mike Buschman and entrepreneur Andrea Michalek.

In January 2014, Plum Analytics became a part of EBSCO Information Services.

Artifacts includes: journal articles, books, videos, presentations, conference proceedings, datasets, source code, cases, and more.

41. Screen shot of my PlumX profile. https://plu.mx/u/tbrigham


42. Screen shot of the different impact metrics it collects.

Plum Analytics collects impact metrics in five major categories: usage, captures, mentions, social media, and citations.

43. PLUM ANALYTICS

PlumX is a fee‐based, tiered subscription tool

Geared mainly towards institutions and pricing is based on the number of researchers tracked by the system

Offers free trial of PlumX profile for individual researchers ‐ Contact them for more information or to ask for a free trial:

http://www.plumanalytics.com/contact.html 44. OTHER altmetrics Most journals also provide some basic alternative metrics Times downloaded or viewed 45. Screen shot of PLoS article.

For example, PLoS is well known for providing metrics on things such as total article views, total times cited, and where it has been discussed (on Twitter, Facebook, etc.)

Article in the screen shot is:

Koffel JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: a cross‐sectional survey of recent authors. PLoS One. 2015 May 4;10(5):e0125931. PMID: 25938454. http://www.plosone.org/article/Metrics/info:doi/10.1371/journal.pone.0125931

46. Your users are social Believe it or not! 47. Screen shot of Nature article.

Take a look at this article from Nature: Online collaboration: Scientists and the social network Giant academic social networks have taken off to a degree that no one expected even a few years ago. A Nature survey explores why. Richard Van Noorden 13 August 2014 Corrected: 15 August 2014. http://www.nature.com/news/online‐collaboration‐scientists‐and‐the‐social‐network‐1.15711.

This survey reflects that scholars are using social media and using it connect, share and discover other papers and peers.

48. ALTMETRIC DATA USERS


Who would be interested in altmetrics? 49. RESEARCHERS, CLINICIANS, ETC. ANYONE CREATING OR WRITING

Anyone interested in promotion or tenure.

Those who are more actively producing videos, blogs, books, & even posters might be very interested in altmetrics.

‐ For example: a local physician produces a YouTube video regarding a public health issue and they are curious how useful it is in the wider community.

Photo by Alex Barth 50. PUBLIC AFFAIRS/ MARKETING

The Public Affairs or Marketing deparment might be interested to know which articles, videos, or 'products' are popular. Then they can highlight articles or research that is making a wider impact on social media and perhaps in society/ general public.

Photo by Stuck in Customs 51. ADMINISTRATORS

Administrators want to track publications of their department or organization/ hospital/ medical center.

altmetrics can possible determine or influence how much dedicated time a researcher/ clinician is given for research or academic ventures.

‐ For example: altmetric numbers might justify research time a clinician or researcher uses).

Photo by Tyler Merbler 52. FUNDERS

Funders are interested and want researchers to document their outreach and engagement initiatives, as well as the "broader impacts" of their work.

Photo by rafael‐castillo 53. LIBRARIANS CAN HELP: Explain pros and cons Educate individuals on how to use altmetrics Provide assistance in tracking altmetrics


Librarians can help by doing what we do best!

Need more encouragement/ ideas? Check out these blog posts: ‐ http://blog.impactstory.org/better‐than‐a‐free‐ferrari/ ‐ http://blog.impactstory.org/4‐things‐librarians‐altmetrics/

Photo by University of the Fraser Valley 54. QUIZ 55. WHAT DO ALTMETRICS NOT MEASURE? 1. Attention 2. Quantity of impact 3. Quality 4. Diversity of impact Correct answer: #3 Quality 56. “The point of altmetrics isn’t to measure quality. It’s to better understand impact: both the quantity of impact and the diverse types of impact.” – Stacy Konkiel http://blog.impactstory.org/beall‐altmetrics/ Photo by Evil Erin 57. LEARN MORE Brigham TJ. An introduction to altmetrics. Med Ref Serv Q. 2014;33(4):438‐47. PMID: 25316077. doi: 10.1080/02763869.2014.957093. Freely available here: http://www.tandfonline.com/eprint/3a5IzE3fPQIJs6BUkVht/full “Altmetrics in Context” from the Canadian Association of Research Libraries: http://www.carl‐abrc.ca/uploads/SCC/CARL2013‐altmetrics‐EN‐FA.pdf Altmetrics: What, Why and Where? http://asis.org/Bulletin/Apr‐13/ Ladisch, M. Altmetrics and Social Media: Publicising, Discovering, Engaging Published on Jun 11, 2015. http://www.slideshare.net/eservice/altmetrics‐aishe‐seminar‐may‐2015 Eysenbach G. Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. J Med Internet Res. 2011 Dec 19;13(4):e123. doi: 10.2196/jmir.2012. Steal these altmetrics LibGuides! http://blog.impactstory.org/steal‐altmetrics‐libguide/ [NOTE! Please pay attention to the comment section. You have to download them, not get them from the Springshare website.]


Duke created their guide from the Impactstory guide: http://guides.mclibrary.duke.edu/c.php?g=217135&p=1434254 Utrecht University Library http://libguides.library.uu.nl/researchimpact/altmetrics If on Twitter follow: #altmetrics @jasonpriem [Impactstory] @researchremix [Heather Piwowar‐ Impactstory] @Stew [Euan Adie‐ Altmetric.com] @altmetric @Impactstory @PlumAnalytics Photo by Pink Sherbet Photography 58. Questions? Feel free to contact me if you have questions: brigham.tara at gmail.com Twitter: @TBrigham LinkedIn: https://www.linkedin.com/in/tarabrigham Photo by pareeerica

altmetrics (medical library perspective)

Education

Transcript of altmetrics (medical library perspective)