Carpenter: Lets remove alt from altmetrics - UKSG Plenary
-
Upload
national-information-standards-organization-niso -
Category
Technology
-
view
1.426 -
download
1
Transcript of Carpenter: Lets remove alt from altmetrics - UKSG Plenary
![Page 1: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/1.jpg)
“Alt” is German for “old” It’s &me to stop talking about metrics in terms of “alterna&ve” metrics.
Todd A. Carpenter
Executive Director, NISO @TAC_NISO
March 30, 2015
![Page 2: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/2.jpg)
! Non-‐profit industry trade associa&on accredited by the American Na&onal Standards Ins&tute
! Mission of developing and maintaining technical standards related to informa&on, documenta&on, discovery and distribu&on of published materials and media
! Volunteer driven organiza&on: 400+ contributors spread out across the world
! Responsible (directly and indirectly) for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, MARC records, and ISBN
About
March 30, 2015 2
![Page 3: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/3.jpg)
1980s Alternative
John Taylor, Duran Duran
![Page 4: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/4.jpg)
1920s Alternative
![Page 5: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/5.jpg)
2010s Not-so Alternative
Cole Plante
![Page 6: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/6.jpg)
Since Jason Priem coined the term “altmetrics”
• There have been some 1,170 ar&cles and posts (per Google Scholar) on the topic of altmetrics
• There have been eight altmetrics conferences • There are now at least four companies focused on providing altmetrics-‐style analy&cs
March 30, 2015 6
![Page 7: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/7.jpg)
When it becomes a standard, its not alterative any more…
![Page 8: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/8.jpg)
But what’s the value in coun&ng Tweets and
Facebook likes?
March 30, 2015 8
![Page 9: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/9.jpg)
More than just popularity
Research is poin&ng to the fact that there is a modest posi&ve correla&on between early-‐signal metrics (altmetrics) and later-‐signal metrics (cita&ons)
Do altmetrics correlate with cita6ons? Extensive comparison of altmetric indicators with cita6ons from a mul6disciplinary perspec6ve by Rodrigo Costas, Zohreh Zahedi, Paul Wouters Do Altmetrics Work? TwiBer and Ten Other Social Web Services by Mike Thelwall , Stefanie Haustein, Vincent Larivière, & Cassidy R. Sugimoto (2006) Earlier web usage sta&s&cs as predictors of later cita&on impact by Brody T, Harnad S & Carr L
March 30, 2015 9
![Page 10: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/10.jpg)
More importantly: Scholarship today is much more than journals
March 30, 2015 10
![Page 11: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/11.jpg)
How do we measure the impact of these different
forms of output?
March 30, 2015 11
![Page 12: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/12.jpg)
Would a researcher focus on only one data source or methodological approach?
March 30, 2015 12 Duke University -‐ Informa&on Ini&a&ve at Duke (IID)
![Page 13: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/13.jpg)
There aren’t metrics and “altmetrics”; there are only metrics!
March 30, 2015 13
![Page 14: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/14.jpg)
We have been using non-‐cita&on-‐based metrics for decades
March 30, 2015 14
![Page 15: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/15.jpg)
March 30, 2015 15
![Page 16: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/16.jpg)
March 30, 2015 16
Traverse Area District Library (TADL)
![Page 17: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/17.jpg)
What has changed is our ability to
collect and analyze data in new ways that are meaningful to others
March 30, 2015 17
![Page 18: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/18.jpg)
What are we lacking?
=
![Page 19: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/19.jpg)
TRUST
=
![Page 20: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/20.jpg)
What are the elements of metrics that we need
to build trust?
![Page 21: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/21.jpg)
Granularity
Time Scale
Identification Definition
Exchange
![Page 22: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/22.jpg)
I omen sound like a broken record
• Defining what is to be counted = standards • How to describe what to count = standards • Iden&fica&on of what to count = standards • Procedures for coun&ng or not = standards • Aggrega&ng counts from network = standards • Exchange of what was counted = standards March 30, 2015 22
![Page 23: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/23.jpg)
![Page 24: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/24.jpg)
![Page 25: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/25.jpg)
Phase 1 Mee6ngs
October 9, 2013 -‐ San Francisco, CA December 11, 2013 -‐ Washington, DC
January 23-‐24 -‐ Philadelphia, PA Round of 1-‐on-‐1 interviews – March/Apr
Phase 1 report published in June 2014 �
![Page 26: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/26.jpg)
Mee&ng Lightning Talks • Expecta&ons of researchers • Exploring disciplinary differences in the use of social media in
scholarly communica&on • Altmetrics as part of the services of a large university library
system • Deriving altmetrics from annota&on ac&vity • Altmetrics for Ins&tu&onal Repositories: Are the metadata
ready? • Snowball Metrics: Global Standards for Ins&tu&onal
Benchmarking • Interna&onal Standard Name Iden&fier • Altmetric.com, Plum Analy&cs, Mendeley reader survey • Twiper Inconsistency March 30, 2015 26
“Lightning" by snowpeak is licensed under CC BY 2.0
![Page 27: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/27.jpg)
March 30, 2015 27
![Page 28: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/28.jpg)
March 30, 2015 28
![Page 29: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/29.jpg)
March 30, 2015 29
![Page 30: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/30.jpg)
30 One-‐on-‐One Interviews
March 30, 2015 30
![Page 31: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/31.jpg)
Lots and lots of ideas
March 30, 2015 31
![Page 32: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/32.jpg)
Alterna6ve Assessment Ini6a6ve
Phase 2 Presenta6ons of Phase 1 report (June 2014)
Priori6za6on Effort (June -‐ Aug, 2014) Project approval (Nov -‐ Dec 2014)
Working group forma6on (Jan -‐ March 2015) Consensus Development (March 2015 -‐ Feb 2016)
Trial Use Period (Feb 15 -‐ May 16)
Publica6on of final recommenda6ons (Aug 16)
![Page 33: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/33.jpg)
White Paper Released
March 30, 2015 33
![Page 34: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/34.jpg)
200 + Ideas 25 Poten6al Work Themes
Defini6ons Applica6on to types of research outputs Discovery implica6ons Research evalua6on Data quality and gaming Grouping, aggrega6ng, and granularity Context Adop6on
March 30, 2015 34
![Page 35: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/35.jpg)
March 30, 2015 35
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Unimportant
Of liple importance
Moderately important
Important
Very important
Community Feedback on Project Idea Themes
n=118
![Page 36: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/36.jpg)
March 30, 2015 36
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Unimportant
Of liple importance
Moderately important
Important
Very important
n=118
Community Feedback on Project Idea Themes
![Page 37: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/37.jpg)
Top-‐ranked ideas (very important & important >70%)
• Develop specific defini&ons for alterna&ve assessment metrics. (87.9%)
• Promote and facilitate use of persistent iden&fiers in scholarly communica&ons. (82.8%)
• Develop strategies to improve data quality through normaliza&on of source data across providers. (80.8%)
• Iden&fy research output types that are applicable to the use of metrics. (79.8%)
• Define appropriate metrics and calcula&on methodologies for specific output types, such as somware, datasets, or performances. (78.1%)
• Explore crea&on of standardized APIs or download or exchange formats to facilitate data gathering. (72.5%)
• Research issues surrounding the reproducibility of metrics across providers. (70.7%)
March 30, 2015 37
![Page 38: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/38.jpg)
Launching three working groups
March 30, 2015 38
A) Development of defini&ons and descrip&ons of use
B) Defini&ons for appropriate metrics and calcula&on methodologies for non-‐tradi&onal output types. Also work toward promo&on and facilita&on of use of persistent iden&fiers
C) Development of strategies to improve data quality through source data providers
![Page 39: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/39.jpg)
Steering Committee
Project Site: www.niso.org/topics/tl/altmetrics_initiative/
White Paper:
http://www.niso.org/apps/group_public/download.php/13295/niso_altmetrics_white_paper_draft_v4.pdf
March 30, 2015 39
![Page 40: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/40.jpg)
Alterna6ve Assessments of our Assessment Ini6a6ve
White paper downloaded 7,400 6mes 21 substan&ve comments received
120 in-‐person and virtual par&cipants at the mee&ngs These 3 mee&ngs apracted >400 RSVPs for live stream
Goal: generate about 40 ideas, in total, generated more than 250 Project materials downloaded more than 22,000 &mes
More than 550 direct tweets using the #NISOALMI hashtag Survey ranking of output by 118 people
Six ar&cles in tradi&onal news publica&ons 15 blog posts about the ini&a&ve
![Page 41: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/41.jpg)
For more
Project Site: www.niso.org/topics/tl/altmetrics_initiative/
White Paper:
http://www.niso.org/apps/group_public/download.php/13295/niso_altmetrics_white_paper_draft_v4.pdf
March 30, 2015 41
![Page 42: Carpenter: Lets remove alt from altmetrics - UKSG Plenary](https://reader030.fdocuments.net/reader030/viewer/2022032420/55a5f08a1a28ab05328b4589/html5/thumbnails/42.jpg)
Questions?
Todd Carpenter
Executive Director [email protected]
@TAC_NISO
National Information Standards Organization (NISO) 3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA +1 (301) 654-2512 www.niso.org
March 30, 2015 42