Carpenter NISO Altmetrics Meeting #2 opening presentation Dec 2013
52
Do We Need Standards for Alternative Metrics? What are we hoping to accomplish? Todd Carpenter, Executive Director, NISO NISO Alternative Assessment Initiative Discussion Forum #2 Washington, DC | December 11, 2013
-
Upload
national-information-standards-organization-niso -
Category
Education
-
view
387 -
download
2
Transcript of Carpenter NISO Altmetrics Meeting #2 opening presentation Dec 2013
Do We Need Standards for
Alternative Metrics?
What are we hoping to accomplish?
Todd Carpenter, Executive Director, NISO NISO Alternative Assessment Initiative
Discussion Forum #2 Washington, DC | December 11, 2013
• Non-profit industry trade association accredited by ANSI with 150+ members
• Mission of developing and maintaining technical standards related to information, documentation, discovery and distribution of published materials and media
• Volunteer driven organization: 400+ spread out across the world
• Responsible in various wasy for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, MARC records, and ISBN
About
We stand on the shoulders of giants...
The Mighty Citation
Impact factor
Thomson-Reuters IMPACT FACTOR
Two Stanford Students
They wrote a paper
They started a company in someone’s garage
Photo: AP
Citations are awesome, aren’t they?
A perfect solution to all the world’s problems, no?
A simple conclusion can be drawn
“What Des-Cartes did was a good step. You have added much several ways, & especially in taking ye colours of thin plates into philosophical consideration. If I have seen further it is by standing on ye shoulders of Giants."
Back to Sir Isaac Newton
Citations aren’t always what we think they are
Caveat emptor
Another problem with citations
So where to from here?
Define altmetrics, please
Usage Based Metrics
Clickstream Analysis
Social Media Metrics
Image: Danny Brown, The State of Social Media Marketing 2012
Clickstream data yields high-resolution maps of science
Johan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009
Clickstream network analysis
Behavioral Metrics
Multivariate analysis
Sentiment Analysis
Image: Domenico, Caron, Davis, et al.
Citation of non-traditional content
Citations to new forms of communication aren’t as simple as one might think. !
Define a use of this?
We’ve got what, now how?
How do we measure things?
At what granularity do we measure things?
Which measurements are valuable?
How do we ensure the validity of those measurements?
How to (or even if we should) deal with other issues
such as gaming?
Issues we face with alternative metrics
Basic Definitions !
(So we are all talking about the same thing)
Comparison across providers
Source: Scott Chamberlain, Consuming Article-‐Level Metrics: Observations And Lessons From Comparing Aggregator Provider Data, Information Standards Quarterly, Summer 2013, Vol 25, Issue 2.
Consistency in what is counted
Source: Scott Chamberlain, Consuming Article-‐Level Metrics: Observations And Lessons From Comparing Aggregator Provider Data, Information Standards Quarterly, Summer 2013, Vol 25, Issue 2.
Authorship -----
Disambiguation &
Contributor Roles
Open exchange of component data
TRUST
What is NISO trying to do?
Alterna(ve Assessment Ini(a(ve
!
Phase 1 Mee(ngs October 9, 2013 -‐ San Francisco, CA December 11, 2013 -‐ Washington, DC
January 23-‐24 -‐ Philadelphia, PA !
Phase 1 report expected in May 2014
Alterna(ve Assessment Ini(a(ve !
Phase 2 Presenta(ons of report (June 2014) Priori(za(on Effort (June -‐ Aug, 2014)
Project approval (Sept 2014) Working group forma(on (Oct 2014)
Consensus Development (Nov 2014 -‐ Dec 2015) Trial Use Period (Dec 15 -‐ Mar 16)
Publica(on of final recommenda(ons (Jun 16)
Criticism
Why now? Isn’t it too soon?
Can NISO to do this?
How soon is now?
These things take time
It will be hard to change
“All I see now is blonde, brunette,
redhead”
One Very Last Thing…
Image from Will Lion: http://www.flickr.com/photos/will-lion/2646213692/
Thank you! Todd Carpenter, Executive Director
[email protected] National Information Standards Organization (NISO)
3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA
+1 (301) 654-2512 www.niso.org
