Post on 12-Apr-2017
Beyond Citation Searching: new tools to measure research impact
Elaine Lasda Bergmanelasdabergman@albany.edu
Professional Activities CommitteeInvited Presentation
January 21, 2016
Overview• Bibliometrics– Origins– Tools• WoS• Scopus• GS
– Issues
Overview• Other Models
• Becker Model– Altmetrics–Open Science
• Takeaways
Photo source: http://upload.wikimedia.org/wikipedia/commons/thumb/1/18/%22Citation_needed%22.jpg/1024px-%22Citation_needed%22.jpg
What is a “metric?”• 4. A system or standard of measurement; a criterion or set of criteria stated in
quantifiable terms.• 1934 H. C. WARREN Dict. Psychol. 166/2 Metric, a system or standard of measurement in
terms of which the conclusions stated hold.• 1951 J. S. BRUNER in R. R. Blake & G. V. Ramsey Perception v. 141 Our basic metric will
involve the comparison of group scores.• 1968 Language 44 715 The simplicity metric demands that one choose the unmarked
segment as the underlying one, since its choice leads to less complexity in the phonological representation.
• 1973 A. H. SOMMERSTEIN Sound Pattern Anc. Greek iii. 94 This statement does not depend on the acceptance of a feature-counting simplicity metric. P. H. Matthews, in a seminar at Cambridge, has criticized the assumption that such a metric is the appropriate evaluation measure for grammars.
• 1993 Computing 19 Aug. 21/3 ‘You can find relevant metrics for all key requirements,’ he says. ‘Ease of learning, for example, can be measured by setting targets for novice users that are a high percentage of the known performance of experts.’
• "metric, n.1 and adj.1". OED Online. December 2015. Oxford University Press. http://www.oed.com/view/Entry/117657?result=1&rskey=bAcR8e& (accessed January 11, 2016).
Origin of the term Bibliometrics
Pritchard, A. (1969). Statistical bibliography or bibliometrics. Journal of Documentation, 25, 348-9.
• Citation count• Impact Factor• Immediacy Index• Citation Half-Life
Origins of Bibliometrics
Photo: http://www.chemheritage.org/Discover/Collections/Oral-Histories/Details/asset_upload_file66_69008_thumbnail.jpg
• Citation count• Impact Factor• 5-year Impact Factor• Immediacy Index• Number of Articles• Cited Half-life• Eigenfactor Score• Article Influence Score
Journal Impact Factor
• Strengths• Limitations
Number of citations to a journal in a given year from articles occurring in the past 2 years
Divided by the number of scholarly articles published in the journal in the past 2 years
InCites Platformhttp://wokinfo.com/products_tools/analytical/jcr/
Cited Reference Searching
Citing vs. Cited References
• Citing References – scholarly journal (or scholarly book) content
indexed in WoS – the articles that CITE the cited references
• Cited References– Any (most) references cited in a scholarly journal
article indexed in WoS– Can be items that are not themselves indexed
in WoS IF an indexed journal article cited it• Example: newspaper articles
Core Collection• Contents
– 12K “highest impact” journals– 160K conference proceedings
– Science Citation Index Expanded (SCI-EXPANDED) --1900-present Social Sciences Citation Index (SSCI)
– Social Sciences Citation Index (SSCI) --1900-present Arts & Humanities Citation Index (A&HCI)
– Arts & Humanities Citation Index (A&HCI) --1975-present Conference Proceedings Citation Index- Science (CPCI-S)
– Emerging Sources Citation Index (ESCI) -- 2015-present– Conference Proceedings Citation Index- Science (CPCI-S) --1990-present– Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH)
--1990-present– Book Citation Index– Science (BKCI-S) --2005-present– Book Citation Index– Social Sciences & Humanities (BKCI-SSH) --2005-present– Current Chemical Reactions (CCR-EXPANDED) --1985-present– Includes Institut National de la Propriete Industrielle structure data back to 1840– Index Chemicus (IC) --1993-present
Book Citation Index• Contents– Scholarly books & chapters (count??)– NOT• Textbooks • Encyclopedias/ Reference Books
– 2005 to present
Data Citation Index• Contents (counts?)– Some pre-1900 but most 1900 to
present– Repositories– Data Studies– Data Sets– Software
Exampleshttp://library.albany.edu/db/resource/?id=4158
Scopus• Content– 22K titles
• 20.8K Peer reviewed journals• 3,850 Journal “articles in press”• 367 Trade journals• 400 Book series• 40K Books• 6.4M Conference papers
– 21M records from 1823-1996– 25.2 M patent records
Journal Metrics• SCImago Journal Rank• Impact Per Publication• Source Normalized Impact per Paper• Citation Count• Document count per year• Percent never cited • Percent review articles
Article Metrics• Citation Count & percentile benchmark• Field Weighted Citation Impact• Mendeley Readership Count & percentile
benchmark• Count of scholarly commentary– Blogs, Wikipedia
• Count & benchmark of social activity– Twitter, Facebook
• Others, broken down by source
Author Metrics• h-index & h-graph• Citation overview tracker• Analysis & Viz of author output
Cited Reference Searching
• Use Author or Author Identifier search
• Limit by Affiliation if you can• DIRTY DATA IS HARDER TO FIND IN
SCOPUS!• Examples
http://library.albany.edu/db/resource/?id=4018
Google Scholar Content
They do, however, try to help
https://scholar.google.com/intl/us/scholar/about.html
Guidelines for Authors
Guidelines for Publishers
Guidelines for Repositories
Cited Reference Searching in GS (the Hard Way)
Examplehttp://scholar.google.com
Publish or Perish• Downloadable tool• Metrics:
– Total # of papers– Total # of citations– h-index– g-index– Contemporary h-index– Still other h-index variations– Average annual increase in h-index– Age weighted citation rate– Authors per paper metrics
http://www.harzing.com/
Author Profiles (Google Scholar Citations)
Impact Factor Scams
http://scholarlyoa.com/2013/08/06/bogus-impact-factor-companies/
Academia Fights Back!
http://www.theguardian.com/science/political-science/2013/may/17/science-policy
Becker Modelhttps://becker.wustl.edu/impact-assessment
• Strengths • Limitations
QUESTION:
WHAT CONSTITUTES SCHOLARLY OUTPUT???
Altmetrics• Altmetrics Manifesto
– Peer review is “slow”– Existing metrics “insufficient”– New tools provide new ways to measure
• Zotero• Mendeley• Twitter
– Scholarly output is NOT merely peer reviewed journal articles• “raw science”• nanopublication• self-publishing
J. Priem, D. Taraborelli, P. Groth, C. Neylon (2010), Altmetrics: A manifesto, 26 October 2010. http://altmetrics.org/manifesto
Altmetrics Tools• Altmetric.com (bookmarklet)
– http://www.altmetric.com/
• Microsoft Academic (author, institution profiles)– http://academic.research.microsoft.com/Default.aspx
• ImpactStory –scholar/user subscription– https://impactstory.org
• Mendeley• Scholarometer• Topsy• PlumX
Open Science/Open Access
• Open Data• Open Access• Open Source• Citizen Science– Etc. etc. etc.
Open Access• Greater visibility – no paywall • “Google-able”• Intellectual freedom• Dissemination of research• Government grant requirements• http://libguides.library.albany.edu/scholcomm
Scholars Archive (repository) Metrics
Scholars Archive (Author) Metrics
Author Dashboardhttp://scholarsarchive.library.albany.edu/cgi/myaccount.cgi?context=
Other Open Science Tools:
• VIVO – http://www.vivoweb.org/
• PeerEvaluation– http://www.peerevaluation.org/
• Kudos – https://www.growkudos.com/
Scholarly Networking• Faculty of 1000 – http://f1000.com/
• Figshare – http://figshare.com/
• GitHub– https://github.com/
• Zenodo – http://zenodo.org/
• ResearchGate• https://www.researchgate.net/home
Regular Social Media• SlideShare• Blogs• Twitter• YouTube/Vimeo• Facebook • LinkedIN
Awesome Resource!
http://blog.impactstory.org/research-impact-challenge-ebook/
Takeaways• Bibliometrics
– Increased number of metrics– Increased sources of cited and citing references– Increased formats covered
• Altmetrics– Fast– Diverse– New forms of scholarly output– New forms referencing scholarly output
• Open Access and Open Science– Increase impact and visibility– Increase quality of scientific methods and research– New measures of open scholarly output
• Not necessarily correlated with traditional bibliometrics
Questions ?