Where and how knowledge on digital library evaluation spreads: a case study on conference literature
14
WHERE AND HOW KNOWLEDGE ON DIGITAL LIBRARY EVALUATION SPREADS: A CASE STUDY ON CONFERENCE LITERATURE LEONIDAS PAPACHRISTOPOULOS, ANGELOS MITRELIS, CHRISTOS PAPATHEODOROU DEPARTMENT OF ARCHIVES, LIBRARY SCIENCE & MUSEOLOGY, IONIAN UNIVERSITY, GREECE GIANNIS TSAKONAS LIBRARY & INFORMATION CENTER, UNIVERSITY OF PATRAS, GREECE
-
Upload
giannis-tsakonas -
Category
Education
-
view
311 -
download
0
description
Scholarly communication has not remained unaffected by the advance of the social networking culture. The traditional bibliometric paradigm is strongly questioned as a tool that accurately portrays the impact of research outcomes. New metrics, such as download or view rates and shares, have been proposed as alternative ways for measuring the impact of digital content published in the form of articles, datasets, etc. Mendeley's Readership Statistics are one of these metrics, based on the assumption that there is a linkage between a paper in a collection and the interests of collection owner. The current study explores the “altmetric” aspects of the literature of the Digital Libraries Evaluation domain, as it is expressed in two major conferences of the field, namely JCDL and ECDL. Our corpus consists of 224 papers, for which we extract readership data from Mendeley and examine in how many collections these papers belong to. Our goal is to investigate whether readership statistics can help us to understand where and to whom DL evaluation research has impact. Therefore the data are analyzed statistically to produce indicators of geographical, and topical distribution of Mendeley Readers as well as to explore and classify their their profession. Finally it derived that there is a loose correlation between the number of Google Scholar citations and the number of Mendeley readers. Paper presented at LIDA2014, on Tuesday, July 17, 2014. Full text paper available at: http://hdl.handle.net/10889/7587
Transcript of Where and how knowledge on digital library evaluation spreads: a case study on conference literature
W H E R E A N D H O W K N O W L E D G E O N D I G I TA L L I B R A R Y E VA L U AT I O N S P R E A D S : A C A S E S T U D Y O N C O N F E R E N C E L I T E R AT U R E
LEONIDAS PAPACHRISTOPOULOS, ANGELOS MITRELIS, CHRISTOS PAPATHEODOROU DEPARTMENT OF ARCHIVES, LIBRARY SCIENCE & MUSEOLOGY, IONIAN UNIVERSITY, GREECE !GIANNIS TSAKONAS LIBRARY & INFORMATION CENTER, UNIVERSITY OF PATRAS, GREECE
A I M & S C O P E O F R E S E A R C H
• Can ‘altmetrics’ - in the form of Mendeley Readership statistics - reveal knowledge diffusion patterns?
• Can data from ‘altmetrics’ - in combination with traditional metrics - help us create quality profiles of conference papers?
A LT M E T R I C S
• A new portfolio of metrics, based on automatically processed web interactions and transactions.
• Downloads, Views, Shares, Likes, Tweets, etc.
• Alternative: contradiction with the existing system of calculation and assessment.
• Complementary: relation of ‘altmetrics’ with citations.
C O N F E R E N C E L I T E R AT U R E
• An overlooked publication venue
• Hard to index / calculate
• differences in periodicity
• unavailability of a commonly agreed quality system
• superabundance of events
M E N D E L E Y : W H Y
• A reference management system.
• A social network for scholars.
• Mendeley’s coverage, especially in cases of very specific venues, such as the conferences, has proven to be very broad.
• www.mendeley.com
R E S E A R C H S E T T I N G
• Corpus
• number of papers: 224
• domain: digital library evaluation
• sources: two conferences, JCDL and ECDL
• period: 2001-2011
• Data
• Mendeley Readership statistics
• Google Scholar citations
R E A D E R S H I P N E T W O R K
• nodes = countries
• reader’s country (vai)
• paper’s country (vbi)
• edges (vai, vbi) = state of readership
• the inclusion of a paper b in the collection of a Mendeley reader a.
W H E R E I T F L O W S : J C D L
W H E R E I T F L O W S : E C D L
R E A D E R S
0
20
40
60
80
100
Practitioner
PhD Student
Researcher
MSc Student
Faculty
27
64
25
7466
26
4853
84
97
0
28
56
84
112
140
Com
puter Science
Social Sciences
Hum
anities
Engineering
Education
Design
Psychology
10816
00
29
98
00101112
27
124
statusdiscipline E C D L J C D L
R E A D E R S v s C I TAT I O N S
0
20
40
60
80
100
0 20 40 60 80 100 120 140 160
JCDL ECDL
Google Scholar Citations
Men
del
ey R
ead
ers
Disambiguating Geographic Names in
a Historical Digital Library [2001]
Enhancing digital libraries with
TechLens [2004]
A LT M E T R I C S A S ‘ C O N F M E T R I C S ’ ?
• An altmetrics powered impact indicator
• readers
• citations
• acceptance rate
• year
• The harmonic mean of two quality rates, the readers’ and the citations’.
Conf. Cites Reads Year Acpt. Rate
Indic.
a ECDL 61 30 2003 29% 0.34
b JCDL 61 47 2007 36% 0.18
C L O S I N G R E M A R K S
• Readership: a balanced kind of metric in the landscape of ‘altmetrics’
• More elaborate statistics are needed - at least in the case of Mendeley.
• We were able to see which countries produce and which consume knowledge on digital library evaluation.
THANK YOU FOR YOUR ATTENTION !full paper at: http://hdl.handle.net/10889/7587
addendum at: http://gtsak.info/blog/gallery/lida-addendum/
!contact: [email protected] / twitter: @gtsakonas
