Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed...
-
Upload
madison-harvey -
Category
Documents
-
view
217 -
download
0
Transcript of Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed...
![Page 1: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/1.jpg)
Bibliometric assessment of research performance in
social sciences and humanities
Henk F. Moed
Centre for Science and Technology Studies (CWTS), Leiden University, the Netherlands
![Page 2: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/2.jpg)
2
Assessment of research performance: Basic assumptions - 1
• The concept of research quality does have a meaning in all fields of science and scholarship
• An assessment methodology should take into account the nature of the field
![Page 3: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/3.jpg)
3
Assessment of research performance: Basic assumptions - 2
• Important contributions to scholarly progress are sooner or later communicated in scholarly publications
![Page 4: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/4.jpg)
4
Research quality: Dimensions
• Contribution to scholarly progress
• Enlightenment of the general public
![Page 5: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/5.jpg)
5
Science versus humanities (Price) - 1
Science Humanities
Orientation Quantitative Qualitative
Status of knowledge
Short term permanence
Permanent significance
Organisation Groups interacting at the research front
Individuals
![Page 6: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/6.jpg)
6
Science versus humanities (Price) - 2
• Different substantive contents ask for different types of social organisation and information exchange
• Science: Citation Index
• Humanities: Normal archival library
![Page 7: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/7.jpg)
7
Social sciences
• Social sciences constitute a heterogeneous domain, with both ‘science-like’ and ‘humanities-like’ orientations
• Even sub-disciplines may be heterogeneous (e.g., sociology)
![Page 8: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/8.jpg)
8
Overall ISI coverage by main field
EXCELLENT (> 80%) VERY GOOD (60-80%) GOOD(40-60%)
Biochem & Mol Biol Appl Phys & Chem Mathematics
Biol Sci ~ Humans Biol Sci ~ Anim & Plants
Economics
Chemistry Psychol & Psychiat Engineering
Clin Medicine Geosciences MODERATE (<40 %)
Physics & Astron Soc Sci ~ Medicine & health
Other Soc Sci
Humanities & Arts
![Page 9: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/9.jpg)
9
Sub-disciplines (non-exhaustive list)
Social sciences related to medicine and health
Other social sciences
Humanities
Public environm & occupat health
Nursing
Sport sciences
Substance abuse
Sociology
Anthropology
Educational sciences
Political science
Law
Literature
Linguistics
Historical sc
Philosophy
![Page 10: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/10.jpg)
11
‘Other’ social sciences and humanities
• ‘National publication model’ is dominant
• Books play an important role
• Basic assumptions of a journal citation index are less valid
• ISI Indexes have inadequate coverage
• They also contain national journals
![Page 11: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/11.jpg)
12
Indicator of a journal’s national orientation (INO)
• The share of the papers from the country most frequently publishing in a journal
• A purely national journal would have an INO value of 100 per cent
![Page 12: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/12.jpg)
13
National orientation of journals (INO) for 4 fields
Discipline No. Journals
Median INO % Journals with
INO>90%
Physics & astronomy
260 37 10
Mol Biol & biochem
530 41 4
Humanities & arts
1,110 71 24
Other social sci
879 72 22
![Page 13: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/13.jpg)
14
Policy assumption for social sciences and humanities
The extent to which research findings
• reach beyond a purely national or local viewpoint
• and are exposed to criticisms from a wide international scholarly audience
is a relevant criterion of research quality
![Page 14: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/14.jpg)
15
The problem: ISI ≠ ‘International’
• Works exposed to an international audience are not necessarily included in the ISI Indexes
• Works included in the ISI Indexes are not necessarily exposed to an international audience
![Page 15: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/15.jpg)
16
This book argues:
• It cannot be taken for granted that the ISI Citation Indexes provide such indicators in all subfields of these domains of scholarship
• A challenge would be to systematically explore alternative data sources and methodologies
![Page 16: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/16.jpg)
17
4 Types of bibliometric studies
Cited/Target Citing/Source ISI coverage Field/Study
1 ISI ISI Excellent – Very Good
Astronomy
2 ISI+non ISI ISI Very Good – Good
Mathematics
3 ISI+non ISI ISI+non ISI Good – Moderate
Computer Science
4 No citation analysis at all Moderate NL Law
![Page 17: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/17.jpg)
18
Alternative approaches
1. Expand the WoS with additional sources
2. Classification of publications and sources based on scholars’ quality perceptions
3. Library collection analysis
![Page 18: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/18.jpg)
19
1. Expand the WoS with additional sources
• University of Granada (Spain):
• Creation of a citation index with 200 Spanish social science source journals not covered by the WoS
• 50,000 uses per year
![Page 19: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/19.jpg)
20
2. Classifications based on scholars’ quality perceptions
• Case study on Flemish Law
• Questionnaires; No citation analysis
• Publications in Dutch: 81 %
• Publications in journals: 59 %
![Page 20: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/20.jpg)
21
Classification of (national) journals based on a questionnaire into:
• Scholarly vs. non-scholarly
• Outstanding (A), good (B) and less good (C)
![Page 21: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/21.jpg)
22
The ‘best’ indicator of scholarly research performance (in Flemish Law):
Count the number of
• Single and multi-authored books (first editions only)
• PhD theses
• Publications with a length > 5 pages
![Page 22: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/22.jpg)
23
3. Library collection analysis
• Focuses on books
• Determine the number of academic library copies per book title
• Example: Use Worldcat (Linmans, CWTS, 2007)
![Page 23: Bibliometric assessment of research performance in social sciences and humanities Henk F. Moed Centre for Science and Technology Studies (CWTS), Leiden.](https://reader030.fdocuments.net/reader030/viewer/2022032414/56649ee65503460f94bf63e6/html5/thumbnails/23.jpg)
END