Ekaterina Dyachenko INTERNATIONALIZATION OF ACADEMIC ... · One of the major differences between...
Transcript of Ekaterina Dyachenko INTERNATIONALIZATION OF ACADEMIC ... · One of the major differences between...
Ekaterina Dyachenko
INTERNATIONALIZATION OF ACADEMIC JOURNALS: IS THERE STILL A GAP BETWEEN SOCIAL AND NATURAL SCIENCES?
BASIC RESEARCH PROGRAM
WORKING PAPERS
SERIES: HUMANITIES WP BRP 28/HUM/2013
This Working Paper is an output of a research project implemented
at the National Research University Higher School of Economics (HSE). Any opinions or claims contained
in this Working Paper do not necessarily reflect the views of HSE.
Ekaterina L. Dyachenko1
INTERNATIONALIZATION OF ACADEMIC JOURNALS:
IS THERE STILL A GAP BETWEEN SOCIAL
AND NATURAL SCIENCES?2
In this study we compare internationalization of academic journals in six fields of science.
Internationalization was investigated through journals' concentration on publishing papers from
particular countries, relationship between the geographical distributions of editors and authors,
and relationship between language of publication and the geographical distribution of papers.
Having analyzed more than 1000 journals we can state that social sciences literature in the fields
considered is still nationally and linguistically fragmented more than natural sciences literature,
but in some cases the gap is not so big. One of the consequences concerning research output
assessment is that usefulness of international databases having national disparity in coverage is
still limited in social sciences.
JEL Classification: Z.
Keywords: scientometrics, Web of Science, sociology, economics, political science.
1 National Research University Higher School of Economics. HSE Branch in Perm, Library.
Electronic resources librarian; E-mail: [email protected] 2 This study was carried out within “The National Research University Higher School of Economics’ Academic Fund Program in
2012-2013, research grant No.11-01-0037”
3
Introduction
While the general idea of using publication and citation counts to measure scientific
productivity and impact is widely accepted in scientific community, there is a debate lasting for
decades on how to use them properly to assess the output in social sciences and humanities.
International citation databases like Web of Science and Scopus and indicators derived from
them, like Impact Factor and h-index, quite fit the needs of those who want scientometric
instruments to be simple, transparent and universal. But the primary requirement for
measurement instruments is validity. Databases and indicators most widely used in research
evaluation were historically developed for and even now mainly oriented to natural sciences and
engineering. With social sciences being different from natural sciences in so many ways, doubts
on applicability of uniform evaluation instruments will probably not be exhausted any time soon.
One of the major differences between social sciences and humanities (SSH) and natural
sciences and engineering (NSE), important in the context of output assessment, is that SSH are
more locally oriented, therefore social sciences community is more fragmented than natural
sciences community. Generally, social scientists have more incentives to publish in national
languages and in national journals, to participate in local conferences and so on. Meanwhile all
the prominent citation databases are biased towards English-language literature. While these
databases keep expanding their coverage of literature in other languages, there is still a
considerable disparity in representation, which affects the indicators of scholars from non-
English-speaking countries [Archambault et al., 2006; de Moya-Anegon et al., 2007; Hicks &
Wang, 2011].
Another difference between scientific fields concerns the role of various publication
types in knowledge dissemination. In SSH fields books generally present a larger share of output
than in NSE fields and journal articles present a smaller share than in NSE fields [Kyvik, 2003;
Nederhof, 2006]. Since major scientometric databases cover mostly academic journals and
conference proceedings the indicators obtained from them are disadvantageous for scientists
publishing largely in books.
Besides a larger role of locally-oriented literature and non-journal literature in SSH
compared to NSE, there are other aspects which should be taken into account when assessing
research output. In social sciences and humanities publications are more often written by a single
author or in small collaborations, which probably affects productivity rate [Kyvik, 2003;
Nederhof, 2006]. NSE have a higher degree of consensus compared to SSH on what are the core
publications in the field [Hicks, 1999]. Also there are differences in citation practices,
concerning the type and age of cited documents [Larivière et al., 2006]. For some of these points
4
the adjustment of scientometric instruments is relatively easy, but not for all. In extensive
reviews written by A. J. Nederhof, D. Hicks, M. Huang and Y. Chang one can observe a plethora
of studies concerning the differences between knowledge dissemination patterns in SSH and
NSE fields [Hicks, 1999; Nederhof, 2006; Huang & Chang, 2008]. The strengths and
weaknesses of various scientometric databases and indicators are constantly debated in natural
sciences as well. Obviously, NSE do not present a homogeneous system. Publication and citation
patterns may differ considerably between disciplines and the establishment of benchmarking
practices both well-grounded and easy-to-use is far from being completed. What makes SSH
fields stand apart in this debate is that there is still no database widely agreed upon as an
adequate source of bibliographic data for research evaluation.
Today there are three multidisciplinary international citation databases widely used for
research evaluation – Web of Science, Scopus and Google Scholar. Web of Science, the product
of Thomson Reuters, consists of several citation indexes and covers journals, books and
conference proceedings. It is most valued for its journal coverage with more than 8500 NSE
journals and more than 3000 SSH journals indexed. Scopus, developed by Reed Elsevier, also
covers journals, proceedings and books. As of November 2012 Scopus had about 19500 journals
indexed with about 22% of them being SSH. Google Scholar is a web search engine indexing
scholarly literature and enhanced with citation indexing. It covers different types of publications
and its coverage is known to be extensive, but the exact details are not provided by the owner.
For a number of reasons, with coverage ambiguity being not the least, Google Scholar, although
highly appreciated in scientific community, is less widely used for scientometric purposes
compared to Web of Science and Scopus. In addition to these international databases scholars
and administrators evaluate SSH output using national/regional publication and citation
databases, such as ‘Russian Scientific Citation Index’ or ‘Flemish Academic Bibliographic
Database for the Social Sciences and Humanities’, and journal lists developed for national
evaluation initiatives, such as ‘Australian Excellence in Research Journal List’. But the
evaluation based on these sources has a limited scope, particularly when it comes to evaluation
of research impact or cross-national comparisons.
The usefulness of different databases for SSH research evaluation has been analyzed in a
number of studies. D. Hicks and J. Wang pointed out several types of methodology used in these
studies [Hicks & Wang, 2011]. Sometimes to compare the databases authors calculate the share
of national or institutional bibliography covered by these databases [Norris & Oppenheim,
2007]. Another type of methodology is database overlap analysis [Norris & Oppenheim, 2007;
Vieira & Gomes, 2009; Hicks & Wang, 2011]. The third approach is to compare the database to
some comprehensive source of bibliography, such as Ulrich’s database for the journal literature
5
[Archambault et al., 2006; de Moya-Anegon et al., 2007; Hicks & Wang, 2011]. When
international citation databases, Web of Science, Scopus and Google Scholar, are analyzed from
this perspective, in most cases the coverage of SSH literature is claimed to be wanting, with
scarce coverage of books and national/linguistic disparity as the weak points.
This study deals with one of the aspects of SSH evaluation problem, namely with the
national/linguistic disparity in literature coverage in international databases. The study does not
focus on characteristics of specific database though. Rather we try to determine whether
national/linguistic disparity is still important, in other words, whether SSH literature is still
fragmented. The scientific landscape is changing constantly, and the changes have come at a
great rate in the last two decades. So the question is: With globalization as one of the major
trends of world development, with Internet making dramatic differences in scientific
communication, with scholars being under substantial pressure to publish in English-language
journals – what if SSH community and SSH literature are not so fragmented today as we used to
think?
The goal of the study is to compare the levels of internationalization for several fields of
science. For the comparison we chose three NSE fields – Applied Physics, Nanoscience,
Biochemistry, and three SSH fields - Sociology, Economics and Political Science. To compare
the levels of internationalization we analyzed the leading international journals in these fields.
Having analyzed more than 1000 journals we can state that SSH literature in the fields
considered is still nationally and linguistically fragmented more than that of NSE fields. Still in
some cases the gap is not so big. For one of the NSE fields the publication patterns we observed
were close to those of SSH.
Methodology
While Web of Science (WoS) coverage is often claimed to be too narrow to benchmark
SSH output, when it comes to separating high-quality journals from low-quality ones it is WoS-
covered journals which present the area of consensus [Hicks & Wang, 2011]. Aiming to consider
the leading international journals in the selected fields we included all publications from the
respective subject categories of Journal Citation Reports (JCR) – “Sociology”, “Political
science”, “Economics”, “Physics, applied”, “Nanoscience & Nanotechnology”, “Biochemistry &
Molecular Biology” – in the analysis. Journal lists were extracted from the latest available
6
version of the database, JCR 2011. All these journals are indexed in Web of Science (WoS) and
almost all have 2011 Impact Factor3.
Totally, the set of journals included in the study consisted of 577 SSH and 453 NSE
publications. The JCR journal lists for chosen fields overlap to some extent. Figure 1 presents the
Venn diagrams of the overlaps. There are 26 SSH journals and 27 NSE journals belonging to
more than one JCR subject category selected for study. For Economics and Biochemistry the
total number of journals is slightly different from that in the JCR subject categories because we
counted merged journals as a single entity while JCR does not. We excluded one Biochemistry
journal from the analysis because in 2010 it transformed and left the subject category. Apart
from considering the whole set of journals in each field we were interested in analyzing high-
impact journals. From each of the six journal lists ranked by the Impact Factor we picked top
25 % journals to consider them high-impact4.
Figure 1. Overlaps of journal sets selected for study.
We obtained the following data for each journal:
journal language and whether the journal is translated or not,
the number of papers published in 2010-2011,
3 There are several publications that are present in JCR Science Edition 2011 but their Impact Factor cells are empty, because
they have no papers published in 2009-2010 and indexed in WoS. 4 The use of Impact Factor quartiles to classify journals by impact level is a common technique, but when the validity of
classification is crucial, for example in national research evaluation systems, this approach is contested [e.g. Garcia et al., 2012]
7
the most publishing country in the journal (i.e. country represented by the largest number
of papers in the journal), the number of papers from this country5,
the number of papers written in foreign language,
the country of origin of editor-in-chief (‘editor’ for short),
the country represented by the largest number of members on the editorial board.
Table 1 shows the number of journals and papers by field. All types of documents
published in the journals during 2010-2011 and indexed in WoS were included in the analysis.
Papers were assigned to countries according to the addresses of authors’ affiliations (full
counting method was used). These data were obtained from WoS between October and
December 2012. Papers containing no address information in WoS were omitted. The share of
such documents varies from journal to journal, on the whole it comprises a negligible fraction of
papers published in NSE journals, a considerable share in Political science, with Sociology and
Economics lying between.
Table 1. The number of journal and papers included in study.
Journals Papers (2010-2011),
total
Papers (2010-2011)
not assigned to any
country (% of total)
Sociology 138 15035 1457 (9.69%)
Political science 149 20366 4939 (24.25%)
Economics 317 39553 1700 (4.30%)
SSH, total 577 72125 7838 (10.87%)
Applied physics 125 92405 1185 (1.28%)
Nanoscience &
Nanotechnology
66 45904 555 (1.21%)
Biochemistry &
Molecular biology
289 127338 2798 (2.20%)
NSE, total 453 246261 4390 (1.78%)
The data on journal languages were taken from JCR and checked in WoS. We also
searched journals websites to find whether a journal is translated (by the publisher) or not.
Countries were associated with languages on the basis of The World Factbook6. The first and all
'official' languages were associated with the country.
We associated editors and editorial board members with countries according to their
affiliations’ addresses, which were obtained from journals websites. The data from journals
websites were collected between January and March 2013. Not all the required data were
available for all the journals. The details are provided in the further analysis.
5 We also obtained data on the most publishing country for a 10-year time span, 2002-2011. 6 https://www.cia.gov/library/publications/the-world-factbook/fields/2098.html
8
Results
When assessing the internationalization of some discipline through its academic journals
one has a wide variety of options: to analyze geographic diversity of authors, or co-authorship
patterns, or citation patterns, or geographic diversity of editors, or even the diversity of
subscriptions. We chose to investigate the internationalization expressed in
a) concentration on publishing papers from particular countries,
b) relationship between the geographical distribution of editors and papers,
c) relationship between language of publication and the geographical distribution of
papers.
First we will give a picture of the geographical distribution of journals included in the
study. Table 2 presents the distribution of journals by language of publication. One can see that
while the NSE sample consists almost entirely of English-language journals, in all the three SSH
fields the proportion of non-English language journals is more discernible, albeit not large.
Table 2. Journal distribution by language of publication according to JCR. Three most
represented languages in each field.
N 1st 2nd 3rd Multi-
language
The rest
Sociology 138 English
81.9%
German, French,
Spanish 2.2% each
5.8%
8 languages
5.8%
Political science 149 English
87.2%
Spanish
4%
German
3.4%
2%
3 languages
3.4%
Economics 317 English
88%
Spanish
2.8%
French
0.9%
5.4%
9 languages
2.8%
SSH, total
577 English
86.1%
Spanish
3.1%
German
1.6%
4.7%
14
languages
4.5%
Applied physics 125 English
96.8%
3.2%
Nanoscience &
Nanotechnology
66 English
100%
Biochemistry &
Molecular
biology
289 English
94.1%*
Chinese, Danish,
Serbian, Turkish
0.3% each
4.5%
NSE, total 453 English
95.4%*
4 languages
0.2% each
3.8%
Note: The N column presents the total number of journals in a field. Each journal in the set has a
unique language attribution in JCR, with journals publishing papers in different languages coded
as multi-language.
*JCR mistakenly attributes Dutch to 9 journals which have all their papers published in English.
Here we count these journals as English-language ones.
9
In Table 3 one can see the journal distribution by country of editor. In all six fields about
half of journals have editors affiliated with some institution in the USA, other countries are left
far behind in this competition. Interestingly, in all the three SSH fields five most represented
countries prove to be the same, and Germany is the only non-English speaking country breaking
top-5, while in NSE fields we observe a wider diversity, and even the second-after-USA position
is contested.
Table 3. Journal distribution by country of editor. Five most represented countries in each field.
N 1st 2nd 3rd 4th 5th
Sociology
124 USA
44.4%
UK
23.4%
Germany
5.6%
Australia, Canada
4.8% each
Political science 134 USA
50%
UK
27.6%
Germany
7.5%
Australia
6%
Canada
4.5%
Economics 281 USA
52.7%
UK
19.6%
Germany
9.3%
Canada
5.7%
Australia
5.0%
Applied physics 104 USA
42.3%
Germany
13.5%
UK
9.6%
France, Japan, Russia
6.7% each
Nanoscience &
Nanotechnology
59 USA
57.6%
Germany
10.2%
China, Japan, UK
8.5% each
Biochemistry &
Molecular
biology
245 USA
50.2%
UK
18%
Germany
9.8%
Canada
6.5%
France
6.1%
Note: The N column presents the number of journals for which the data on editors affiliations
were available. This value is used as denominator for calculating the shares.
Journal can be attributed to more than one country, so the shares in each row are comparable but
non-summable.
There exists wide agreement in scientometric research and practice that a paper should be
attributed to a country according to the institutional affiliation of its authors. As for journals
there is no common practice. Traditionally a journal has been attributed to a country by the
location of the publishing company. Today when we have journals with international editorial
boards and authors from all over the world, with editorial office located in one country and the
publishing company located in the other, the country attribution based on the location of the
publishing company is rather a convention. In addition, there is no agreement on this matter even
among the leading journal databases. For example, JCR and SCImago databases provide
different data on the journal’s country of origin for about 30% of the SSH journals we
considered. In this study we will not use journal’s country of origin data provided by any
particular database. We just note that the journal distribution by country according to location of
publishing company (data being taken from JCR) would be less biased towards the USA in all
the six fields when compared with Table 3, but still the USA would be the dominating country,
10
and in all the six fields one of the prominent positions would be taken by the Netherlands due to
big publishing companies located in this country.
Figure 2 presents the distribution of journals based on the most publishing country. We
classified journals in each field into three groups – journals where the most publishing country is
the USA, journals where some other English-speaking country is the most publishing, journals
where some non-English-speaking country is the most publishing – and presented the proportion
of each group on the graph. One can see that in all the three SSH fields and Biochemistry the
largest group consists of journals having the USA as the most publishing country. In Applied
Physics and Nanoscience the largest group is the one where some non-English language
countries are the most publishing. China was the leading country within this group. Concerning
the group “another English language country” we should note that while in all SSH fields its
share is considerable, in all NSE fields it is negligible.
Figure 2. Journal distribution by the most publishing country in the journal.
The distributions we presented here to describe the set of journals selected for study are
also useful for assessment of the WoS coverage in particular fields. However, one should
interpret them with caution and not to mix up a leading position with over-representation. When
E. Archambault with colleagues assessed national and linguistic journal coverage rates in WoS,
they reported that in SSH the only other over-represented language of publication besides
English was Czech, which was rather unexpected, because the number of WoS-indexed journals
in Czech was small. Still, their proportion was higher than the proportion of journals in Czech
registered in Ulrich’s, which was used as a benchmarking tool, and this resulted in the reported
over-representation [Archambault et al., 2006]. In the same study one can find that while
journals with USA-affiliated editors were over-represented in WoS for both NSE and SSH fields,
still the level of over-representation was not so high for the USA as for the UK, Switzerland and
Germany in NSE, and for the UK and Russia in SSH. Another important point, made by the
11
authors is that it is not geographic disparity in coverage per se which should be interpreted as
database fault in the context of research assessment. After all, NSE journal representation is also
unbalanced but this does not cause much critique of WoS-based indicators. It is the lower level
of internationalization in SSH which makes disparity matter. To see if there is indeed a
significant difference in internationalization we will start with comparing the concentration on
publishing papers from particular countries.
a) Concentration on publishing papers from particular countries.
In this part of analysis we aimed to determine for which fields journals concentrated on
publishing authors from a particular country are typical, and for which they are not typical. For
each journal we calculated the share of papers coming from the most publishing country in this
journal. Distribution of this indicator, which we will call max-share, was used to compare
internationalization of disciplines. It is worth noting that when it comes to measuring a national
orientation of a particular journal, max-share is considered to be unsatisfactory measure. For
example, M. Zitt with colleagues noted that for such a purpose relative indexes should be used
instead, meaning that to judge on national orientation of a journal one should compare
distribution of papers in this journal with distribution of papers in the whole discipline [Zitt et
al., 2003]. However, to implement this approach one needs to know the geographic profile of the
discipline, which is a problem for SSH as long as we do not have a valid benchmarking tool.
Anyway, here we used max-share not to evaluate separate journals, but to judge on a discipline
as a whole.
The calculated max-share proved to have a wide range of values within each journal set.
It started from less than 20% in each field and ended at 100% in all disciplines except
Nanoscience. Figure 3 presents the median max-share for each discipline. One can see that for all
NSE fields the median max-share does not exceed 40% (34% in Applied Physics, 35% in
Nanoscience and 36% in Biochemistry), which is considerably lower than the median max-share
for any of three SSH fields (69% in Sociology, 61% in Political Science, 49% in Economics).
When comparing two combined samples – SSH journals (n=575, the median is 55%) and NSE
journals (n=451, the median is 36%) – the distributions in the two groups prove to differ
significantly (Mann–Whitney U test, p < 0.05 two-tailed).
12
Figure 3. The median share of papers from the most publishing country in a journal
Figure 3 also shows the median share of papers from the most publishing country for
high-impact journals in each field. One can observe that for most fields, all NSE fields among
them, the median for high-impact journals was above the general level. But in Sociology the
picture was different, with high-impact journals less focused on publishing authors from a
particular country as compared with all Sociology journals. As for the cross-field comparison,
the median max-share of high-impact journals in any NSE field again was lower than that of any
SSH field. The difference between distributions in combined samples of high-impact SSH
journals (n=143, the median is 61%) and high-impact NSE journals (n=114, the median is 44%)
was also significant (Mann–Whitney U test, p < 0.05 two-tailed).
Finally, to find disciplines for which the most publishing country is a relatively stable
characteristic of a journal we compared the most publishing countries for 2010-2011 papers and
for 2002-2011 papers in each journal7. We found that only 3% of Sociology journals have
different most publishing countries in these time-spans. In Political Science and Economics the
shares of journals which have had a recent shift in national profile were also low - 6% and 8%
respectively. A similar indicator in Biochemistry was 14%, which does not differ considerably
from values in SSH fields. The highest rates of altered national profile were seen in Applied
Physics (30%) and Nanoscience (26%). In these two fields the shift occurred most often in favor
of China.
In summary, we can say that SSH journals are more concentrated on publishing authors
from a particular country than NSE journals. The same holds true when only high-impact
journals are considered instead of all journals in each field.
7 Only journals indexed in WoS throughout 2002-2011 were included in this analysis.
13
b) Relationship between the geographical distribution of editors and papers.
In the previous part we used the concentration of papers from the most publishing
country in a journal to compare the internationalization of fields, premising on the assertion that
in less internationalized fields journals in general are more concentrated. Another approach is
based on the assumption that in less internationalized fields journals in general have a closer
relationship between editors’ and authors’ geographical distribution. In order to compare
internationalization levels for six fields we obtained for each field 1) the proportion of journals
for which the most publishing country in a journal is the country where editor works (or one of
editors), and 2) the proportion of journals for which the most publishing country is the same as
the ‘most editing’ country (the country most represented on the editorial board). The first
proportion for each field is shown in Figure 4.
Figure 4. Share of journals for which the most publishing country is the country of editor.
Note: The number of journals in a field for which the data on the most publishing country and
editor’s country were available is taken as 100%. Such journals present between 83% and 90%
of the total set of journals depending on the field.
We can see that the situation when the most publishing country in a journal is the country
where the editor works is quite typical for all fields. Still, the difference between SSH and NSE
fields is considerable. In NSE such journals account for about half in each field (56% in
Biochemistry, 49% in Applied Physics and 51% in Nanoscience). If we considered the country
of editor as the journal’s country of origin, we could say that in about half of NSE journals the
most publishing country is not “home” country. Meanwhile, in all the three SSH fields such
journals account for less than a quarter (16% in Political Science, 22% in Sociology and 24% in
Economics). Figure 4 also represents the share of high-impact journals in each field for which
the country leading in terms of paper counts is the country where the editor works. For all fields
except Nanoscience there is no big difference between all journals and high-impact journals. As
14
for high-impact Nanoscience journals one should take into account that only 13 journals
qualified for this part of analysis, while in other fields number of journals was much higher,
which could be the reason why Nanoscience is shown as an outlier in Figure 4.
In Figure 5 one can see the share of journals in each field for which most publishing
country is the country most represented in the editorial board. First we should note that in all the
six fields the national composition of editorial board is more predictive of the most publishing
country in a journal than the country of the editor-in-chief. Another important thing is that in all
SSH fields a vast majority of journals (91% in Sociology, 84% in Political Science and 87% in
Economics) have the same country as the most publishing and the most represented on the
editorial board. Biochemistry also has a rather high proportion of such journals. Although its
proportion is lower than in any of three SSH fields, it would be factitious to contrast NSE and
SSH here because Biochemistry with 75% of such journals is closer to SSH fields than to other
NSE fields (both Applied Physics and Nanoscience have 54% of such journals).
Figure 5. Share of journals for which the most publishing country is the country most
represented in editorial board.
Note: The number of journals in a field for which both data on most publishing country and
country most represented on the editorial board were available is taken as 100%. Such journals
present between 83% and 99% of the total set of journals depending on the field.
As for those relatively few SSH journals which have the most publishing country
different from the ‘most editing’ country we were interested if this property could be easily
derived from the profile of the editorial board. In his study of three fields (Business, Genetics
and Political Science) T.E. Nisonger had shown that journals published in the USA have a lower
proportion of international members in editorial boards than journals published in other countries
[Nisonger, 2002]. Later, Anne-Wil Harzing and Isabel Metz in a study of geographical diversity
in editorial boards of management journals found a strong evidence that journals with USA-
15
based editor have a lower proportion of non-home country editorial board membership than
journals with non-USA based editor8 [Harzing & Metz, 2013]. We wondered if the same pattern
could be applied to editors-authors relationship, in other words, if journals having the USA as the
most represented country on the editorial board tend to give priority to home authors more often
than other SSH journals. Having compared two groups of SSH journals – those which have
USA-oriented editorial boards and those with non-USA-oriented editorial boards – we did not
find a considerable difference in the proportion of the journals that have the same country
leading in paper counts and editorial board members. Thus, the hypothesis of the USA/non-USA
orientation of editorial board being the discriminating factor could be rejected.
c) Relationship between the language of publication and the geographical distribution of
papers.
When a study of internationalization of a journal or a scientific field is based on ‘country’
category, the implicit assumption is that the barriers between different nations are of the same
height, which in fact is a simplistic view. Language is one of the most important shaping factors
of scientific communication. With high quality English-language journals posing strict
requirements for the language level of submitted manuscripts and (quite often) refusing to
consider poorly written papers, it is more difficult for scholars from non-English-speaking
countries to prepare a paper for publication. One can suppose that in social sciences and
especially humanities where the language constructions are far less standardized the language
barrier is especially perceptible.
In this part of analysis we considered internationalization in context of language barriers.
For each journal we found a number of papers written by authors for whom the journal language
is not native. For example, for English-language journals we counted papers without authors
from English-speaking countries. And then we compared six fields by the share of papers written
in a foreign language. We excluded multi-language and translated journals and journals which
did not have papers attributed to any country from consideration. Journals left after the filtering
process present between 80% and 97% of the total set depending on the field. We should note
that applying these criteria resulted in a set consisting almost entirely of English-language
journals. More specific, in NSE fields only English-language publications were left for
consideration (403 journals), in SSH fields about 2.4% of journals included in language analysis
were non-English publications (12 of 493 journals).
8 The home country of the journal was measured in this study as the country of affiliation of the editor
16
In Figure 6 one can see the share of papers published in a non-native language in each
discipline. Economics has a considerably higher share of foreign-language papers (35%) than
Sociology and Political Science (18% and 20%). Still the difference of all the SSH fields with
Applied Physics and Nanoscience (65% and 58%) is crucial. In these two fields a typical paper is
a paper written in a foreign language. As for Biochemistry (46%) it is again closer to Economics
than to other NSE fields. High-impact journals did not show a significant difference from all
journals.
One can assume that the presence of non-English language publications in SSH set could
affect the difference between NSE and SSH. To examine this we calculated the share of foreign-
language papers in three SSH fields considering only English language journals. The results
were nearly the same as those presented in Figure 6 (18% in Sociology, 19% in Political Science,
and 36% in Economics).
Figure 6. Share of papers written in a foreign language in each field.
If we consider the distributions of the share of foreign-language papers by journals, the
medians will give a picture quite close to that shown in Figure 6. According to Mann–Whitney U
test the distribution in a combined SSH sample (n=493, the median share of foreign-language
papers is 24%) differs significantly (p < 0.05 two-tailed) from the distribution in NSE (n=403,
the median is 55%). The same holds true for high-impact journals (the medians are 17% for SSH
and 39% for NSE respectively).
It is worth noting that while the proportion of foreign-language papers in SSH fields is
considerably less than half, we have quite a few SSH journals where this proportion exceeds
50%, with several high-impact publications among them, for example, “European Sociological
Review” and “European Journal of Social Theory” in Sociology, “European Journal Of Political
17
Research” and “European Union Politics” in Political Science, “Technological and Economic
Development of Economy” in Economics9.
To summarize this section we can state that while the usage of English is growing and
now we even have examples of non-English language regions where the largest proportion of
officially recognized SSH output is published in English [Engels et al., 2012], we still can not
refer to English as to a universal communication language in SSH. While in some fields
(Applied Physics and Nanoscience, for example) a typical paper is a paper written in foreign
language, in SSH fields such papers comprise far less than half. We found that Sociology and
Political Science have the most visible language barriers among the disciplines considered, with
Biochemistry and Economics taking an intermediate position in this cross-field comparison.
Conclusions
In this study we compared internationalization of six disciplines, three from social
sciences and three from natural sciences. To investigate internationalization of a field we
analyzed how academic journals from this field are concentrated on publishing papers from
particular countries, how the editorial board composition is predictive of the geographical
distribution of published papers, and how big the share of papers written in foreign language is.
Speaking of the limitations of the study we should note that indicators we have used are
quite crude. Using dichotomies such as home/non-home, native/foreign is a widely accepted
practice in studies of internationalization [e.g. Nisonger, 2002; Harzing & Metz, 2013]. Still, it is
easy to build up examples when the results based on this approach are wrong. Robust indicators
of internationalization are those that not only reflect weights of home and non-home groups but
also reflect diversity. Another limitation of the study concerns the language analysis. We used
authors’ affiliations to assign a paper to “native language” or “foreign language” category.
Today with increasing international mobility being one of global trends in science, an
unprecedented and still growing number of researchers are working outside their native
countries. According to survey data in some countries, including major players such as the
United States, Switzerland, Canada and Australia, close to half or even more than half of
researchers have come from abroad [Van Noorden, 2012]. To regard the language of the country
where researcher works at the moment as his native language could be considered as somewhat
out-of-date.
9 This journal with another Lithuanian publication “Journal of Business, Economics and Management” to a large extent provide
high Impact Factor values for each other.
18
Accepting these limitations, we can state that the results we obtained show that SSH
fields, taken together or separately, are less internationalized than NSE fields. To be more
specific, SSH journals in general are more concentrated on publishing papers from particular
countries, they tend to publish fewer papers written in foreign language, and they expose more
similarities between editors’ and authors’ geographical distribution. Despite the evidence of
growing internationalization of social sciences and humanities [Kyvik, 2003; Archambault et al.,
2006; Engels et al., 2012; Harzing & Metz, 2013], the difference between the fields is still
considerable.
We should point out that according to our results, contrasting of SSH and NSE fields is
quite factitious. With Applied Physics and Nanoscience demonstrating high internationalization
and Sociology and Political Science occupying the other pole, the other two fields, Economics
and Biochemistry, have intermediate and quite close positions. It is quite possible that having
considered a wide variety of disciplines we will find some NSE fields less internationalized that
some SSH fields. Still, in context of research output assessment the same level of
internationalization should be interpreted differently for different fields. For most NSE
disciplines we do not have many reasons to consider them locally-oriented. In such cases we can
suppose that a relatively low level of internationalization is due to the existence of a considerable
disparity in countries’ scientific performance, and to be a valid measuring tool it is not necessary
for a bibliographic database to be geographically balanced as long as we know that most high-
impact publications are covered. On the contrary, in SSH fields, we can assume that at least in
part the local orientation of the problems studied affects the level of internationalization, and if
internationalization of a particular field is relatively low we can not ignore the disparity of
databases when using them to assess scientific performance.
Nowadays multidisciplinary international databases like Web of Science and Scopus
have become a little less than a standard tools used to assess and compare research performance
at the individual, institutional, or national level. When we use such a database to assess the
output in some SSH field, which has the majority of papers published in home journals and in
native languages, the indicators we obtain are heavily influenced by the database coverage
biases. When the presence of one journal in a database changes dramatically the performance
indicators for university [e.g. Gorraiz et al., 2009], or even for the whole country [e.g. Savelieva
& Poletayev, 2009], we should recognize that using this database is insufficient for obtaining the
adequate picture.
We want to underline that the international databases we mentioned are invaluable tools
for evaluation of internationally oriented research. But to obtain a comprehensive assessment of
not only internationally oriented but all SSH research performed by some university or some
19
country today one should have, in addition to international databases, an extensive, quality-based
and, desirably, citation-indexing national bibliographic database. One can only hope that some
day we will have scientometric tools powerful enough to make cross-field and cross-country
comparisons and balanced enough so these comparisons would be valid.
References
Archambault, E., Vignola-Gagne, E., Côté, G., Larivière, V., & Gingras, Y. (2006).
Benchmarking scientific output in the social sciences and humanities: the limits of existing
databases. Scientometrics, 68(3), 329-342.
Basu, A. (2010). Does a country’s scientific ‘productivity’ depend critically on the number of
country journals indexed? Scientometrics, 82(3), 507-516.
de Moya-Anegon, F., Chinchilla-Rodriguez, Z., Vargas-Quesada, B., Corera-Alvarez, E.,
Munoz-Fernandez, F.J., Gonzalez-Molina, A., & Herrero-Solana, V. (2007). Coverage
analysis of Scopus: A metric approach. Scientometrics, 73(1), 53–78.
Engels, T.C.E., Ossenblok, T.L.B., & Spruyt, E.H.J. (2012). Changing publication patterns in the
Social Sciences and Humanities, 2000–2009. Scientometrics, 93(2), 373-390.
Garcia, J.A., Rodriguez-Sanchez, R., Fdez-Valdivia, J., & Martinez-Baena, J. (2012). On first
quartile journals which are not of highest impact. Scientometrics, 90(3), 925-943.
Gorraiz, J., Greil, M., Mayer, W., Reimann, R., & Schiebel, E. (2009). International publication
output and research impact in social sciences: comparison of the Universities of Vienna,
Zurich and Oslo. Research Evaluation, 18(3), 221-232.
Harzing, A.-W., & Metz, I. (2013). Practicing what we preach. Management International
Review, 53(2), 169-187.
Hicks, D. (1999). The difficulty of achieving full coverage of international social science
literature and the bibliometric consequences. Scientometrics, 44(2), 193-215.
20
Hicks, D., & Wang, J. (2011). Coverage and overlap of the new social sciences and humanities
journal lists. Journal of the American Society for Information Science and Technology,
62(2), 284-294.
Huang, M., & Chang, Y. (2008). Characteristics of research output in social sciences and
humanities: from a research evaluation perspective. Journal of the American Society for
Information Science and Technology, 59(11), 1819-1828.
Kirchik, O., Gingras, Y., & Larivière, V. (2012). Changes in publication languages and citation
practices and their effect on the scientific impact of Russian science (1993-2010). Journal
of the American Society for Information Science and Technology, 63(7), 1411-1419.
Kyvik, S. (2003). Changing trends in publishing behaviour among university faculty, 1980-2000.
Scientometrics, 58(1), 35-48.
Larivière, V., Archambault, E., Gingras, Y., & Vignola-Gagné, E. (2006). The Place of Serials in
Referencing Practices: Comparing Natural Sciences and Engineering With Social Sciences
and Humanities. Journal of the American Society for Information Science and Technology,
57(8), 997-1004.
Nederhof, A. (2006). Bibliometric monitoring of research performance in the Social Sciences
and the Humanities: A review. Scientometrics, 66(1), 81-100.
Nisonger, T.E. (2002). The relationship between international editorial board composition and
citation measures in political science, business, and genetics journals. Scientometrics,
54(2), 257–268.
Norris, M., & Oppenheim, C. (2007). Comparing alternatives to the Web of Science for coverage
of the social sciences’ literature. Journal of Informetrics, 1(2), 161-169.
Savelieva, I.M., & Poletayev, A.V. (2009). Publications of Russian Authors in Foreign Journals
in Social Sciences and Humanities in 1993–2008: Quantity and Quality: Working Paper
P6/2009/02. Moscow: State University — Higher School of Economics. (in Russian).
Van Noorden, R. (2012). Science on the move. Nature, 490(7420), 326-329.
21
Vieira, E.S., & Gomes, J.A.N.F. (2009). A comparison of Scopus and Web of Science for a
typical university. Scientometrics, 81(2), 587-600.
Zitt, M., Ramanana-Rahary, S., & Bassecoulard, E. (2003). Correcting glasses help fair
comparisons in international science landscape: Country indicators as a function of ISI
database delineation. Scientometrics, 56(2), 259-282.
22
Ekaterina L. Dyachenko
National Research University Higher School of Economics. HSE Branch in Perm, Library.
Electronic resources librarian; E-mail: [email protected]
Any opinions or claims contained in this Working Paper do not necessarily
reflect the views of HSE.