Published article: Buela-Casal G, Perakakis P, Taylor M ... · Jointly published by Akadémiai...

Post-print Version

Published article: Buela-Casal G, Perakakis P, Taylor M, Checa P (2006) Measuring internationality: reflections

and perspectives on academic journals. Scientometrics 67(1):45–65. doi:10.1556/Scient.67.2006.1.4.

Please address all correspondence to M. Taylor: [email protected]

Jointly published by Akadémiai Kiadó, Budapest Scientometrics, Vol. 67, No. 1 (2006) 45–65

and Springer, Dordrecht DOI: 10.1556/Scient.67.2006.1.4

Received July 11, 2005

Address for correspondence:GUALBERTO BUELA-CASALFaculty of Psychology, University of Granada, Campus Universitario Cartuja, 18011 Granada, SpainE-mail: [email protected]

0138–9130/US $ 20.00Copyright © 2006 Akadémiai Kiadó, BudapestAll rights reserved

Measuring internationality: Reflections and perspectiveson academic journals

GUALBERTO BUELA-CASAL,a PANDELIS PERAKAKIS,a MICHAEL TAYLOR,b

PURIFICACIÓN CHECAa

a Department of Psychology, University of Granada, Granada (Spain)b Department of Theoretical Physics, Autonomous University of Madrid, Madrid (Spain)

Internationality as a concept is being applied ambiguously, particularly in the world ofacademic journal publication. Although different criteria are used by scientometrists in order tomeasure internationality and to supplement its minimal literal meaning, the present study suggeststhat no single criterion alone is sufficient. This paper surveys, critically-assesses and extends theexisting measures of internationality in the context of academic publishing and identifies thosecriteria that are most clearly resolved and amenable to quantitative analysis. When applied,however, to a case study of four thematically-connected journals from the field of Health andClinical Psychology using descriptive statistics and the Gini Coefficient, the measurement ofinternationality using these criteria was found to be ambiguous. We conclude that internationalityis best viewed as a mathematically fuzzy entity and that a single measure Internationality Index,constructed from a combination of suitably weighted criteria, is the only way to unambiguouslyquantify the degree of internationality.

Introduction

Although the technological developments in the electronic era have introduced newmodes of interaction between academics, publications in academic journals, either inprint form or online, remain a vital element in the process of academic communicationand evaluation. In particular, publications in prestigious journals are used as importantquality indicators. This emphasizes the need for carefully contemplated criteria andquantitative indices to assess journal information.

Several criteria have been proposed; one of them being the internationality of ajournal. This criterion is constantly receiving more attention in a world that is tendingtoward a globalization of ideas. Thus, a single quantitative measure of internationalityin the form of an internationality index, akin to the Journal Impact Factor, would be anindispensable tool in the hands of authors, readers, editors, publishers and generallyanyone interested in the evaluation of journals and consequently of the articles

G. BUELA-CASAL et al.: Measuring internationality

46 Scientometrics 67 (2006)

published in such journals. In this paper, we lay the foundation for the possibledevelopment of such an index by identifying its principal components.

It should be made clear that internationality per se is not to be equated with quality.It is quite possible that a non-global, non-international journal may publish the best,most advanced and innovative research. We speculate, however, that journals withwider national representation could increase the diversity of ideas and criticisms and bebeneficial to the advancement of knowledge. Therefore, in this paper we focus on theidentification of a complete set of factors necessary to accurately define and measureinternationality in relation to academic journals publishing.

We first reveal the problems associated with the lack of a suitable definition ofInternationality or International Journal and address common misconceptions. Then, thecriteria that have been used to measure Internationality are presented, extended andcritically assessed. We identify those criteria that form a set of well-defined andquantifiable measures and use them in a document analysis of journal articles from thethematic area of Clinical and Health Psychology. This analysis leads to the convictionthat internationality is best viewed as a fuzzy set comprising several weighted factors,an approach that could allow the construction of an unambiguous and quantifiablemeasure of Internationality in the form of an Internationality Index. We conclude with areview of our findings and a perspective for the future of internationality.

Internationality and international journal: Undefined concepts

Two central problems facing the measurement of the internationality of academicjournals are:

1. Global under-representation as a result of the use of the literal definition ofthe word international in its minimal sense and,

2. Widespread usage of the term international without any evaluation of itsdegree with reference to appropriate measures or indices.

The first is a linguistic problem which arises from the literal meaning of the word“international”. Literally, international means “relating to, or affecting two or morenations”.1 Thus, a product of the collaboration between two countries would qualify asinternational. This is an observation that should always be kept in mind whendiscussing internationality and academic journals, since it is obvious that there is a greatdifference between a journal publishing articles by authors from two neighboringcountries and another journal publishing authors from all over the globe; i.e.multinational. Nevertheless, both of these journals would literally be classified asinternational with regard to authorship. In the literature it is common to seescientometrists using the terms “international” and “multinational” interchangeably withmultinational loosely meaning “more international”. This vagueness, we argue is due to


Scientometrics 67 (2006) 47

the lack of clear definition of what international is or means. Generally, whencontemplating an international journal one expects that it not only fulfills the literaldefinition, but that it also reflects a global perspective. This suggests the need for aquantitative index to evaluate the degree of internationality.

The second problem is that, in spite of the literal ambiguity around the term“international”, the characterization international is also being used extensively withoutqualification. Although this could clearly manifest itself in many distinct disciplines, wewould like to give some examples from the area of Social and Behavioural Sciences toillustrate the point.

Internationality as perceived in Social and Behavioural Sciences

The Encyclopedia of Psychology of the American Psychological Association, forexample, does not define “international journal” rather, it describes other related termsthat can help us appreciate the concept of international in relation to journals. Thus, forexample, Holtzman2 asserts that “international psychology” refers to “scientific orprofessional activities (organizations, exchanges, and research enterprises) of apsychological nature involving groups of psychologists in two or more nations”.Wilpert3 claims that the constitutional aims of the International Association of AppliedPsychology (IAAP) are to “establish contact among those who, in different countries,devote themselves to scientific work in various fields of applied psychology…”. Davis4

indicates that the purpose of the International Council of Psychologists (ICP) is “toadvance psychology as a worldwide discipline. The ICP promotes internationalcommunication, understanding, and cooperation among individual psychologists of allnationalities and fields of interest”, and Hambleton5 states that the International TestCommission is an association of 24 national psychological societies “which have aninterest in improving tests and testing practices around the world”. But why should 24societies alone be globally-representative?

The International Encyclopedia of the Social and Behavioral Sciences, whose verytitle includes the word “international” and which has a multinational publishingcommittee, does not define what an “international journal” is. The most relevantreference in the encyclopedia is by Haas6 who defines international organization (IO) ashaving: “norms of conduct and processes of interaction among more than six thousand(1999) concrete organizations and their members, bureaucracies wishing to attainobjectives that are seen by them as unrealizable without cooperating across political andorganizational boundaries”. In the same encyclopedia, Jacobson,7 when describing theterm “international sciences” speaks about “the formal infrastructure that supports andpromotes international collaboration in the social and behavioral sciences whichconsists of several associations based on disciplines or sub-disciplines: the InternationalSocial Science Council (ISSC) and to a certain extent the International Council for



Science”. Again no guideline is given as to when it is appropriate to use the term“international”. Furthermore, when reviewing a specialized manual such as theInternational Handbook of Psychology,8 and other specific chapters on internationalpsychology such as those by Jing9 or David and Buchanan,10 we encountered only animplicit or explicit statement that “international” refers to “several countries”.

These examples illustrate that, although the term “international” (referring or not tojournals) is widely used, this is being done without qualification or any reference toquantitative indices. Instead we mostly encounter the term in its literal meaning(relating to two or more nations), or with other subjective interpretations such as:“different countries”, “around the world”, “worldwide”, etc. Nevertheless, a range ofcriteria have been proposed to assess journal internationality.11-13 What follows is a briefdiscussion of these criteria, supplemented by additional criteria that we have identifiedin order to establish a coherent set as a basis for a quantitative measure and therefore, amore precise definition of internationality in the context of assessing academic journals.

Internationality criteria

One of the principal aims of this work is to construct a more accurate and completeset of criteria that could be used to measure the degree of internationality. The criterialisted below have been collected from the literature and assessed. As we will argue,some criteria do not play an important role in the assessment of internationality. Wehave included additional criteria which we believe enhance and extend the set.

Publication language

The publication language (or languages) affects linguistic accessibility and thereforea journal’s internationality due to the geographical distribution of readers. Whenconsidering this criterion we find three parameters that are crucial in the assessment ofeach language’s influence on a journal’s internationality:

1. the percentage of the world population speaking each language,2. the number of countries where it the language is official, and,3. the academic impact of the language or languages used.

According to data taken from the New York Times Almanac 1988, Chinese(Mandarin) is the most widely spoken language in the world (14.75% of the worldpopulation), followed by Spanish (5.53%) and then English (5.37%). A journalpublishing all of its articles in all three of these languages would be linguistically opento 25.65% of the world population.

Concerning the number of countries where each language is official, the Englishlanguage is most prevalent (51 countries), followed by Spanish (20 countries).



A journal therefore publishing its articles in these two languages would providelinguistic access to readers from 37% of the official world states.

Finally, it has been suggested that English, being the native language of two of themost scientifically productive countries in the world (the USA and the UK) is thelanguage having the greatest academic impact. A recent study has demonstrated that achange of the publication language from German to English in two German psychologyjournals led to a significant increase in the rate of articles published by foreignauthors.14 Nevertheless, it is difficult to objectively assess the academic repercussion ofa given language since, for example, the Latindex database15 includes 13,364 registeredacademic journals published in Spanish and/or Portuguese which is significantly more thanthe 8.700 journals published in English cited in the Journal Citation Reports.16 Moreover,one should keep in mind those socio-political effects that bias or bar access topublication such as externally-imposed embargoes or internally-imposed controls ondistribution and access through censorship, classification, copyrighting and patenting.

What is apparent is that a journal’s publication language is an important althoughhard to quantify criterion and that publishing in the English language alone is not asufficient condition to classify a journal as being international.

Publication country

A journal’s publication country does not affect its internationality and a journalpublished in a foreign country is nothing more than a foreign journal. If this was not thecase, a Japanese journal would be international for a French reader, and simultaneouslya French journal would be international for a Japanese reader, while both would beinternational for a Greek reader. Publication country then, is a criterion used by thosewho confuse “international” with “foreign”. A journal, however, published in severaldifferent countries simultaneously and in different language editions would be moreinternational; something that could be facilitated with the advent of online journals andthe development of intelligent, electronic translation software, in conjunction withincreased global access to internet technology.

Inclusion of the term “international” in the title

Although this seems a reasonable criterion to consider as a qualitative index forassessing internationality, encyclopedias, associations and academic committees, ofteninclude the term “international” in the title of their journals without justification ofwhich norms are implemented to qualify them as international. Hence, thedenomination “international journal” does not guarantee internationality or globalperspective.



Inclusion in international databases

This criterion is extensively used but is an unreliable measure of internationalitysince the journals that are included in international databases such as PsycLIT, Medline,ERIC or Ingenta, are not necessarily international in nature. This criterion however,does affect the diffusion of knowledge and awareness of publications and therefore hasa bearing on the multinational distribution of readers and journal users, and hence, oninternational accessibility.

The Impact Factor

It has been demonstrated17-19 that in most research academic areas, multinationalcollaboration results in greater visibility and higher citation rates. Thus, there is anempirically-confirmed positive correlation between internationality measured bymultinational collaboration and Impact Factor. This does not mean however, that thereis a causal relationship between citation impact and internationality. The inclusion of ajournal in the catalogue of Journal Citation Reports, which automatically assigns to itan Impact Factor based on citation analysis, does not imply anything aboutinternationality in terms of journal content or with respect to the multinationaldistribution of authors. Besides, Buela-Casal et al.20 have recently demonstrated thatjournals exist which are not published in English nor are included in the JournalCitation Reports and yet have a considerable Impact Factor.

Even though the Impact Factor has been the subject of various critiques not only forits invalidity,21-26 but also due to particular deficiencies with respect to therecommended number of references per article27,28 and the number of self-references,29

in this article we do not wish to judge it as a quality indicator, rather to indicate itsinadequacy in accounting for a journal’s internationality.

Affiliation to an international institution or association

It may seem logical that a journal published by an association with an internationalreputation would be an international journal, but this depends sensitively on theinternationality of the parent institution or association. Furthermore, affiliation to aninternational institution or association typically ensures the use of internationalpublication norms; norms which, however, do not explicitly refer to internationalpublishing criteria such as the national makeup of journal committees or thegeographical distribution of authors, and are therefore not a sufficient guarantee ofinternationality. In this sense, to consider a priori that all of the journals published by,for example, the American Psychological Association as international is no less than aclear “Americanization” as opposed to “internationalization” of psychology. This is in



agreement with the idea of “academic colonization” emphasized by variousauthors.10, 30, 31 The issue here seems to be a dominance of Anglo-Saxon culture32, 33 inacademic publishing reflecting the western-dominated globalization of the past fewdecades.

Multinational distribution of the editorial board members

This refers to the international composition of editorial boards and is another of themost commonly mentioned criteria referring to journal’s internationality.34 Obviously,an editorial board comprising reviewers from various countries facilitates that therevision of the articles is made from a more international perspective. This criterionhowever, is not by itself sufficient for a journal to be international since for example, itmay draw authors only from a single country. In addition, in several cases, the“international” editorial boards attributed to some academic journals are no more thanan adornment since the actual reviewing of articles is performed by editorial membersfrom the host nation of the journal.

Furthermore, we believe that this criterion requires clarification since there is adistinction to be made between the editorial board and the pool of associate editorsselected by the editorial board to review articles “out of house”. These two groups areoften lumped together under the term “editorial committee” in the literature. Thisdistinction is highly pertinent to the degree of internationality when one considers thatthe external pool of editors tends to be both more numerous and more multinational innature. In this work, we denote editorial board members as those permanent, “in house”members of the journal and we distinguish them from the pool of associate editors thatare selected and are “out of house”. In this new sense, we believe that the multinationaldistribution of editorial board members directly influences the internationality of ajournal.

Multinational distribution of the associate editors

As described in the previous subsection, we have introduced this criterion to reflectthe effect on internationality of the associate editors selected by the “in-house” journaleditorial boards. In general, the associate editors are drawn from a wider range ofcountries meaning that they are likely to represent a more global perspective. However,since, in the majority of cases, their role is a secondary one with them reviewing articlesselected by the editorial boards, the global filtering bias may have already beenintroduced at the primary level of the editorial boards.



Multinational distribution of authors

This is probably the most common measure of journal internationality. Zitt andBassecoulard11 have argued that there are absolute and relative approaches to calculatedegrees of internationality based on the multinational distribution of authors. Theabsolute approaches use concentration indices that consider the percentage of foreignauthors in a journal’s total output, ignoring each country’s general academiccontribution. In this way, absolute indices, such as the Gini coefficient, fail todifferentiate between, for example, an 80% United States: 20% United Kingdom journaland an 80% Czech: 20% Austrian journal. Absolute indices however, are easier tocalculate and have been used in relevant studies in the past.35-40 Other indices toevaluate relative internationality have been proposed11 that use well-contemplatednormalization options to avoid biases caused by the different national academic sizes ineach area. Nevertheless, the authors also recognize the value of reference-free,concentration indices in providing a wider perspective. Despite the normalizationoptions here, the authors as the central engine of a journal represent one of the key testsof internationality.

Multinational distribution of users

A journal’s users are its readers, subscribers and citers. However, bibliometricresearch has direct access only to the geographical distribution of citers. Obviously,equalizing citers and users leaves out all readers that are not academic authors as well asthe academics that are consulting journals outside their particular research area and sodo not cite them in their work. Nevertheless, the majority of the studies that examinedthis criterion have done so through the analysis of the geographical distribution ofciters, making the assumption that it reflects the actual multinational distribution of alljournal users.35

International collaboration patterns

A journal’s capability to attract multinational collaboration is another useful andextensively investigated criterion. International collaboration is assessed by co-authorship indices that provide information about the share of internationally co-authored papers in a journal’s total publication output.41 Several authors haveinvestigated such collaboration patterns and co-authorship maps and have providedgreat insight on the fundamental mechanisms that underlie the dynamics of internationalcooperation.19,42,43 Nevertheless, it has been observed that the international collaborationlinkages between academics and research groups are not only influenced by variablessuch as the inherent cooperation tendencies of the academic community and the



academic size of each country, but also from sociological, geographical, ideological andother parameters that are especially difficult to quantify.44 In this study, in order toattempt a quantitative assessment of this criterion, we use the concept “share ofinternational articles” as a measurable following the example of Rey-Rocha and Martín-Sempere.45 By “international article” we mean an article published by at least twoauthors affiliated to institutes from two different countries. In this way, we are able todistinguish international articles from purely “domestic” articles published by authorsbased in the host journal nation, and “foreign” articles with authors based in only onecountry other than the host (e.g., Ref. 46,47).

Online access

Since 1996, the vast majority of journals have online archives of their print articles,which could increase global access. Similar to the publication language, onlineaccessibility depends sensitively on the dominant languages used in web pages andelectronic journal subscription rates. The journals selected for the bibliometric analysisof the next section all have online access to articles published after 1997.

From the above list, it is clear that there are many criteria commonly used tomeasure internationality in the context of academic journals, though only a few arerelevant to the quantitative measurement of internationality.

Pruning and testing the criteria

Methodology

Following the analysis of the set of criteria in the previous section II, we identifiedthe following core sub-set as being the most semantically precise and quantitative, inorder to define and measure internationality:

• Criterion 1: Multinational collaboration patterns (assessed through theshare of international articles as described in the previous section)

• Criterion 2: Multinational distribution of editorial board members• Criterion 3: Multinational distribution of associate editors• Criterion 4: Multinational distribution of authors.

We hypothesized that these criteria would yield different measures ofinternationality when considered separately. We also wanted to test the generally-heldsubjective impression that journals published in English from countries having a largeracademic size are more international in the context of a well-defined measurement of



internationality. In order to test our hypotheses, we conducted a thorough bibliometricanalysis of four journals from the area of clinical and health psychology:

1. Journal of Clinical Psychology (ISSN: 0021-9762),2. Health Psychology (ISSN: 0278-6133),3. International Journal of Clinical and Health Psychology

(ISSN: 1576-7329), and,4. International Journal of Psychology and Psychological Therapy

(ISSN: 1577-7057).

These journals were selected since:

• they are all connected thematically• two of them contain the term “international” in their title• two are published only in English while the other two are published in

English and an additional language (Spanish and/or Portuguese)• two are US journals, two are Spanish journals• all journals have online access.

The first two journals which are published in the United States of America aregenerally considered as two of the most prestigious journals in the area and are typicallycharacterized as international probably based on the scientific quality of the publishedarticles and the multinational distribution of their readers. Their internationalcharacterization is also due to their inclusion in highly esteemed and widely useddatabases. The last two journals on the other hand are published in Spain and althoughthey include the term “international” in their title, are commonly considered as “non-international” journals. An important point to note here is that the two journal hostnations differ substantially in academic size and, although the number of academicnationals is irrelevant to a measure of internationality, the number of academic foreignnationals working in the host nation is not. In fact, one would expect that a country witha large academic size and resources, comprising many locally-based foreign scientists,would have a greater international pool of editors and authors to draw from. However,this is not a guarantee of internationality as it depends sensitively on the internationalperspective of the journals; which is what we intend to measure.

The raw data is presented using standard techniques of descriptive statistics. Wealso have used an absolute statistical measure, the Gini Coefficient of inequality;48 suchthat a coefficient of 1 represents total inequality while 0 represents total equality. Thetabulated raw data used in the determination of these quantitative measures is availableonline.49



Results

In this test of the criteria, a total of 710 articles published by the four journals from2001–2003 (both inclusive) were analyzed. The descriptive characteristics of the fourjournals are given in Table 1. It is worth mentioning that the journals examined differwith respect to the number of issues published per year (their annual publication rate).

Table 1. Descriptive statistics of the number of published articles in each journalduring the period 2001–2003 (inclusive)

Journal Publi-

cation

languages

Founded Total

number of

published

articles

Publication

frequency

(articles

per year)

Mean

number

of articles

per year

Standard

deviation

Mean number

of articles

per issue

Standard

deviation

Journal of Clinical Psychology

English 1945 377 12 125.7 9.9 10.5 0.8

Health Psychology

English 1982 197 6 65.7 11.8 11.0 2.0

International Journal of Clinical and Health Psychology

English,Spanish,Portuguese

2001 92 3 30.7 3.9 10.2 1.3

International Journal of Psychology and Psychological Therapy

English,Spanish

2001 44 2 14.7 1.7 7.4 0.9

Health Psychology has a large standard deviation (11.8) relative to its mean annualpublication rate (65.7) owing to a doubling of its publication rate in the years 2002 and2003 relative to 2001.

Criterion 1: Multinational collaboration patterns. Table 2 presents the share ofdomestic, foreign and international articles in the total publication output for eachjournal as described in the previous section.



Table 2. Share of international articles for each journal

Articles

Journal Year Domestic Foreign International Total

2001 109 19 7 135

2002 106 14 10 130

2003 80 21 11 112

Total 295 54 28 377


% 78.25% 14.32% 7.43%

2001 43 5 1 49

2002 59 8 6 73

2003 55 13 7 75

Total 157 26 14 197

Health Psychology

% 79.70% 13.20% 7.11%

2001 11 13 3 27

2002 18 7 4 29

2003 20 12 4 36

Total 49 32 11 92


% 53.26% 34.78% 11.96%

2001 5 7 2 14

2002 10 0 3 13

2003 11 3 3 17

Total 26 10 8 44


% 59.09% 22.73% 18.18%

We observe that the International Journal of Psychology and PsychologicalTherapy has the largest share of international articles (18.18%) followed by theInternational Journal of Clinical and Health Psychology (11.96%). The two USjournals have a smaller share of international articles: Journal of Clinical Psychology(7.43%) and Health Psychology (7.11%). Also of note is that the two Spanish journals,the International Journal of Clinical and Health Psychology and the InternationalJournal of Psychology and Psychological Therapy, having the term “international” intheir titles, also have higher percentages of foreign articles (34.78% and 22.73%respectively). However, in all four journals the share of domestic articles is high (above50%) compared to the share of international articles.

Criterion 2: Multinational distribution of editorial board members. The Journal ofClinical Psychology has all five of its editorial board members from the United States,



the International Journal of Clinical and Health Psychology has four editors originatingfrom three different countries: Spain (2), Portugal (1) and Colombia (1), while theInternational Journal of Psychology and Psychological Therapy has nine editors fromfive different countries: Spain (4), USA (2), Belgium (1), Ireland (1) and Mexico (1).Data for Health Psychology is absent from this analysis as the journal refused theauthors’ request for data.

Thus, the International Journal of Psychology and Psychological Therapy has themore diverse country representation in its editorial board, followed by the InternationalJournal of Clinical and Health Psychology. Based solely on this criterion, the Journalof Clinical Psychology fails to meet even the minimal, literal definition of international.

Criterion 3: Multinational distribution of associate editors. With regard to thenationalities of associate editors appointed or selected for use in the peer-review processby the editorial board, we found that the Journal of Clinical Psychology has 85.0% USeditors, while the International Journal of Psychology and Psychological Therapy has60.3% Spanish and 22.3% North American editors. The greatest diversity in terms ofcountry representation is observed in the International Journal of Clinical and HealthPsychology which has 54.8% editors from Spain, 9.6% from USA, 9.6% from Portugal,8.2% from Colombia and 4.1% from Mexico.

To measure quantitatively this criterion, we plotted the Lorentz Curves for eachjournal and calculated the Gini Coefficient of inequality. The Lorentz Curves are shownin Figure 1.

The cumulative proportion of the countries (ranked according to the number ofeditors) is represented by the horizontal axis and the cumulative proportion of editors bythe vertical axis. The straight line is called the equality line and represents a perfectlyequal distribution of editors per country. Therefore, the further the Lorenz curve is fromthe equality line, the greater the degree of concentration and therefore inequality interms of editors from any particular country or subset of countries.

All three of the journals for which we had data for, have a high Gini Coefficient(0.65 to 0.75) reflecting their poor (i.e. globally unequal) multinational distribution. Itcan be seen that the International Journal of Clinical and Health Psychology, liesclosest to the line of equality over its whole length and thus performs best in terms ofthis criterion.



Figure 1. Lorentz curves for the Gini Coefficient measure of the inequality of themultinational distribution of associate editors

Criterion 4: Multinational distribution of authors. We ranked countries according tothe number of contributing authors for the four journals. We observed a strongconcentration of authors originating from a small proportion of countries. The first threeranked countries in all journals account cumulatively for 83%–97% of all authors. Withregard to the Journal of Clinical Psychology, the USA and Italy account for 84.2% ofthe contributing authors even though they compromise only 8.3% of all of the countriesappearing in the journal. The International Journal of Clinical and Health Psychologyhas a slightly smaller concentration, since the first country (Spain) accounts for less(61.9%) authors. Nevertheless, in the same journal we note that 83.9% of all authorsoriginate from only 25% of contributing countries.



To evaluate quantitatively the pattern of multinational distribution of authors, weagain calculated the Gini Coefficient for each journal.

Figure 2. Lorentz curves for the Gini Coefficient measure of the inequality of themultinational distribution of authors

Figure 2 shows a great degree of concentration for all four journals, meaning that asmall proportion of countries, contribute a large proportion of all published authors. TheGini Coefficients for the journals range from 0.76 to 0.86 revealing a very high degreeof inequality in each case (i.e. a poor multinational distribution of authors). Once again,the Spanish journals lie closest to the line of equality over their whole length andtherefore perform better in terms of this criterion.



Assessment of the criteria

In Table 3 we summarize the results of the internationality assessment of the fourjournals based on each of the selected criteria.

Table 3. Measurement of Journal Internationality for each criteron and journal rankingin terms of internationality

Journal

Criterion 1

Share ofinternational

articles

Criterion 2

Number of countriesrepresented in the

editorial board

Criterion 3

Inequality in thedistribution ofeditorial board

members per country(Gini Coefficient)

Criterion 4

Inequality in thedistribution of

authors percountry

(Gini Coefficient)Value Rank Value Rank Value Rank Value Rank


7.43% 3 1USA

3 0.75 2 = 0.86 4

Health Psychology

7.11% 4 – – – – 0.83 3


11.96% 2 3Spain

PortugalColombia

2 0.65 1 0.73 1


18.18% 1 5SpainUSA

BelgiumIrelandMexico

1 0.75 2 = 0.76 2

It is clear that no single criterion provides an unequivocal measure of internatio-nality since the ranking of the journals by internationality is different for each criterion.Moreover, none of the journals are ranked first (i.e. having the highest degree of inter-nationality) for all criteria. Internationality therefore, measured even with well-definedand quantitative criteria fails to be unambiguously defined when considered throughsingle criterion measures as hypothesized in the previous subsection. This suggests theneed for the development of a multi-criteria, single-valued and unambiguous measure ofinternationality in the form of a journal Internationality Index as described in thefollowing section.



Towards an internationality index

It is clear then, that internationality ranges from the unusual case of a journalrelating to or affecting only one nation (an entirely national project having a zero degreeof internationality) to the more usual case of academic journals involving and affectingmore than one nation – ranging from two (the minimal case of the literal definition) toas many as 192 nations, i.e. the number of independent world states as illustrated inFigure 3.

Figure 3. Internationality as a fuzzy set associated with the degree of geographical participation

The designation “international journal” is therefore imprecise in its literal sense, andincapable of differentiating quantitatively between, for example, a journal that circulatesbetween two neighbouring countries and one with a global representation or impact.

We have also shown that the actual measure of internationality is sensitive to thecriteria used in the evaluation and that no single criterion is sufficient. We propose thata journal Internationality Index can and should be constructed from a suitably-weightedcombination of all relevant and quantifiable criteria. In a separate paper50 we haveshown theoretically how a neuro-fuzzy system can be used to accomplish this. Untilsuch a study is performed, internationality remains ambiguous and undefined.



Conclusion

In this paper, we have reviewed the contemporary view of internationality in thecontext of academic journal publication as well as the criteria proposed in the literatureto measure its degree. We have shown how the literal definition of internationality failsto describe the degree of internationality associated with a potentially diverse range ofcountry representation. We also argued that some of the published criteria are redundantor ambiguously defined, and how several new distinctions and definitions help tocomplete the list. Furthermore, we showed how even the well quantitatively definedcriteria produce an ambiguous measure of internationality when consideredindividually.

The ambiguity of the results highlight the need for a new journal internationalityindex to be constructed and we have argued what structure such an index should have.

We believe that a valid and quantitative internationality index will help differentiatequality from internationality, two concepts that have overlapped and been confuseduntil now. It has been common practice, for example, to measure publication qualityusing reference indices such as the Impact Factor,51 or other qualitative criteriaconcerning validity and utility52 or the type of methodology of the study.53-55 Theconfusion over the concepts has most probably arisen since publication quality is alsooften measured through the perceived “internationality” of journals which themselvesare viewed as quality indicators of knowledge. As we have argued, however, qualityand internationality are not necessarily correlated. Therefore, in the absence of anunambiguous, quantitative and holistic measure of internationality, limited perceptionsbased on one or another single criterion can penetrate the area of journal evaluationdistorting the image of certain journals and affecting the assessment of academicquality.

The next step for scientometrists interested in measuring internationality, webelieve, is to suitably weight the criteria identified here using, for example, a large-scalecensus of journal data. We propose that the journal internationality index might then beunambiguously constructed using the neuro-fuzzy system we have described in acompanion paper.50 Until then, we reiterate that internationality and therefore the notionof an international journal have yet to be adequately defined.

*

We would like to thank Dr. Varvara Trachana and Professors Evaristo Jimenez and Emilio Delgado fortheir support and advice.



References

1. Merriam-Webster Dictionary (Indexed & Unabridged Edition) 2002;Available at http://www.m-w.com/cgi-bin/dictionary?book=Dictionary&va=international&x=0&y=0

2. W. HOLTZMAN, International Council of Scientific Unions. In: A. E. KAZDIN (Ed.), Encyclopedia ofPsychology. Oxford University Press, 2000, pp. 343–345.

3. B. WILPERT, International Association of Applied Psychology. In: A. E. KAZDIN (Ed.), Encyclopedia ofPsychology. Oxford University Press, 2000, pp. 336–337.

4. J. M. DAVIS, International Council of Psychologists. In: A. E. KAZDIN (Ed.), Encyclopedia ofPsychology. Oxford University Press, 2000, pp. 341–343.

5. R. HAMBLETON, International Test Comission. In: A. E. KAZDIN (Ed.), Encyclopedia of Psychology.Oxford University Press, 2000, pp. 349–350.

6. E. HASS, International Organization. In: P. B. BALTES, N. L. SMELSER (Eds), International Encyclopediaof the Social and Behavioral Science. Elsevier, 2001, pp. 7819–7824.

7. H. K. JACOBSON, International Science: Organizations and Associations. In: P. B. BALTES,N. J. SMELSER (Eds), International Encyclopedia of the Social and Behavioral Sciences. Elsevier, 2001,pp. 7834-7837.

8. K. PAWLIK, M. ROSENZWEIG, The International Handbook of Psychology, Sage, London, 2000.9. Q. JING, International psychology. In: K. PAWLIK, M. ROSENZWEIG (Eds), The International Handbook

of Psychology . Sage, 2000, pp. 570–584.10. H. P. DAVID, J. BUCHANAN, International psychology. In: D. K. FREEDHEIM, I. B. WEINER (Eds),

Handbook of Psychology: History of Psychology. John Wiley and Sons, 2002, pp. 509–534.11. M. ZITT, E. BASSECOULARD, Internationalization of scientific journals: a measurement based on

publication and citation scope, Scientometrics, 41 (1998) 255–271.12. M. ZITT, E. BASSECOULARD, Internationalization of communication – a view on the evolution of

scientific journals, Scientometrics, 46 (1999) 669–685.13. G. BUELA-CASAL, La psicología española y su proyección internacional. El problema del criterio:

internacional, calidad y castellano y/o inglés, Papeles Del Psicólogo, 79 (2001) 53–57.14. A. DINKEL, H. BERTH, A. BORKENHAGEN, E. BRAHLER, On raising the international dissemination of

german research: Does changing publication language to English attract foreign authors to publish in aGerman basic psychology research journal? Experimental Psychology , 51 (2004) 319–328.

15. Latindex Database. Available at http://www.latindex.unam.mx/16. Journal Citation Reports. Available at http://www.isinet.com/products/evaltools/jcr/jcrweb/17. W. GLÄNZEL, A. SCHUBERT, H. J. CZERWON, A bibliometric analysis of international scientific

cooperation of the European Union (1985–1995), Scientometrics, 45 (1999) 185–202.18. W. GLÄNZEL, National characteristics in international scientific co-authorship relations, Scientometrics,

51 (2001) 69–115.19. W. GLÄNZEL, C. DE LANGE, A distributional approach to multinationality measures of international

scientific collaboration, Scientometrics, 54 (2002) 75–89.20. G. BUELA-CASAL, A. GARCÍA-MEDINA, M. A. VIEDMA, V. GODOY, S. LORENZO, G. TORRES, Factor de

impacto de tres revistas españolas de psicología, Psicothema, 16 (2004) 681–689.21. E. ECHEBURÚA, Evaluación de los sexenios en la evaluación de la actividad investigadora en el ámbito

de la psicología clínica: Una primera reflexión, Análisis y Modificación de Conducta, 28 (2002)391–404.

22. G. BUELA-CASAL, Situación actual de la productividad científica de las universidades espñaolas,International Journal of Clinical and Health Psichology, 5 (2005) 175–190.

23. V. PELECHANO, Valoración de la actividad científica en psicología ¿pseudoproblema, sociologismo eidealismo?, Análisis y Modificación de Conducta, 28 (2002) 323–362.

24. J. GIL ROALES-NIETO, M. C. LUCIANO, A la calidad por the quantity (porque la Cantidad no vale).Algunas reflexiones sobre los criterios de evaluación de la calidad de la investigación en psicología,Análisis y Modificación de Conducta, 28 (2002) 431–454.



25. G. BUELA-CASAL, La evaluación de la investigación científica: el criterio de la opinión de la mayoría, elfactor de impacto, el factor de prestigio y los “Diez Mandamientos” para incrementar las citas, Análisis yModificación de Conducta, 28 (2002) 455–475.

26. G. BUELA-CASAL, Evaluación de la calidad de los artículos y de las revistas científicas. Propuesta delfactor de impacto ponderado y de un índice de calidad, Psicothema, 15 (2003) 23–35.

27. J. G. ADAIR, N. VOHRA, The explosion of knowledge, references, and citations: Psychology’s uniqueresponse to a crisis, American Psychologist, 58 (2003) 15–23.

28. C. D. GREEN, On (not) trimming one’s toenails with a bazzoka, American Psychologist, 59 (2004) 51.29. F. ANSEEL, W. DUYCK, W. DE BAENE, M. BRYSBAERT, Journal Impact Factors and Self-Citations:

Implications for Psychology Journals, American Psychologist, 59 (2004) 49–51.30. P. J. VAN STRIEN, The American “colonization” of northwest European social psychology after World

War II, Journal of the History of the Behavioral Sciences, 33 (1997) 349–363.31. C. F. GRAUMAN, Psychology in postwar Germany: The vicissitudes of internationalization, World

Psychology, 3 (1997) 253–277.32. R. ARDILA, International psychology, American Psychologist, 37 (1982) 323–329.33. R. ARDILA, La psicología en el futuro, Pirámide, Madrid, 2002.34. A. UZUN, Assessing internationality of scholarly journals through foreign authorship patterns: the case of

major journals in information science, and scientometrics, Scientometrics, 61 (2004) 457–465.35. F. H. CHRISTENSEN, P. INGWERSEN, I. WORMELL, Online determination of the journal impact factor and

its international properties, Scientometrics, 40 (1997) 529–540.36. J. C. SIERRA, I. ZUBEIDAT, Análisis bibliométrico de la revista Journal of Sex and Marital Therapy,

International Journal of Clinical and Health Psychology, 2 (2002) 351–362.37. D. AGUDELO, J. BRETÓN-LÓPEZ, G. BUELA-CASAL, Análisis comparativo de las revistas de Psicología

de la Salud editadas en castellano, Revista Latinoamericana de Psicología, 35 (2003) 359–377.38. D. AGUDELO, J. BRETÓN-LÓPEZ, G. BUELA-CASAL, Análisis bibliométrico de las revistas de Psicología

Clínica editadas en castellano, Psicothema, 14 (2003) 507–516.39. D. AGUDELO, J. BRETÓN-LÓPEZ, G. BUELA-CASAL, Análisis bibliométrico de las revistas relacionadas

con Psicología de la salud editadas en castellano, Salud Mental, 27 (2004) 70–85.40. J. BRETÓN-LÓPEZ, G. BUELA-CASAL, Análisis comparativo de las revistas de Psicología Clínica editadas

en castellano, Revista Mexicana de Psicología, 20 (2003) 141–155.41. G. MELIN, O. PERSSON, Studying research collaboration using co-authorships, Scientometrics, 36 (1996)

363–377.42. N. NARVAEZ-BERTHELEMOT, An index to measure the international collaboration of developing

countries based on the participation of national institutions – the case of Latin-America, Scientometrics,34 (1995) 37–44.

43. T. LUUKKONEN, R. J. W. TIJSSEN, O. PERSSON, G. SIVERTSEN, The measurement of internationalscientific collaboration, Scientometrics, 28 (1993) 15–36.

44. G. MELIN, Impact of national size on research collaboration – A comparison between Northern Europeanand American universities, Scientometrics, 46 (1999) 161–170.

45. J. REY-ROCHA, M. J. MARTIN-SEMPERE, Patterns of the foreign contributions in some domestic vs.international journals on Earth Sciences, Scientometrics, 59 (2004) 95–115.

46. J. VIRUES-ORTEGA, S. N. HAYNES, Functional analysis in behavioral therapy: Behavioral foundationsand clinical application, International Journal of Clinical and Health Psychology, 5 (2005) 567–587.

47. J. VIRUES-ORTEGA, Causes of unity and disunity in Psychology and Behaviorism: An encounter withArthur W. Staats’ Psychological Behaviorism. International Journal of Clinical and Health Psychology,5 (2005) 161–173.

48. P. M. DIXON, J. WEINER, T. MITCHELLOLDS, R. WOODLEY, Bootstrapping the Gini coefficient ofinequality, Ecology , 68 (1987) 1548–1551.

49. http://astro.ft.uam.es/~mtaylor/data.pdf50. P. PERAKAKIS, M. TAYLOR, G. BUELA-CASAL, P. CHECA, A neuro-fuzzy system to calculate a journal

internationality index, Symposium on Fuzzy Logic and Soft Computing, Proc. CEDI (2005), Granada,Spain. Available online at http://astro.ft.uam.es/~mtaylor/CEDI2005.pdf



51. E. GARFIELD, The meaning of the Impact Factor, International Journal of Clinical and HealthPsychology, 3 (2003) 363–369.

52. G. BUELA-CASAL, J. R. FERNÁNDEZ-HERMIDA, W. LÓPEZ, IBEINDEX: Indice iberoamericano deevaluación de publicaciones científicas, Infocop, 84 (2003) 41–44.

53. J. M. RAMOS, A. CATENA, Normas para la elaboración y revisión de artículos originales experiencialesen ciencias del comportamiento, International Journal of Clinical and Health Psichology, 4 (2004)173–189.

54. H. CARRETERDO-DIOS, C. PEREZ, Normas para el desarrollo y revisión de estudios instrumentales,International Journal of Clinical and Health Psychology, 5 (2005) 521–551.

55. I. MONTERO, O. G. LEON, Sistemas de clasificicación del método en los informes de investigación enPsicología, International Journal of Clinical and Health Psychology, 5 (2005) 99–114.

Published article: Buela-Casal G, Perakakis P, Taylor M ... · Jointly published by Akadémiai...

Documents

Transcript of Published article: Buela-Casal G, Perakakis P, Taylor M ... · Jointly published by Akadémiai...