Ways to judge a journal

Impact factor and other measures

Dilip R Karnad

What is a good journal?

Circulation – number of

copies per month?

Readership – many readers

per copy

Regular readers – browse

current issues

Referred to by researchers

or experts – specific articles

located by literature search

Scientometrics or

Journalology

Journal impact factor

1955 - Eugene Garfield in Science - Citation is key

1961 – Institute for Scientific Information - Science citation

index

1992 – bought by Thomson Scientific – now Thomson-Reuters

Impact factor = Citations in current year to articles in previous 2 years

Number of articles published in previous 2 years

Journal impact factor =Number of articles (2007,2008) from the journal cited by any journal in 2009

Number of articles published in that journal in 2007,2008

Number of years – current status = ?state of the art

Impact factors 2008

Rank Abbreviated Journal Title Impact Factor

1 Cancer Journal for Clinicians 74.5752 New England Journal of Medicine 50.0173 Annual Review of Immunology 41.0594 NATURE REVIEWS MOLECULAR CELL BIOLOGY 35.4235 PHYSIOLOGICAL REVIEWS  356 REVIEWS OF MODERN PHYSICS 33.985

7Journal of the American Medicel Association 31.718

8 NATURE 31.4349 CELL 31.253

10 NATURE REVIEWS CANCER  30.76211 NATURE GENETICS  30.25912 ANNUAL REVIEW OF BIOCHEMISTRY 30.01613 NATURE REVIEWS IMMUNOLOGY 30.00614 NATURE REVIEWS DRUG DISCOVERY 28.6915 LANCET 28.40916 SCIENCE 28.10317 NATURE MEDICINE  27.55318 ANNUAL REVIEW OF NEUROSCIENCE  26.40519 NATURE REVIEWS NEUROSCIENCE  25.94

20ANNUAL REVIEW OF ASTRONOMY AND ASTROPHYSICS  25.826

Are other parameters required?

Evaluation of quality of references cited Citation density - average number of

references cited in the new article Half-life - number of retrospective

years required to find 50% of the cited references New England Journal of Medicine:

Cited Half-Life: 7.3 years

JIF add-ons - Immediacy Index Immediacy Index is the average number of times an article

is cited in the year it is published.

The journal Immediacy Index indicates how quickly articles in a journal are cited.

Immediacy Index = number of citations to articles published in 2008

number of articles published in the journal in 2008

Frequently issued journals have an advantage.

Can identify journals specializing in cutting-edge research.

New England Journal of Medicine

Cites in 2008 to items published in 2008 = 4352 

Number of items published in 2008 = 356 

Immediacy index = 12.225   

Problems with JIF: Specialty Specialty with large number of

researchers/practitioners – more citations Large specialty:

more authors, more articles, more citations – numerator more journals, more articles to cite – denominator

JIF - Basic sciences journals >> Clinical journals Lowry OH, Rosebrough NJ, Farr AL, et al. Protein measurement

with the folin phenol reagent. J Biol Chem. 1951;193:265-275. Cited 300,000 times Southern EM. Detection of specific sequences among DNA

fragments separated by gel-electrophoresis. J Mol Biol. 1975;98:503-517.

Cited 30,000 times

Problems with JIF: Skewness

Skewness of citations The so-called 80/20 phenomenon

20% of articles account for 80% of citations

Citation rates in 1986 or 1987 of articles published in 3 biochemical journals in 1983 or 1984, respectively

Seglen, P. O BMJ 1997;314:497

Problems with JIF

Delay in manuscript acceptance, review and publication = low citation

Many articles on same topic in one issue =increases citation of all articles

Long-term impact of articles missed – 5-year or 10-year IF

Types of articles: research, review, letters, commentaries, perspectives, news stories, obituaries, editorials, interviews, and tributes

JAMA published 1905 items - 680 were letters and 253 were editorials.

Cited in same year – not included in numerator

Exclude from denominator?

What if cited in numerator in later years?

Problems with JIF

Selective journal self citation: articles tend to preferentially cite other articles in the same journal

Self author self citations

Review articles are heavily cited and inflate the impact factor of journals

Long articles collect many citations and give high journal impact factors

Short publication lag permits self citations - high journal impact factor

Journal set in database may vary from year to year

Impact factor is a function of the number of references per article in the research field

Problems with JIF: commercial

Owned by Garfield / ISI / Thomson Reuters

Includes only journals indexed in “Current

Contents” – another ISI publication – “get

indexed with us...”

Coverage of the database is not complete

Database has an English language bias

Database is dominated by American

publications

Alternatives to JIF - Eigenfactor Eigenfactor Project™ is a non-commercial academic

research project

Sponsored by the Bergstrom lab in the Department of Biology at the University of Washington.

Aim to use recent advances in network analysis and information theory to develop novel methods for evaluating the influence of scholarly periodicals and for mapping the structure of academic research.

Committed to sharing findings with interested members of the public, including librarians, journal editors, publishers, and authors of scholarly articles.

The Eigenfactor Score measures the number of times articles from the journal published in the past five years have been cited in the JCR year.

Like the Impact Factor, the Eigenfactor Score is essentially a ratio of number of citations to total number of articles. However, unlike the Impact Factor, the Eigenfactor Score:

Counts citations to journals in both the sciences and social sciences.

Eliminates self-citations. Every reference from one article in a journal to another article from the same journal is discounted.

Weights each reference according to a stochastic measure of the amount of time researchers spend reading the journal.

Alternatives to JIF – Eigenfactor Score

Alternative to JIF: Article Influence Score

Article Influence Score calculates measures the relative importance of the journal on a per-article basis.

It is journal's Eigenfactor Score divided by fraction of articles published by the journal.

That fraction is normalized so that the sum total of articles from all journals is 1.

The mean Article Influence Score is 1.00. Score > 1.00 = each article in journal has above-

average influence. Score < 1.00 = each article in the journal has below-

average influence.

Alternatives to JIF – Impact of Internet

Webometrics Hits = readership, What is the meaning of downloads? Scholar Google – gives citations

What is the significance? Cites only from journals? Citation from websites included? May not include only peer-reviewed

citations “Sitations”

Perneger TV. Br Med J 2004;329:546–7

Impact of Internet – Hits = Citations ?

Davis PM, et al. Br Med J 2008;337:a568

1:8 articles made open-

access

Davis PM, et al. Br Med J 2008;337:a568

Favaloro EJ, Semin Thromb Hemost 2008;34:7–25

Google scholar citationsVs

Science Citation Index

BMJ, doi:10.1136/bmj.38422.611736.E0 (published 12 April 2005)

BMJ, doi:10.1136/bmj.38422.611736.E0 (published 12 April 2005)

BMJ, doi:10.1136/bmj.38422.611736.E0 (published 12 April 2005)

Thank You