CITATION INDICES

CITATION INDICES

Philip Purnell

March 2010

PART IRESEARCH EVALUATION

HOW DO WE EVALUATE RESEARCH?

• Research grants– Number and value

• Prestigious awards – Nobel Prizes

• Patents– Demonstrating innovative research

• Faculty– Number of post-graduate researchers

• Citation analysis– Publication and citation counts

– Normalised by benchmarks

• Peer Evaluation– Expensive, time consuming and subjective

DO WE NEED MORE METRICS? A SIMPLE COUNT IS GOOD ENOUGH (1962)

CHAMPIONS LEAGUE 2009METRICS ARE HERE TO STAY !

5

2 Goals 01 1st Half Goals 07 Shots on Target 24 Shots off Target 71 Blocked Shots 24 Corners 77 Fouls 101 Offsides 51 Yellow Cards 30 Red Cards 087 Passing Success 82.918 Tackles 2172.2 Tackles Success 85.752.7 Possession 47.3

55.3 Territorial Advantage 44.7

Barcelona Team Statistics Manchester United

A BRIEF HISTORY OF THE CITATION INDEX

• Concept first developed by Dr Eugene Garfield – Science, 1955

• The Science Citation Index (1963)– SCI print (1960’s)– On-line with SciSearch in the 1970’s – CD-ROM in the 1980’s– Web interface (1997) Web of Science

• Content enhanced:– Social Sciences Citation Index (SSCI)– Arts & Humanities Citation Index (AHCI)

• The Citation Index – Primarily developed for purposes of information retrieval– Development of electronic media and powerful searching tools

have increased its use and popularity for purposes of Research Evaluation

WEB OF SCIENCE JOURNAL SELECTION POLICY

• Why do we select journals?

THOMSON REUTERSJOURNAL CITATION REPORTS

0

20

40

60

80

100

120

0 1000 2000 3000 4000 5000 6000

# of journals

% o

f dat

abas

e

Articles Citations

40% of the journals:

• 80% of the publications

• 92% of cited papers

4% of the journals:

• 30% of the publications

• 51% of cited papers

COMPARE LIKE WITH LIKEWHAT IS THE VALUE OF A CITATION?• Why do people cite?

– Pay homage / give credit to pioneer

– Identifying a methodology

– Provide background reading

– Quotations

– Authenticating data, reproducing work etc

– Corrections

– Criticizing/Disclaiming someone's work/opinions

• Citations are an indicator of an article’s impact and usefulness to the research community; they are the mode by which peers acknowledge each other’s research.

• The value of a citation is only as important as its source. – Clearly a citation from a prestigious peer review journal has more value than a

citation from non-scholarly material.

– How can you be sure that the citing source is reputable?

“When to Cite”, E. Garfield, Library Quarterly, v66, p449-458, 1996

WEB OF SCIENCE JOURNAL SELECTION POLICY

• Approx. 2000 journals evaluated annually

– 10-12% accepted

• Thomson Reuters editors– Information professionals

– Librarians

– Experts in the literature of their subject area

Web of Science

Journals under evaluation

Journal ‘quality’

THOMSON REUTERSJOURNAL SELECTION POLICY

• Publishing Standards– Peer review, Editorial conventions

• Editorial content– Addition to knowledge in specific subject field

• Diversity– International, regional influence of authors, editors, advisors

• Citation analysis– Editors and authors’ prior work

Regional Journal Content Expansion 2007-2009Contribution of Each Region to Subject Areas

Region A&H AB&ES CM EC&T LS PC&ES S&BS TotalAP 13 55 105 45 47 41 89 395EU 150 88 161 81 50 116 200 846LA 29 55 33 13 14 15 42 201MA 13 23 32 10 15 7 31 131NA 8 1 13 2 1 0 3 28

Total 213 222 344 151 127 179 365 1601

Regions:AP : Asia PacificEU : European UnionLA : Latin AmericaMA : Middle East/AfricaNA : North America

Subject Areas:A&H : Arts &HumanitiesAB&ES : AgBio & Environmental SciCM : Clinical MedicineEC&T : Engineering Computing & TechnologyLS : Life SciencePC&ES : PhysChem & Earth ScienceS&BS : Social & Behavioral Science

12

GROWTH IN COVERAGE OF REGIONAL AND CZECH JOURNALS

Web of Science

2005 2010 Growth Variation

Total Journals Indexed

8,834 11,423 2,589 + 29%

Regional Journals Indexed

1,716 3,605 1,889 + 110%

Czech Journals Indexed

30 56 26 + 87%

13

GROWTH IN COVERAGE OF CZECH JOURNALS IN WEB OF SCIENCE

14

Web of Science

2005 2010 Growth Variation

Science Citation Index – Expanded

23 36 13 + 57%

Social Sciences Citation Index

4 11* 7 + 175%

Arts & Humanities Citation Index

3 12** 9 + 300%

*1 title is covered in both SCIE and SSCI**2 titles are covered in both SSCI and A&HCI

Region # Journals from Region in Web of Science

Europe 5,573 49%

North America 4,251 38%

Asia-Pacific 965 9%

Latin America 272 2%

Middle East/Africa 200 1%

Language # Journals in Web of Science

English 9114 81%

Other 2147 19%

GLOBAL RESEARCH REPRESENTATIONWEB OF SCIENCE COVERAGE

GOVERNMENTS AND INSTITUTIONS USING TR DATA FOR EVALUATION (INCL.)

• Czech Republic: Czech Academy of Sciences; Government

• France: Min. de la Recherche, OST - Paris, CNRS

• Germany: Max Planck Society, several gov’t labs, DKFZ, MDCUS: National Institutes of Health

• United Kingdom: King’s College London; HEFCE

• European Union: EC’s DGXII(Research Directorate)

• US: NSF: biennial Science & Engineering Indicators report (since 1974)

• Canada: NSERC, FRSQ (Quebec), Alberta Research Council

• Australian Academy of Science, gov’t lab CSIRO

• Japan: Ministry of Education, Ministry of Economy, Trade & Industry

• People’s Republic of China: Chinese Academy of Science

16

EVALUATING COUNTRIES

CHANGING NATURE OF CZECH RESEARCH

18

Czech researchers averaged just over 4,000 papers a year in the 70s – 90s.Now over 10,000 papers per year

CHANGING NATURE OF CZECH RESEARCHINTERNATIONAL COLLABORATION

19

Between 1970 and 1999, 14% of Czech papers had at least one international co-author

CHANGING NATURE OF CZECH RESEARCHINTERNATIONAL COLLABORATION

20

In the last decade 2000 – 2010, this figure has risen to 63%

SCIENTIFIC RESEARCH PRODUCTIVITY IN CENTRAL EUROPE

21Thomson Reuters InCites

SCIENTIFIC RESEARCH IMPACT IN CENTRAL EUROPE


CZECH RESEARCH BY OUTPUT AND IMPACT

23

Ranking by percentage share of Thomson Reuters (ISI) indexed papers 2004 - 2008

EVALUATING INSTITUTIONS (1)

EXTERNAL COMPARISONS

EVALUATING INSTITUTIONS

Source: Thomson Reuters

North America University Science Indicators

Number of citations to North American scientific papers

PUBLICATION AND CITATIONS TOP CZECH ORGANISATIONS

26

COMPARE YOUR INSTITUTION’S PERFORMANCE AGAINST GLOBAL PEERS


INSTITUTIONAL COMPARISONSCOMPARING IMPACT IN BIOLOGY?

29

Cambridge

Oxford

Stirling

Thomson Reuters InCites

INSTITUTIONAL COMPARISONSIN SPECIFIC FIELDS


INSTITUTIONAL COMPARISONSBIOLOGY PERFORMANCE


EVALUATING INSTITUTIONS (2)

INTERNAL ANALYSIS

WHAT IS OUR RESEARCH OUTPUT?

33

BENCHMARKING YOUR PAPERS AGAINST GLOBAL AVERAGES

34

Articles published in ‘Blood’ from 2004 have been cited 34.30 times

Hematology articles from this year have been cited 18.83 times

This article is in the 12.92nd percentile in its field by citations

WHAT IS OUR RESEARCH OUTPUT?

35

WHICH ARE OUR CENTERS OF EXCELLENCE?

36

Physical Chemistry

Computer Science

41% below average

220% above average


WITH WHOM DOES OUR FACULTY COLLABORATE?

37

WHICH OF THOSE COLLABORATIONS ARE THE MOST VALUABLE?

38

INCITESCITING ARTICLES LISTING


CITING PAPERSIN THE FIELD OF PHYSICS


EVALUATING INDIVIDUALS

FRANTISEK VYSKOCILPROF. AND D.Sc CHARLES UNIV, PRAGUE

• Described: Potassium movement in the brain during migraine

• Member: Physiological Society of Cambridge and London

• Founding Member: Learned Society of the Czech Republic

• Awards: – Prize of the Czech Academy of Sciences

– Lifelong Contribution Award of the Czech Academy of Sciences

– Purkynje Medal of the Czech Academy of Sciences

• First violinist: Quartet of the Herold Chamber Music Club

• Author: >60 popular science articles in "Vesmír“

42

FRANTISEK VYSKOCILWEB OF SCIENCE PUBLICATIONS

43

247 Publications

FRANTISEK VYSKOCIL GLOBAL INFLUENCE

Citations: 2.375Cites per paper: 9,73

H-Index = 30

Citations by country

Citations by journal

WHO ARE OUR MOST PRODUCTIVE AUTHORS?


WHO ARE OURMOST INFLUENTIAL RESEARCHERS?


WHICH AUTHORS HAVE MOST IMPACT?


WHICH AUTHORS’ PAPERS HAVE PERFORMED BEST IN THEIR FIELD?


HOW CAN WE COMPARE RESEARCHERS?

49

Author A: 60 papers Author B: 117 papers


EVALUATING JOURNALS

EFFICIENCYJOURNAL IMPACT FACTOR

2008 Impact Factor

200820072006

Source paper – published in 2008

Cited reference – published in 2006 or 2007

Citations

All Previous Years

2005 2009

CALCULATING 2008 IMPACT FACTOR COLLECTION OF CZECHOSLOVAK CHEMICAL COMMUNICATIONS

Citations in 2008To items published in 2007 = 78

To items published in 2006 = 100

Sum = 178

Number of itemsPublished in 2007 = 118

Published in 2006 = 109

Sum = 227

178

227

= 0,784

JOURNAL IMPACT FACTORCZECH JOURNALS

53

CITATION BEHAVIOUR VARIES BETWEEN SUBJECT CATEGORIES

JOURNAL 5-YEAR IMPACT FACTORCZECH JOURNALS

55

HOW DO CZECH INSTITUTIONS EVALUATE JOURNALS FOR LIBRARIES?

• Faculty head request

• Publisher packages

• Budget constraints

• Library recommendation

56

IN WHICH JOURNALS DO OUR BIOLOGISTS PUBLISH?


AND THE GLOBAL COMMUNITY INFLUENCED BY YOUR RESEARCH?


SUMMARY:THE DATA SOURCE• Thomson Reuters solutions utilize the data from the Web of Science

to provide reliable and consistent evaluation tools

• It is essential to have a consistent, authoritative and clearly defined body of data to create meaningful and dependable statistics. This is why we use Web of Science:– The gold standard citation resource, used by over 4,000 institutions in

more than 90 countries

– A comprehensive resource covering over 11,000 journals and thousands of conference proceedings in the fields of Science, Social Science, Arts and Humanities

– Unmatched retrospective depth of citation data (up to 110 years!)

– Selected content using unbiased, time tested and stringent journal selection process

– Web of Science is also the source of the Impact Factor the most widely accepted indicator of journal performance

PART IITHE DATA

THE DATA

• Web of Science

– Conference Proceedings Citation Index – Century of Social Sciences

• ISI Web of Knowledge - The Citation Universe– Web of Knowledge 5 upgrade

– Biosis Citation Index

– Chinese Science Citation Database

• Research Analytics

61

THE DATA

• Web of Science






62

CPCI – ON WEB OF SCIENCE

63

Search by - Conference title - Location - Date - Sponsor

• Two Editions:- Science & Technology - Social Sciences & Humanities

• Approx 5.5 million records• Updated Weekly• Over 385,000 records per year • More than 120,000 conferences covered in total• Approximately 12,000 conferences covered annually• Coverage from 1990, cited references from 1999• Comprehensive, multidisciplinary and international coverage from

conferences proceedings published in books, journals, reports, series, and preprints

• Details of each conference are indexed and searchable, data includes: conference title, sponsor, location, description and date

WEB OF SCIENCECONFERENCE PROCEEDINGS CITATION INDEX

A combined set of results from both Conference Proceedings and Journal literature.

Refine options are a useful tool to narrow a search. For example narrow by subject classifications or document type to focus on the most relevant materials

This Citation Report is based on the entire publication portfolio of 2008 Nobel laureate Roger Y. Tsien. Many records are from Conference Proceedings, without this content the portfolio is incomplete.

Full Web of Science functionality with Conference Proceedings content

Details of both conference and source publication

Classified by Document Type

Source item is assigned to standard subject classifications

THE DATA

• Web of Science






69

WEB OF SCIENCEDEPTH OF CITATION INDICES

Science Citation Index

SSCI

Century of Science1900

1956

A&HCI1975

1945

Century of Social Sciences1900

CPCI199056 years of added coverage in the Social Sciences

>300 journals added that focus on the first half of last century

CENTURY OF SOCIAL SCIENCESBREAKDOWN BY FIELD

CoSS distribution by field Title CountANTHROPOLOGY 24

COMMUNICATION 18

ECONOMICS & MANAGEMENT 47

EDUCATION 31

GEOGRAPHY 12

HISTORY & PHILOSOPHY OF THE SOCIAL SCIENCES 1

LAW 30

POLITICAL SCIENCE 10

PSYCHIATRY 21

PSYCHOLOGY 65

PUBLIC HEALTH 20

SOCIAL ISSUES & SOCIOLOGY 29

Total 308

COVERAGE IN THE FIELD OF GEOGRAPHY

• There are more than 40,000 records in the field of Geography between the years of 1900 and 1955

• The key journals from this period have been evaluated, sourced and indexed to provide a unique resource:

– Geographical Journal– Scottish Geographical Magazine– Geography– Geographical Review– Annales de Geographie

• The works of many ground breaking researchers from around the world have been included, such as:

– C. Warren Thornthwaite

– Harlan H. Barrows

– John R. Borchert

– Petermanns Geographische Mitteilungen– Economic Geography– Annals of the Association of

American Geographers

CASE STUDY:Charles Warren Thornthwaite

• C. Warren Thornthwaite was a Geographerand Climatologist best known for devising a Climate Classification System proposed in his landmark paper in 1948 which also discusses the importance of evapotranspiration on climate.

• Thornthwaite published 28 papers between 1931 and 1965, mostly in the fields of Geography and Geochemistry & Geophysics.

• His works have gone on to be highly influential to researchers in many fields and in recent years have been particularly utilized by researchers involved in climate change and water management

The works of C. Warren Thornthwaite

Thornthwaite produced 28 highly influential works in the Social Sciences which have been consistently cited throughout their history

THE DATA

• Web of Science






76

ISI WEB OF KNOWLEDGE 5 webofknowledge.com

77

ISI WEB OF KNOWLEDGE 5 CITATION UNIVERSE

78

ELIMINATION OF STOPWORDS

79

LEMMATIZATION

80

run, runs, ran and running are forms of the same lexeme, with run as the lemma.

FULL RESULT COUNTSNO MAXIMUM ON ‘ANALYZE RESULTS’

81

LEFT-HAND TRUNCATION

82

All users are accustomed to using right-hand truncation in their searches

Now with left-hand truncation, searches can be even more accurately defined

LEFT-HAND TRUNCATION

83

ABSTRACT PREVIEW

84

ABSTRACT PREVIEW

85

CITATION COUNTS USING WEB OF KNOWLEDGE VALUES

86

WEB OF KNOWLEDGECITATION BREAKDOWN TABLE

8787

MARKED LIST ENHANCEMENTS- ANALYZE TOOL & CITATION REPORTS

88

Analyze tool and citation report avaiable for specific products. Future release will include these features for the All Databases results

CITED REFERENCE SEARCHINGNOW FOR ALL CITATION INDICES

89

New cited reference search parameters facilitate a more accurate search

CITATION COUNTS IN FULL

90

THE DATA

• Web of Science






91

BIOSIS CITATION INDEXBIOSIS CONTENT WITH CITATION DATA

• Biosis Previews– Widest collection of life sciences fields in the world

– agriculture, biodiversity, biotechnology, botany, drug discovery, gene therapy, marine biology, wildlife conservation, zoology

• Biosis Citation Index – Provides cited references for BIOSIS Previews

• Cited references for all Previews content (beginning 2006)

• Cited references for 60% of high impact Previews journals (1926 - 2005)

• Exclusive to ISI Web of Knowledge platform

BIOSIS CITATION INDEXON ISI WEB OF KNOWLEDGE 5

93

BIOSIS CITATION INDEXSEARCHING

94

Set your search parameter defaultsSelect years, use lemmatization and sort by publication date

BIOSIS CITATION INDEXREFINE RESULTS

95

Refine results or exclude results from your dataset via a series of parameters

Identify the most abundant sources of relevant results or concentrate on harder-to-find document types

BIOSIS CITATION INDEXSORTING AND EXPORTING RESULTS

96

Sort your results to find breaking new research papers or to identify the most influential or relevant papers in your fieldSelect the records of your

choice and print, email or export them via your favourite bibliographic management system

BIOSIS CITATION INDEXMARKED LIST

97

BIOSIS CITATION INDEX CITATION BREAKDOWN TABLE

98

BIOSIS CITATION INDEXCITATION REPORT

99

BIOSIS CITATION INDEXANALYZE RESULTS

100

Analyze by Super Taxa

BIOSIS CITATION INDEXCITATION MAP

101

2nd generation citations reveal citation pathways

Citation maps can find relevant papers not returned in your search results

BIOSIS CITATION INDEXCITED REFERENCE SEARCHING

102

Cited reference searching helps users to find ‘missing citations’

BIOSIS CITATION INDEXSPECIALIZED INDEXING

103

Searching Biosis Citation Index for ‘Fossil* and Africa and humans’ returns this paper

The same search in ‘All Databases’ reveals that this record is also present in Web of Science

But the same search in Web of Science does not find the article

It is the Biosis specialized indexing fields that has found these terms that are not indexed in Web of Science


• All BIOSIS records are indexed by biology specialists using the following unique fields

Major Concepts Concept Code(s)

Taxonomic Data Disease Data

Chemical Data Gene Name Data

Sequence Data Geographic Data

Geologic Time Data Methods and Equipment Data

Parts & Structures Data Miscellaneous Descriptors

104


105

All the specialized indexing fields are searchable

The concept code 00506 codes for Explorations and Expeditions

This paper is returned because the concept code is found on the BIOSIS record

BIOSIS CITATION INDEXSUMMARY

• Combines – World-leading Biosis life sciences content

– with valuable cited references back to 1926

• Enhances – Web of Science searches via the All Database Search

using common indexing backbone

• Employs – ISI Web of Knowledge cited reference searching

• Exclusive – ISI Web of Knowledge

106

THE DATA

• Web of Science






107

EMERGENCE OF CHINESE RESEARCH

108

CHINESE SCIENCE CITATION DATABASEAVAILABLE NOW!

• Over 2 million records

• Over 13 million cited references

• Subject Areas– Mathematics, Physics, Chemistry, Geosciences, Biology,

Agricultural Sciences, Engineering (Computer Science, Mechanics, etc.), Medicine, and Management

• Dual Language– Since 2002, 80% of records have full English bibliographic

information and most have English abstracts

• Content is searchable in both English and Chinese

CSCDCITATION BREAKDOWN TABLE

111

Bibliographic information in English and Chinese

THE DATA

• Web of Science






112

AUTHOR NAMES

113

AUTHOR NAMES

114

RESEARCHERIDSCHOLARLY RESEARCH COMMUNITY

• Accurate Identification

• Organize and Manage

• Increase Visibility and Recognition

• Measure Performance

• Collaboration

• Security

Science, March 2009

INDIVIDUAL LEVELRESEARCH EVALUATION

View accurate publication list due to unique author identification

See personalized metrics using Web of Science citation data

RESEARCHER IDANALYZE COLLABORATION NETWORK

Seek global collaboration opportunities by author, field, institution or country

RESEARCHER IDVISUALIZE CITING ARTICLES NETWORK

RESEARCHER IDELECTRONIC CV RESOURCE

ResearcherID Profile: http://www.researcherid.com/rid/A-9180-2008

Top Five CountriesTop Five InstitutionsTop Ten Institutions

University College LondonThe University of Queensland

Monash UniversityHarvard University

University of MichiganUniversity of Pennsylvania

ETH ZurichUniversity of Cambridge

Stanford UniversityMcGill University

RESEARCHER IDGLOBAL PARTICIPATION

RESEARCHERID UPLOAD SERVICE

• Institutions can upload content on behalf of their researchers– Upload researcher names Obtain a ResearcherID

account

– Upload individual publication portfolios Articles are matched to Web of Science records and ResearcherID portfolios are generated

RESEARCHERIDDOWNLOAD SERVICE

• Download data about the individuals at your institution– Names and name variants, current and past affiliations

• Download ResearcherID publication portfolios– Bibliographic details of each item in the portfolio

– For articles that were successfully matched to Web of Science records there will be a Times Cited count and UT tag.

• Can be used by the customers for their own internal systems

CITATION INDICES

Documents

Transcript of CITATION INDICES