How to find and reward peer reviewers · 2020-03-02 · How to find and reward peer reviewers Alex...

Information Classification: General

How to find and reward peer reviewers

Alex LazzariTaylor & Francis

Alan ReidEnvironmental Education Research

Carolyn StevensJapanese Studies


How to find and reward peer reviewers

Alex Lazzari

Head of HSS International Portfolio,

Routledge Taylor & Francis


Finding the right reviewer

• Significant task for editors

• 2016-2018: average 29% of invitations led to a completed review

• Variable across subjects

Reviewer searches

Journal database

Previous authors

Reference list

PubMed

WoS

Google Scholar

Matching tools


Matching tools to support editors

JANE:

• PubMed data

• http://jane.biosemantics.org/

Reviewer Connect:

• Publons and Web of Science

• Pilot completed on six journals

http://jane.biosemantics.org/


Key features of Reviewer Connect

• Click to add text

Additional information about publications, ORCiD and possible COI


Trial of Reviewer ConnectWhat the team select for• Subject expertise

• 20-150 publications

• Not at the same institution as any of the authors or publishing with them

• Not at the same institution as another invited reviewer

• Primary affiliation not a pharmaceutical company

• Not currently reviewing

Trial outcomes

• 27% of reviewers checked invited by the team

• Varied extensively by article/ subject area (0-70% suitable)

• Extra information displayed helped use list efficiently

• Email information reliable


wizdom.ai – potential future reviewer finder tool

• Currently in pilot and testing phase

• Generates a list of reviewers which the user can filter

• Displays additional visual information about a potential reviewer in addition to the option to view their full publication record


Engaging reviewers from more diverse locations

Expanding and increasing the diversity of the reviewer pool is essential

Data from 2016-2018 of reviewers for whom a country was recorded:

% of overall reviewers

% acceptance of invitations to review

USA 46% 31%

China 4% 54%

India 3% 53%

Australia 3% 48%

New Zealand <1% 52%


Excellence in Peer Review: Taylor & Francis Reviewer Training Network

In-person workshops

• How to be a peer reviewer:

• Ethics

• Responsibilities

• How to assess different articles

• How to write a report

• Tailored to subject area

• Case study discussion

Journal experience

• Participants from in-person workshops with enough experience

• Connected to relevant journals where editors interested

• Feedback on 2-3 reports managed by Taylor & Francis team

Online resources

• Adding to current online resources

• Webinars

• Additional resources in Chinese


Training launched in 2019: China and India

• Wuhan 30th July 2019 – in collaboration with Institute of Science and Technology in China

• Beijing 2nd August 2019 – in collaboration with Chinese Academy of Social Science Evaluation Studies

• Delhi 9th September 2019 – in collaboration with Maulana Azad Medical College

• Jinhua 22nd September 2019 – in collaboration with the Asian Journal of Sport History & Culture


What rewards do reviewers want?

Survey by Wiley in 2015 (Warne 2016: Learned Publishing 29: 41–50)

Incentives to review:

• Feedback on quality of review

• Certificate

• Published acknowledgement in the journal

Financial rewards ranked poorly, with the exception of personal access to

content


Rewarding reviewers at Taylor & Francis

• Publons – integration on all journals in Scholar One

• Published acknowledgements

• 30 days free access to online content

• Discount on books

• Reviewer certificates


Thank you

Alex Lazzari

[email protected]

@TandF_Australia

mailto:[email protected]


Finding and rewarding peer reviewers

A few reflections on best practice, tips and advice

(NB “The essence of strategy is choosing what not to do” Michael Porter)

Alan ReidEnvironmental Education Research


Examples of our goals – and yours?

• Immediate – ensure 30-45 day turnaround for decision letters

• Mid term – renew refereeing pool as the journal / field shifts

• Longer term – identify likely candidates for effective board membership, particularly in relation to quality refereeing


The basics?

Strategy is doing the right things.

Tactics is doing things right.

i.e. the strategy provides a framework for action while the tactics define those actions

Image: SBI


Key questions for this journal

1. How is the editorial board

structured, and does this change?

2. What is stated about workload

types and expectations, availability

rhythms, and quality of refereeing?

3. How is the referee pool generated,

renewed, and filtered?

1. How is the editorial board structured, and does this change?

2. What is stated about workload types and expectations, availability rhythms, and quality of refereeing?

3. How is the referee pool generated, renewed, and filtered?


Typical tactics

1. Pinches of salt: for any author recommendations, referee locator, ORCID entry – unless checked?

2. Citations: which merit leads?

3. Public profile and networking at internationalised events:who’s hungry, humble and helpful?

Image: Bob Pautke


Possible strategies

1. Editorials - e.g. showcasing grad work & thesis summaries

2. Editorial office - e.g. learning from alerts from databases, other publications, etc.

3. Ensuring meaningful rewards – e.g. in acknowledgement, related contributions / events, board membership?


Key Questions Challenges

1. How is the editorial board structured, and does this ^need to^ change?

2. What is ^should be^ stated about workload expectation, availability rhythms, and quality of refereeing?

3. How ^When^ is the referee pool generated, renewed, and filtered?


Thank you!

Social media @

eerjournal


Reviewing Japanese Studies, 1998 ~

Carolyn S. StevensEditor-in-Chief, Japanese Studies

Professor of Japanese Studies

School of Languages, Literatures,

Cultures and Linguistics

Monash University


• It’s ok to desk reject!

• Dearth of available reviewers/lack of readership interest in the topic/ poor m/s

• Go for ‘big’ names when appropriate, but no obvious rejects.

• Review requests are also advertising for the journal.

• Vary reviewers’ attributes (junior vs senior; local vs overseas; disciplinary vs area specialists; same institutions).

• The review is a training process for ECRs

• For interdisciplinary or area studies journals: senior disciplinary specialists can and often do give very good reviews, and it helps to mainstream journal readership in other disciplines.

• Use contacts on the Editorial Advisory Board regularly - this is part of their job!

• Nominate AEs with international networks to enlarge the journal’s pool of reviewers

JS Best Practice*

* Drawn from AE Guidelines, created by A/Prof Judith Snodgrass (first T&F EiC) and myself in 2011, updated regularly and passed to new AEs


To Keep in Mind• No PhD supervisors (or former supervisors) known friends/collaborators,

and colleagues in the same university or school… this can be tricky, though.

• Keep an eye on reviewer/submission histories

• One AE rejects a submission while another AE asks that same author for a review of another m/s

• Tread carefully with special issues as guest editors have vested interests

• Don’t disregard the importance of a timely response.

• Short and sweet might be much more appreciated than a lengthy review later

• Authors need the opportunity to resubmit elsewhere

• Rewarding Reviewers

• Building a relationship with reviewer as future author


What if….?

• I really can’t think of anyone…..?

• Troll the m/s reference list for ideas. Reviewers like seeing their work cited!

• The reviewer generator option on ScholarOne can occasionally help

• I got skunked! (reviewer accepts but never submits)

• We have a list…..

• An original reviewer refuses to read a revised manuscript?

• AE/ EiC can certainly make the call in a minor revision. Send to third reviewer with caution, however

• Looks like a great paper, but I’ve asked 7 people to read it and they’ve all said no!

• Ok to desk reject with a fit for purpose rejection letter


Reviewing Episodes

• Author issues

• Unsolicited book reviews

• Submissions from outside of academia

• Editorial issues (more common)

• Special Issue Headaches!

• Limited pool of experts: suggest one ‘global reader’, one separate reader

• EiC vs AE recommendations: when they don’t agree

• Internationalising reviewers

• Bad reviewers: do they make good authors?

How to find and reward peer reviewers · 2020-03-02 · How to find and reward peer reviewers Alex...

Documents

Transcript of How to find and reward peer reviewers · 2020-03-02 · How to find and reward peer reviewers Alex...