Critical Review of Key Papers in Biomedical Imaging (The Journal Club)

1

Critical Review of Key Papers in Biomedical Imaging (The Journal Club) This booklet describes the Critical Review of Key Papers in Biomedical Imaging (The Journal Club) set up as an assessed module as part of the UCL MRes Doctoral Training Programme in Medical and Biomedical Imaging held jointly by the Centre for Medical Image Computing (CMIC) and the Centre for Advanced Biomedical Imaging (CABI). The course has been run over three academic years by Andrew Melbourne, Tammy Kalber, Ivana Drobnjak, Simon Walker-Samuel and Jack Wells. The aim of the course is threefold:

• To develop skills for paper review and effective presentation, • To introduce students to fundamental papers in the area of medical and

biomedical imaging, • To learn about work carried out by academic staff members in and around

UCL. The DTP Journal Club aims to equip students with the relevant skills to review the literature in their research field and improve

• Presentation skills, • Critical thinking, • Scientific discussion, • Literature review skills, • Scientific reading, writing and reviewing, • their awareness of medical and biomedical imaging at UCL

Students are formally assessed in three areas: their general presentation skills, strength of conclusions and ability to handle questions; their contribution to the class by raising questions and contributing to the discussion; and by an individual written review summarising the paper and reflecting upon contemporary literature. Papers are chosen by invited academic members of staff who chose papers that they consider a key work in their field. In general the chosen papers have been followed by a substantial body of work although the choice is also influenced by modality, authorship, paper style and journal.

2

Contributors Many academic staff in and around UCL have contributed papers to the journal club including: Danny Alexander (CMIC), Behrens 2003, Moseley 1990. Simon Arridge (CMIC), Kaipio 1999. Erik Arstad (Chemistry), Burns 2007, Klunk 2004. David Atkinson (UCH), Pruessman 1999, Pipe 2002. Dean Barratt (CMIC), Sato 1998. Paul Beard (Medical Physics), Zhang 2006. David Becker (Biology) Gaietta 2002. Anna David (UCH), Nicolaides 1992. Xavier Golay (IoN), Ogawa 1990, Frisoni 2010. David Hawkes (CMIC), Rueckert 1999, Maes 1997. Jem Hebden (Medical Physics) White 2010 Brian Hutton (UCH), Townsend 2008. Louis Lemieux (IoN), Schmuel 1992. Mark Lythgoe (CABI), Louie 2002, Lee 2010. Roger Ordidge (Medical Physics), Ordidge 1989. Sebastien Ourselin (CMIC), Van Leemput 1999, Cootes 1995, Ashburner 2000. Will Penny (FIL), Garrido 2007. Shonit Punwani (UCH), Villers 2009. Karin Shmueli (Medical Physics) Duyn 2007 Danail Stoyanov (CMIC) Mirota 2012 David Thomas (IoN), Williams 1992. Nikolaus Weiskopf (FIL), Kwong 1992. Claudia Wheeler-Kingshott (IoN), Song 2002. Gary Zhang (CMIC) Stanisz 1997 (CMIC=Centre for Medical Image Computing, UCH=University College Hospital, IoN=Institute of Neurology, CABI=Centre for Advanced Biomedical Imaging, FIL=Wellcome Trust Centre for Neuroimaging) For more information visit the journal club website: http://cmic.cs.ucl.ac.uk/home/doctoral_training_programme/journal_club/

3

UCL researcher geography Below is map of the location of UCL researchers and departments involved in the DTP Journal Club

4

A graphical guide to the journal club papers Paper authors/dates distributed by modality and application

(DTI=Diffusion Tensor Imaging, FMRI=functional MRI, ASL=Arterial Spin Labelling, DCE=Dynamic Contrast Enhanced MRI, NM=Nuclear medicine, US=Ultrasound, EEG=Electroencephalography, Microscopy includes electron microscopy and photoacoustic imaging)

5

A brief guide to journal paper reading 1) What makes a good paper? There are two fundamental requirements of a good paper: reproducibility and predictivity. A paper should contain sufficient detail that the methods (and thus the results) are reproducible by other authors. A paper should also make predictions that allow its hypothesis to be falsified from results not tested by the authors. Without these two components a paper is useless. However, a paper might contain a useful idea that can be adopted and refined by subsequent authors. In addition emphasis should be placed on writing clarity so that readers of multiple nationality and background may understand the paper. 2) How is a paper structured? A paper is quite often structured in the following format: • Abstract, a good abstract will summarise the entirety of the paper including the key results, thus the reader can quickly determine the contribution of the paper and whether the work is relevant to them. • Introduction, this should contain all the background information that is required to understand the problem being solved and setting the scene for the method that follows. • Methods, this should describe in sufficient detail the technique and any assumptions being made so that the reader can recreate the method entirely. • Results, the results should be achievable from the methods described. superfluous results may detract from the key points being made. The results maybe interpreted and analysed in this section or in a separate discussion section. • Conclusion, This will summarise the paper and allude to the future importance of the work. It should not over-state the results or make unsupported claims. • References, the reference list should contain sufficient background material to understand the problem being solved and to recreate the methods. The reference list often highlights the track record of the problem. 3) The following list is a checklist when reading a journal paper.

i. What are the key points? ii. What are the key limitations? iii. Why is the paper relevant (what is the contribution)? iv. Are the results plausible? v. Are the conclusions supported by the results? vi. Has the paper been useful to other authors?

6

A brief guide to peer review 1) Why do scientists write papers? Writing short papers enables rapid communication of ideas (as opposed to perhaps compiling an entire textbook). Ideas can be disseminated and judged; good ideas are used and bad ideas rejected. Reading a paper quickly and assessing its worth is a key research skill, and this is aided by the short paper format. 2) Why publish in journals? Authors traditionally publish in periodical journals that are widely read by those in their area. In addition, readers will also seek out these journals when looking for recent work. As a result there is an element of reputation associated with a journal. Since journals wish to increase their readership, a process is needed to select the best articles from a large number of submissions; this is the peer review process. 3) What is peer review? Peer review is the process by which work submitted to a journal is sent out to external reviewers working in the same field so that an informed decision can be made by the editor on whether to publish. 4) Why peer review? Fundamentally, the peer review process is a first filter that acts to remove work not suitable for publication; it cannot assess the likely impact a paper will have. Peer review principally filters out poor quality work and to some extent mitigates plagiarism (reducing the time wasted by future potential readers), however, the process assumes honesty by the authors and so is unlikely to catch fraudulent work. The implications of poor peer review can be large, particularly if fraud is involved, but the impact is generally only over a short period of time (e.g. the UK MMR controversy). Fortunately, an implicit peer review continues after publication and consensus is established over the longer term.

The peer review process is highly regarded by most researchers and is often regarded as a minimum requirement for scientific publishing. Research workers are often happy to carry out reviews since they are themselves likely to be authors, thus reviewers often claim they aim to review as they themselves would like to be reviewed. Reviewing also keeps researchers abreast of very recent work and it could be claimed that as an expert in a certain field, researches have a responsibility to ensure subsequent literature is up to standard. The peer review process is often aided by blinding where the authors are not aware of the reviewers (single blinding) or the authors and reviewers are not aware of each other (double blinding). Single blinding is common for journal articles to reduce susceptibility to recrimination and bribery of the reviewers. Double blinding is more common in conferences, in which there is an increased element of competition by author-reviewers.

7

5) What are some of the criticisms of peer review? The peer review process is not without criticism, particularly of its susceptibility to reviewer bias, inability to catch fraudulent work and unintended suppression of useful ideas. With regards to bias, the process relies on a stable state arising as a result of reviewers acting on a review as they themselves would like to be reviewed. Within a sufficiently large research community, author and reviewer interests do not conflict, but for small research communities there may be pressure for authors to conform to established ideas or reject papers that in some way might impact their own funding. The inability to catch fraudulent work is not a fault of the peer review process, but it is an important missing feature of research publication to which there is no obvious answer. Peer review is occasionally criticised for rejecting ideas that are outlandish or far outside the accepted scientific consensus; however, if reviewers adhere to reproducibility & predictivity as opposed to novelty & precedent as review criteria, this bias should be reduced. 6) Consensus is established over time Although poor or fraudulent papers sometimes pass through peer review, this is far from the end of the process. The impact of a paper can only be assessed after publication; if the method or results are useful, the technique will be used, recreated and extended by other authors. Conversely if a technique does not recreate an author’s results, or is found to be flawed, the paper will not be used (although some time will have been wasted). A record of this process is summarised by counting citations from subsequent authors (although this is not the only measure of a papers success). Thus implicitly, the peer review process continues long after publication and consensus is established over time. 7) Different types of citation There are three flavours of citation counting: the most direct is to count the number of citations a paper has (being aware of when the paper was published), although high citations might reflect a good idea rather than a good paper. Alternatively the ‘impact factor’ of the journal in which it is published can be assessed (the average number of citations per paper in that journal). An assessment can also be made of the track record of the author, perhaps by using the so-called H-factor which is the number, n, of papers published by that author that have at least n citations. In practice this is not seriously used (or needed)!. For more information see: http://www.senseaboutscience.org/