Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles...
Transcript of Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles...
![Page 1: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/1.jpg)
Be an All-Star Manuscript Reviewer!
Charles E. Kahn, Jr., MD, MS
Editor, Radiology: Artificial Intelligence
![Page 2: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/2.jpg)
Conflict of Interest
• Nothing to disclose
![Page 3: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/3.jpg)
Learning Objectives
• Understand the manuscript review process
• Describe the most important criteria to evaluate a manuscript
• Define special considerations when reviewing work about AI in
radiology
![Page 4: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/4.jpg)
Overview
• Process
▫ Timeliness
• Content
▫ Guidelines + standards
▫ Critical thinking
• Ethics
▫ The Golden Rule
![Page 5: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/5.jpg)
Review Timeline
Journal Office
• Complete manuscript?
• IRB/ethics statement?
Deputy Editor
• Appropriate?• Novel?• Good enough to
review?• Assign reviewers
Reviewers
• Important?• Scientifically
valid?
Deputy Editor
• Integrate reviews
• Recommend
Editor
• Make decision• Accept• Revise• Reject
1-3 days 5 days 14 days 7 days 7 days
Time to First Decision: 31 – 37 days
![Page 6: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/6.jpg)
Time is Key
• Respond promptly!
▫ Quick “No” better than Slow “Yes”
• Provide dates you’re unavailable
• Honor the deadline
![Page 7: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/7.jpg)
Your Review
• Strengths
▫ Timeliness of topic, novelty, size of study
• Weaknesses
▫ Scientific concerns, lack of generalizability
• Specific comments
▫ Page + line numbers
![Page 8: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/8.jpg)
Focus Your Review
• Give constructive comments
▫ Seek to improve the work
• Help the editors
▫ Weaknesses
▫ Scientific quality
▫ Priority
“Would I want to read this article in this journal?”
![Page 9: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/9.jpg)
The Golden Rule of Reviewing
• “Do unto others as you would have them do unto you”
▫ Be constructive
Try to improve the work
▫ Maintain confidentiality
Your “sneak peek” at another’s work is a privilege!
Don’t quote or share the content
![Page 10: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/10.jpg)
The Rewards
• Reviewer recognition
▫ Journal appreciation
▫ Publons
• Editorial Board appointment
• Honor + Glory
![Page 11: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/11.jpg)
![Page 12: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/12.jpg)
Before You Begin…
• Correct manuscript category?
• Any potential conflict or bias?
• Important problem?
• Previously published?
![Page 13: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/13.jpg)
The Review
• Abstract
• Introduction
• Methods
• Results
• Discussion
• Figures
• Tables
• References
• Summary
![Page 14: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/14.jpg)
The Abstract
• Summarizes the manuscript appropriately
▫ Discrepancies between Abstract and rest of manuscript
• Stands alone
▫ Understandable without reading the manuscript
![Page 15: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/15.jpg)
The Introduction
• Concise
• Clearly defines the purpose of the study
• Explains why the study is important
• Defines terms
• Well-defined and testable hypothesis
![Page 16: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/16.jpg)
The Methods Section
• Reproducible methods
• Justified study design
▫ Choice of model
▫ Cohort size
• Methods test the hypothesis
![Page 17: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/17.jpg)
The Results Section
• Clearly explained
• Order of results parallels the methods
• Reasonable? Unexpected?
• Any results not tied to methods?
![Page 18: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/18.jpg)
The Discussion Section
• Summarize results
• Place results in context
▫ Relate to prior literature
▫ Indicate impact on the field
• Describe limitations
• Envision future work
![Page 19: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/19.jpg)
The Discussion Section
• Concise
• Hypothesis verified?
▫ Research question answered?
• Unexpected results
• Limitations
• Next steps
![Page 20: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/20.jpg)
Figures
• Necessary
• Understandable
• Appropriate
![Page 21: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/21.jpg)
![Page 22: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/22.jpg)
Hypothesis
• Quantitative vs.
qualitative
• Retrospective vs.
prospective
• Feasibility vs.
performance
Data
• De-ID
• Data protection
• Ethics
• Preparation
• Augmentation
• Partitions
• Ground truth
Model
• Architecture
• Software
• Initialization
• Pretraining /
transfer learning
• Hyperparameters
• Training rules
Evaluation
• Metrics
• Sensitivity
analysis
• External
testing
• Statistical
analysis
![Page 23: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/23.jpg)
Data
• Where did the data come from?
• How were variables defined?
▫ Common Data Elements (RadElement.org), where applicable
• Inclusion / exclusion criteria
• How was the quantity of data to be used determined?
• How well do the training data match the intended clinical use?
![Page 24: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/24.jpg)
Trouble in Paradise
• Of 516 eligible published studies, only 6% (31 studies)
performed external validation
• None of the 31 studies adopted all three design features:
▫ Diagnostic cohort design
▫ Inclusion of multiple institutions
▫ Prospective data collection for external validation
Kim DW, et al. Korean J Radiol. 2019;20:405-10
![Page 25: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/25.jpg)
doi.org/10.1016/j.jacr.2019.06.009
![Page 26: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/26.jpg)
ACR “TOUCH-AI”
acrdsi.org/DSI-Services/Define-AI/Use-Cases/Acute-Appendicitis
radelement.org/element/RDE195
![Page 27: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/27.jpg)
De-Identification
• Images
▫ DICOM header
Conventional + “private" fields
▫ Image data
Jewelry, implant IDs, burned-in labels,
facial views
• Reports
▫ Patient + provider names
“Hide in Plain Sight” and others
▫ DatesParks CL, Monson KL.
J Digit Imaging 2017; 30:204
![Page 28: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/28.jpg)
“Ground Truth”
• Well defined
• Who (or what) annotated the data?
▫ Qualifications / training
▫ Instructions
▫ How was inter-rater variability
measured?
▫ Was intra-rater variability assessed?
• Single vs. multiple annotation
▫ Blinding
▫ Adjudication of discrepancies
![Page 29: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/29.jpg)
Data Preparation
• “Data wrangling”
▫ Specific software, version number, specified options
• Normalization
• Resampling
▫ Image matrix size
E.g., 512 x 512 224 x 224
▫ Bit depth
E.g., 16-bit grayscale 8-bit RGB
![Page 30: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/30.jpg)
Data Augmentation
• Was it used? If so, how?
▫ Horizontal flip
▫ Vertical flip
▫ Translation (sliding)
▫ Rotation
▫ Affine transformation
![Page 31: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/31.jpg)
Data Partitions
• How?
▫ Any differences? If so, why?
Data source, annotation source, or
preparation
• Disjoint
▫ By image, study, patient, or
institution
▫ Should be at least by patient
Training70%
Tuning (Validation)
20%
Testing10%
![Page 32: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/32.jpg)
Architectures
• LeNet
• ResNet34
• U-Net
• VGG
• LSTM
• Inception
![Page 33: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/33.jpg)
Model Evaluation
• How many models were evaluated against the test set, and how
were these models selected?
▫ Ideally, one
▫ If greater than one, justify reasoning
![Page 34: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/34.jpg)
Metrics
• Sørensen-Dice coefficient = 2 𝑋∩𝑌
𝑋 + 𝑌
▫ Dice similarity coefficient (DSC)
=2𝑇𝑃
2𝑇𝑃 + 𝐹𝑃 + 𝐹𝑁
.
![Page 35: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/35.jpg)
Metrics
• Jaccard index =
Intersection over Union = IoU = |𝑋∩𝑌|
|𝑋∪𝑌|
![Page 36: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/36.jpg)
Metrics
• Hausdorff distance
▫ Measures how far apart two subsets are
The greatest of all the distances from a point
in one set to the closest point in the other set
![Page 37: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/37.jpg)
Model Performance
Handelman GS, et al. AJR 2019; 212:38-43
Overfitting Underfitting
![Page 38: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/38.jpg)
arxiv.org/abs/1807.00431
Irrelevant
Features
![Page 39: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/39.jpg)
Leakage
• Unintended use of known information as unknown
• Outcome leakage
▫ Independent variables can be used to infer outcomes
For example, a risk factor that spans into the future may be used to predict the
future itself
• Validation leakage
▫ Ground truth from training set propagates to validation set
For example, same patient is used in both training and validation
![Page 40: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/40.jpg)
K-fold Cross-validation
• Split the data into K equal parts
▫ Train the model on K−1 parts
▫ Validate on the remaining part
• Repeat the process K times
• Report average results for K-folds
• For small classes and rare categorical factors, stratified K-fold
splitting ensures equal presence of classes and factors in each fold
![Page 41: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/41.jpg)
Evaluation
• ROC analysis
• Calibration curve
• Confusion matrix
• Review of misclassifications
![Page 42: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/42.jpg)
http://arogozhnikov.github.io/2015/10/05/roc-curve.html
![Page 43: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/43.jpg)
Allen B Jr, et al. J Am Coll Radiol 2019; 16:1179
AI Need Not Be Superhuman
![Page 44: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/44.jpg)
Calibration
Actual prevalence of
malignancy versus
estimated risk of
malignancy for each decile
of the probability scale.
Ayer T et al. Cancer 2010; 116:3310-21
![Page 45: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/45.jpg)
Misclassification
https://doi.org/10.1148/ryai.2019180001
False Positives False Negatives
![Page 46: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/46.jpg)
Good Science
• Hypothesis
▫ Well-defined
▫ Testable
▫ Innovative
• Methods
▫ Appropriate to stated problem
▫ Described in detail
▫ Correct metrics
• Results
▫ Appropriate level of detail
• Discussion
▫ Summarize results
▫ Place work into context
▫ Describe limitations
▫ Envision future work
![Page 47: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/47.jpg)
http://jasonya.com/wp/wp-content/uploads/2015/04/car_peer_review_comic_12.jpg
![Page 48: Be an All-Star Manuscript Reviewer! · 2020-01-14 · Be an All-Star Manuscript Reviewer! Charles E. Kahn, Jr., MD, MS Editor, Radiology: Artificial Intelligence. Conflict of Interest](https://reader034.fdocuments.net/reader034/viewer/2022050119/5f4feabd7166a954ec59184b/html5/thumbnails/48.jpg)
http://jasonya.com/wp/wp-content/uploads/2015/04/car_peer_review_comic_12.jpg