Crowdsourcing for medical image analysis · 2018-05-31 · Crowdsourcing for medical image...
34
Crowdsourcing for medical image analysis Veronika Cheplygina @vcheplygina http://www.veronikach.com
Transcript of Crowdsourcing for medical image analysis · 2018-05-31 · Crowdsourcing for medical image...
Crowdsourcing for medical image analysis
Veronika Cheplygina @vcheplygina
http://www.veronikach.com
Case courtesy of A.Prof Frank Gaillard, Radiopaedia.org, rID: 8095
Does this person have COPD?
Where is the emphysema?
Case courtesy of Radswiki, Radiopaedia.org, rID: 11384
Case courtesy of Radswiki, Radiopaedia.org, rID: 11384
Learning curve
Size labeled data
Performance
Learning curve
Labeled data
Performance>10M images
<100 - 5K
CheXNet: 100K
Learn from other data/labels
Learn from other data/labels
https://arxiv.org/abs/1804.06353
Learn from other tasks
Case courtesy of A.Prof Frank Gaillard, Radiopaedia.org, rID: 51158
Which datasets to use? (Most similar? Most different?)
Case courtesy of A.Prof Frank Gaillard, Radiopaedia.org, rID: 51158
Crowdsourcing
You do it all the time!
13Von Ahn, L., & Dabbish, L. (2004).
2004 - The ESP Game
14Von Ahn, L., & Dabbish, L. (2004).
2004 - The ESP Game
Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009, June). Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on (pp. 248-255). IEEE.
2009: ImageNet
https://qz.com/1034972/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world/
20
Crowd: easy taskExpert: difficult task
Case courtesy of A.Prof Frank Gaillard, Radiopaedia.org, rID: 8095
How large are the airways?
Cheplygina, V., Perez-Rovira, A., Kuo, W., Tiddens, H. A., & de Bruijne, M. (2016). Early Experiences with Crowdsourcing Airway Annotations in Chest CT. In Large-Scale Annotation of Biomedical Data and Expert Label Synthesis (MICCAI LABELS), pp. 209-218
Work by Dylan Dophemont
https://challenge.kitware.com/#challenge/n/ISIC_2017%3A_Skin_Lesion_Analysis_Towards_Melanoma_Detection
Melanoma classification
26
Human experts: “ABCDE”
• A – Asymmetry
• B - Border
• C – Color
• D – Diameter
• E - Evolving
http://www.skincancer.org/skin-cancer-information/melanoma/melanoma-warning-signs-and-images/do-you-know-your-abcdes#panel1-2
Image analysis project for 1st year students
1. Measure features with algorithms
2. Measure features yourself
3. Evaluate
Principal component analysis of crowd features• Blue = healthy, red = melanoma
Next
• How do different annotators compare?• Is disagreement informative?• How do different features (asymmetry, etc) compare?• …
• Train network to predict crowd features (easy), then melanoma (difficult)
100 images
• ID• Feature_Group_Annotator• Group 7
– 5 features– 6 annotators
• Keratosis, Melanoma, Age, Sex
• Dropbox with images: – https://goo.gl/UPhMpy
@vcheplygina
http://www.veronikach.com
• Need more labeled data• Crowdsourcing successful, but challenging for medical• Transfer learning with easy task?
Thanks to:
IMAG/e, Eindhoven University of TechnologyBIGR, Erasmus MC RotterdamPRLab, Delft University of Technology
?
https://pixabay.com/en/psychics-crystal-ball-fortune-teller-1026092/
https://pixabay.com/en/anatomy-bone-bones-check-doctor-3003099/
34
