ECCV 2020 TUTORIAL ON ACCELERATING COMPUTER VISION … · 2020. 8. 23. · 3 TUTORIAL ORGANIZATION...
Transcript of ECCV 2020 TUTORIAL ON ACCELERATING COMPUTER VISION … · 2020. 8. 23. · 3 TUTORIAL ORGANIZATION...
![Page 1: ECCV 2020 TUTORIAL ON ACCELERATING COMPUTER VISION … · 2020. 8. 23. · 3 TUTORIAL ORGANIZATION •3 sections –Basics and Fundamentals, Code Optimization Tricks, Application](https://reader034.fdocuments.net/reader034/viewer/2022051605/600935432079924f49334000/html5/thumbnails/1.jpg)
Arun Mallya, 23 August 2020
ECCV 2020 TUTORIAL ON ACCELERATING COMPUTER VISION WITH MIXED PRECISION
![Page 2: ECCV 2020 TUTORIAL ON ACCELERATING COMPUTER VISION … · 2020. 8. 23. · 3 TUTORIAL ORGANIZATION •3 sections –Basics and Fundamentals, Code Optimization Tricks, Application](https://reader034.fdocuments.net/reader034/viewer/2022051605/600935432079924f49334000/html5/thumbnails/2.jpg)
2
WHY IS THIS TUTORIAL USEFUL FOR ME?
• New levels of accuracy are being obtained by using larger networks and larger datasets
• Training of models remains a bottleneck - need fast turn-around times while iterating on the design of models
• Reduced / Mixed precision training can help speed up training without loss of accuracy
• Faster arithmetic computation
• Lower memory requirements – larger batch size
• This tutorial teaches you how to use mixed precision training in your experiments
![Page 3: ECCV 2020 TUTORIAL ON ACCELERATING COMPUTER VISION … · 2020. 8. 23. · 3 TUTORIAL ORGANIZATION •3 sections –Basics and Fundamentals, Code Optimization Tricks, Application](https://reader034.fdocuments.net/reader034/viewer/2022051605/600935432079924f49334000/html5/thumbnails/3.jpg)
3
TUTORIAL ORGANIZATION
• 3 sections – Basics and Fundamentals, Code Optimization Tricks, Application Case Studies
• Basics and Fundamentals
• Introduction to NVIDIA Tensor Cores (NVIDIA A100 GPUs and TensorFloat-32)
• Introduction to Mixed Precision training and the AMP library
• Accuracy and Performance Considerations
• Code Optimization Tricks
• Simple techniques to improve training performance in PyTorch
• Application Case Studies
• Examples of application of AMP to research produced at NVIDIA
• GANs, 3D CNNs, LSTM, etc.
![Page 4: ECCV 2020 TUTORIAL ON ACCELERATING COMPUTER VISION … · 2020. 8. 23. · 3 TUTORIAL ORGANIZATION •3 sections –Basics and Fundamentals, Code Optimization Tricks, Application](https://reader034.fdocuments.net/reader034/viewer/2022051605/600935432079924f49334000/html5/thumbnails/4.jpg)