Opening Keynote at GTC 2015: Leaps in Visual Computing
-
Upload
nvidia -
Category
Technology
-
view
2.556 -
download
0
Transcript of Opening Keynote at GTC 2015: Leaps in Visual Computing
![Page 1: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/1.jpg)
LEAPS IN VISUAL COMPUTINGJEN-HSUN HUANG, CO-FOUNDER & CEO | GTC 2015
![Page 2: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/2.jpg)
FOUR ANNOUNCEMENTS
A New GPUand
Deep Learning
A Very Fast Boxand
Deep Learning
Roadmap Revealand
Deep Learning
Self-Driving Carsand
Deep Learning
![Page 3: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/3.jpg)
AMAZING YEAR IN VISUAL COMPUTING
© 2015 Industrial Light & Magic. All Rights Reserved.
![Page 4: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/4.jpg)
10X GROWTH IN GPU COMPUTING2008
150,000CUDA Downloads
4,000Academic Papers
60Universities Teaching
77Supercomputing Teraflops
6,000Tesla GPUs
27CUDA Apps
![Page 5: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/5.jpg)
2008
150,000CUDA Downloads
4,000Academic Papers
60Universities Teaching
77Supercomputing Teraflops
6,000Tesla GPUs
27CUDA Apps
2015
3 MillionCUDA Downloads
10X GROWTH IN GPU COMPUTING
![Page 6: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/6.jpg)
2015
3 MillionCUDA Downloads
10X GROWTH IN GPU COMPUTING
319CUDA Apps
2008
150,000CUDA Downloads
4,000Academic Papers
60Universities Teaching
77Supercomputing Teraflops
6,000Tesla GPUs
27CUDA Apps
![Page 7: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/7.jpg)
2015
3 MillionCUDA Downloads
800Universities Teaching
10X GROWTH IN GPU COMPUTING
319CUDA Apps
2008
150,000CUDA Downloads
4,000Academic Papers
60Universities Teaching
77Supercomputing Teraflops
6,000Tesla GPUs
27CUDA Apps
![Page 8: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/8.jpg)
2015
3 MillionCUDA Downloads
800Universities Teaching
10X GROWTH IN GPU COMPUTING
319CUDA Apps
2008
150,000CUDA Downloads
4,000Academic Papers
60Universities Teaching
77Supercomputing Teraflops
6,000Tesla GPUs
27CUDA Apps
60,000 Academic Papers
![Page 9: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/9.jpg)
2015
3 MillionCUDA Downloads
800Universities Teaching
10X GROWTH IN GPU COMPUTING
319CUDA Apps
2008
150,000CUDA Downloads
4,000Academic Papers
60Universities Teaching
77Supercomputing Teraflops
6,000Tesla GPUs
27CUDA Apps
60,000 Academic Papers
450,000Tesla GPUs
![Page 10: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/10.jpg)
2015
3 MillionCUDA Downloads
60,000 Academic Papers
800Universities Teaching
54,000Supercomputing Teraflops
10X GROWTH IN GPU COMPUTING
450,000Tesla GPUs
319CUDA Apps
2008
150,000CUDA Downloads
4,000Academic Papers
60Universities Teaching
77Supercomputing Teraflops
6,000Tesla GPUs
27CUDA Apps
![Page 11: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/11.jpg)
![Page 12: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/12.jpg)
8 Billion Transistors 3,072 CUDA Cores7 TFLOPS SP / 0.2 TFLOPS DP12GB Memory
TITAN XTHE WORLD’S FASTEST GPU
![Page 13: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/13.jpg)
![Page 14: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/14.jpg)
01234567
TITAN X FOR DEEP LEARNINGTraining AlexNet
Days
16-core Xeon CPU TITAN TITAN BlackcuDNN
TITAN XcuDNN
~
43
…
![Page 15: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/15.jpg)
8 Billion Transistors 3,072 CUDA Cores7 TFLOPS SP / 0.2 TFLOPS DP12GB Memory
TITAN XTHE WORLD’S FASTEST GPU
$999
![Page 16: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/16.jpg)
FOUR ANNOUNCEMENTS
A New GPUand
Deep Learning
A Very Fast Boxand
Deep Learning
Roadmap Revealand
Deep Learning
Self-Driving Carsand
Deep Learning
![Page 17: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/17.jpg)
A SHORT HISTORY OF DEEP LEARNING
Convolutional Neural Networks for Handwritten Digital Recognition
LECUN, BOTTOU, BENGIO, HAFFNER, 1998ImageNet Classification with NVIDIA GPUsKRIZHEVSKY, HINTON, ET AL., 2012
1995 2000 2005 2010 2015
Accuracy %
2010 201420122011 2013
74%
84%
DNN
CV
72%
![Page 18: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/18.jpg)
“Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification”
— Microsoft: 4.94%, Feb. 6, 2015
“Deep Image: Scaling up Image Recognition”— Baidu: 5.98%, Jan. 13, 2015
“Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariant Shift”
— Google: 4.82%, Feb. 11, 2015
IMAGENET CHALLENGE
Accuracy %
2010 201420122011 2013
74%
84%
DNN
CV
72%
![Page 19: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/19.jpg)
THE BIG BANG
![Page 20: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/20.jpg)
DEEP LEARNINGVISUALIZED
![Page 21: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/21.jpg)
GPU-ACCELERATED DEEP LEARNINGSTART-UPS
![Page 22: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/22.jpg)
Detecting Mitosis in Breast Cancer Cells— IDSIA
Predicting the Toxicity of New Drugs— Johannes Kepler University
Understanding Gene Mutation to Prevent Disease— University of Toronto
DEEP LEARNING REVOLUTIONIZING MEDICAL RESEARCH
![Page 23: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/23.jpg)
“Automated Image Captioning with ConvNets and Recurrent Nets”
—Andrej Karpathy, Fei-Fei Li
![Page 24: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/24.jpg)
DIGITSDEEP GPU TRAINING SYSTEMFOR DATA SCIENTISTS
Design DNNs
Visualize activations
Manage multiple trainings
USER INTERFACE
Visualize Layers
Configure DNN
Process Data
GPUGPU HW CloudGPU ClusterMulti-GPU
TheanoTorch
Monitor Progress
CaffecuDNN, cuBLAS
CUDA
![Page 25: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/25.jpg)
Monitor Progress
DIGITS
Configure DNNProcess Data Visualize Layers Test Image
![Page 26: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/26.jpg)
DIGITS DEVBOXWorld’s fastest GPU
Max GPU out of a plug
Multi-GPU training & inference
![Page 27: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/27.jpg)
“ I’ve never seen AlexNetrun this fast…TitanX is a monster, Crazy Fast”
DIGITS DEVBOX — EARLY RESULTS
“DIGITS makes it way easier to design the best networkfor the job”
0x
1x
2x
3x
4x
1 2 4
Multi-GPU scaling on Torch
AlexNet VGG
— Simon OsinderoA.I. Architech
— Soumith ChintalaResearch Engineer
![Page 28: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/28.jpg)
DIGITS DEVBOX
Available May 2015$15,000
![Page 29: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/29.jpg)
FOUR ANNOUNCEMENTS
A New GPUand
Deep Learning
A Very Fast Boxand
Deep Learning
Roadmap Revealand
Deep Learning
Self-Driving Carsand
Deep Learning
![Page 30: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/30.jpg)
SGEM
M /
W
2012 20142008 2010 2016
48
36
12
0
24
60
2018
72
Tesla Fermi
Kepler
Maxwell
PascalMixed Precision3D MemoryNVLink
Volta
GPU ROADMAPPascal 2x SGEMM/W
![Page 31: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/31.jpg)
Fram
e Bu
ffer
Cap
acit
y (G
B)
2012 20142008 2010 2016
40
30
10
0
20
50
2018
60
Tesla FermiKepler
Maxwell
PascalMixed Precision3D MemoryNVLink
Volta
GPU ROADMAPPascal 2.7x Memory Capacity
![Page 32: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/32.jpg)
HG
EMM
/ W
2012 20142008 2010 2016
96
72
24
0
48
120
2018
144
Tesla Fermi Kepler
Maxwell
PascalMixed Precision3D MemoryNVLink
Volta
GPU ROADMAPPascal 4x Mixed Precision
![Page 33: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/33.jpg)
STRE
AM G
B/s
2012 20142008 2010 2016
600
450
150
0
300
750
2018
900
Tesla
FermiKepler
Maxwell
PascalMixed Precision3D MemoryNVLink
Volta
GPU ROADMAPPascal 3x Bandwidth
![Page 34: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/34.jpg)
PASCAL 10X MAXWELL
CONVOLUTION FULLY CONNECTED FULLY CONNECTED CONVOLUTION(compute) (bandwidth) (bandwidth) (compute)
WEIGHT UPDATE(interconnect)
4x (FP16) 6x 6x 4x 10x
Mixed Precision 3D Memory NVLINK
forward backward
Mixed Precision3D Memory
5x 2x
* Very rough estimates
![Page 35: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/35.jpg)
FOUR ANNOUNCEMENTS
A New GPUand
Deep Learning
A Very Fast Boxand
Deep Learning
Roadmap Revealand
Deep Learning
Self-Driving Carsand
Deep Learning
![Page 36: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/36.jpg)
TODAY’S ADAS
PLAN ACT
CPU
WARN
FPGACV ASIC
SENSE
BRAKE
![Page 37: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/37.jpg)
NEXT-GENERATION ADAS
PLAN ACT
CPU
WARN
FPGACV ASIC
SENSE
BRAKE
STEER
ACCELERATE
![Page 38: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/38.jpg)
NVIDIA DRIVE PX SELF-DRIVING CAR COMPUTER
PLAN ACT
CPUWARN
FPGACV ASIC
DNN
SENSE
BRAKE
STEER
IMAGENET CHALLENGE
Accuracy %
2010 201420122011 2013
74%
84%
DNN
CV
72%
ACCELERATE
![Page 39: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/39.jpg)
NVIDIA DRIVE PX SELF-DRIVING CAR COMPUTER
PLAN ACT
CPUWARN
FPGACV ASIC
DNN
SENSE
BRAKE
STEER
IMAGENET CHALLENGE
Accuracy %
2010 201420122011 2013
74%
84%
DNN
CV
72%
ACCELERATE
![Page 40: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/40.jpg)
NVIDIA DRIVE PX SELF-DRIVING CAR COMPUTER
PLAN ACT
CPUWARN
FPGACV ASIC
DNN
SENSE
BRAKE
STEER
IMAGENET CHALLENGE
Accuracy %
2010 201420122011 2013
74%
84%
DNN
CV
72%
ACCELERATE
![Page 41: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/41.jpg)
NVIDIA DRIVE PX SELF-DRIVING CAR COMPUTER
PLAN ACT
CPUWARN
FPGACV ASIC
DNN
SENSE
BRAKE
STEER
IMAGENET CHALLENGE
Accuracy %
2010 201420122011 2013
74%
84%
DNN
CV
72%
ACCELERATE
![Page 42: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/42.jpg)
NVIDIA DRIVE PX SELF-DRIVING CAR COMPUTER
PLAN ACT
CPUWARN
FPGACV ASIC
DNN
SENSE
BRAKE
STEER
IMAGENET CHALLENGE
Accuracy %
2010 201420122011 2013
74%
84%
DNN
CV
72%
ACCELERATE
![Page 43: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/43.jpg)
NVIDIA DRIVE PX SELF-DRIVING CAR COMPUTER
PLAN ACT
CPUWARN
FPGACV ASIC
DNN
SENSE
BRAKE
STEER
IMAGENET CHALLENGE
Accuracy %
2010 201420122011 2013
74%
84%
DNN
CV
72%
ACCELERATE
![Page 44: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/44.jpg)
DNN-based self-driving robot
Training data by human driver
No hand-coded CV algorithms
PROJECT LEADSUrs Muller: Chief Architect, Autonomous Driving, NVIDIA
Yann LeCun: Director,AI Research, Facebook
PROJECT DAVE — DARPA AUTONOMOUS VEHICLE
IMAGENET CHALLENGE
Accuracy %
2010 201420122011 2013
74%
84%
DNN
CV
72%
![Page 45: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/45.jpg)
DAVE IN ACTION
![Page 46: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/46.jpg)
TRAINING DATA225K Images
![Page 47: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/47.jpg)
TEST DRIVENo Training
![Page 48: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/48.jpg)
TEST DRIVEPartially Trained (52K images)
![Page 49: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/49.jpg)
TEST DRIVEFully Trained (225K images)
![Page 50: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/50.jpg)
3,000x Faster
DAVEAlexNet onDRIVE PX
3.1 Million
12
38 Million
630 Million
184
116 Billion
Number of Connections
Frames / Second
Connections / Second
![Page 51: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/51.jpg)
NVIDIA DRIVE PX SELF-DRIVING CAR COMPUTER
PLAN ACT
CPUWARN
FPGACV ASIC
DNN
SENSE
BRAKE
STEER
IMAGENET CHALLENGE
Accuracy %
2010 201420122011 2013
74%
84%
DNN
CV
72%
ACCELERATE
![Page 52: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/52.jpg)
NVIDIA DRIVE™ PXSELF-DRIVING CAR COMPUTER
Available May 2015$10,000
![Page 53: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/53.jpg)
ELON MUSK
![Page 54: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/54.jpg)
LEAPS IN VISUAL COMPUTINGTITAN X
The World’s Fastest GPUDIGITS DevBox
GPU Deep Learning PlatformPascal — 10x Maxwell
For Deep LearningNVIDIA DRIVE PX
Deep Learning Platform for Self-Driving Cars
![Page 55: Opening Keynote at GTC 2015: Leaps in Visual Computing](https://reader036.fdocuments.net/reader036/viewer/2022062419/55a523231a28abc6348b4903/html5/thumbnails/55.jpg)