GTC DC 2017 Isaac Presentation Final...
Transcript of GTC DC 2017 Isaac Presentation Final...
![Page 1: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/1.jpg)
Gavriel State | NVIDIA | November 1st 2017
ISAAC LABSIMULATING VIRTUAL WORLDS FOR ROBOT INTELLIGENCE
![Page 2: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/2.jpg)
2
AI: A NEW ERA OF COMPUTING
PC INTERNET
AI & INTELLIGENT DEVICES
MOBILE-CLOUD
![Page 3: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/3.jpg)
3
AI PROCESSOR FOR AUTONOMOUS MACHINES
XAVIER30 TOPS DL
30WCustom ARM64 CPU512 Core Volta GPU
10 TOPS DL Accelerator
GeneralPurposeArchitectures
DomainSpecificAccelerators
Energy Efficiency
CPU
CUDAGPU
DLA
Volta
+
![Page 4: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/4.jpg)
4
ROBOTS
LEARN& PLAN
SEE, HEAR,TOUCH
ACT
![Page 5: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/5.jpg)
5
REINFORCEMENT LEARNING
Robot Environment
Critic
ActorAction
State + Reward
![Page 6: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/6.jpg)
6
REINFORCEMENT LEARNING - EXAMPLE
Credit: Yevgen Chebotar, Karol Hausman, Marvin Zhang, Sergey Levine
![Page 7: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/7.jpg)
7
ISAAC ROBOT SIMULATOR
NVIDIA GPU Computer
ISAACRobot Simulator
OpenAIGYM
Robot &Environment
Definition
Virtual Jetson
LEARN& PLAN
SEE, HEAR,TOUCH
ACT
![Page 8: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/8.jpg)
8
ISAAC HOCKEY
![Page 9: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/9.jpg)
9
ISAAC ROBOT SIMULATOR
Train robots in virtual environments to perform tasks in the real world
Use state-of-the-art rendering (based on Unreal Engine) and realistic physics (PhysX, FleX) to create virtual learning environments
Faster, safer, and cheaper than reinforcement learning with real robots!
Leverage the compute power of NVIDIA GPUs to train with multiple episodes and/or agents simultaneously
![Page 10: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/10.jpg)
10
PHYSICS SIMULATION
![Page 11: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/11.jpg)
11
ISAAC SIMULATION MODEL
GAME ENGINE
Rendering
Environment
Physics
VIRTUAL ROBOT
Virtual Sensors
Intelligence
Virtual Effectors
![Page 12: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/12.jpg)
12
RL Environment
SIMULATION + RL
ISAAC
Rendering Engine
Robot Definition
Environment
Gym Interface
ActionsPhysics Engine
Virtual Sensors
VirtualEffectors
Observations OpenAIBaselines
TensorFlow
![Page 13: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/13.jpg)
13
LETS DO THE TIME WARPSpeed up training using multiple agents
![Page 14: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/14.jpg)
14
ISAAC GOLF EXAMPLE
![Page 15: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/15.jpg)
15
ISAAC DOMINOES
![Page 16: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/16.jpg)
16
ISAAC DOMINOES IN VR
![Page 17: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/17.jpg)
17
ISAAC DOMINOES DETAILS
Four software components:
• Vision system
• Gameplay
• Physical movement
• Human Interaction
How all the pieces fit together
AI server
Client
![Page 18: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/18.jpg)
18
VISION – SUPERVISED LEARNINGRequires labeled data
![Page 19: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/19.jpg)
19
VISION TRAINING - SYNTHETIC DATA
![Page 20: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/20.jpg)
20
VISION RESULTS – REAL WORLD
![Page 21: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/21.jpg)
21
VISION RESULTS – REAL WORLD
![Page 22: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/22.jpg)
22
REINFORCEMENT LEARNINGContinuous Action Space
![Page 23: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/23.jpg)
23
REINFORCEMENT LEARNINGLimited Action Space
![Page 24: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/24.jpg)
24
HUMAN / ROBOT INTERACTION
![Page 25: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/25.jpg)
25
NVIDIA POWERING THE AI REVOLUTION
NVIDIA GPU Cloud
ISAAC
NVIDIA GPU in Every Cloud
Xavier DLA Open Source
DGX-1 and DGX StationTesla V100
TensorRT
Tensor Core
NVIDIAGPU CLOUD
CSPs
![Page 26: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/26.jpg)
26
Fundamentals
Autonomous Vehicles Media & Entertainment
Finance
NVIDIA DEEP LEARNING INSTITUTE
Training available as online self-paced labs and instructor-led workshops
Take self-paced labs at www.nvidia.com/dlilabs
Find or request an instructor-led workshop at www.nvidia.com/dli
Educators: download the Teaching Kit at developer.nvidia.com/teaching-kit and contact [email protected] for info on the University Ambassador Program
Machine Vision - IVA
Healthcare
…and more
![Page 27: GTC DC 2017 Isaac Presentation Final Speakernoteson-demand.gputechconf.com/gtcdc/2017/presentation/...ODEV DQG LQVWUXFWRU OHG ZRUNVKRSV 7DNH VHOI SDFHG ODEV DW ZZZ QYLGLD FRP GOLODEV)LQG](https://reader036.fdocuments.net/reader036/viewer/2022070905/5f75f67907aa081cac5fa26a/html5/thumbnails/27.jpg)