Nov 2018
NVIDIA AUTONOMOUS VEHICLE STACK
KEY TAKEAWAYS
One Architecture
From L2 to L5HW, OS, SWData CenterTest and Simulation
Safety
ECU, ChipHypervisor, OS ToolchainSoftware, Algorithm
Security
No-trust ModelDeep InspectionPrevention-DetectionAudit
Real Time
High PerformanceLow Latency
High Accuracy
DNNsMotion EstimationOnline Calibration…
Customizable
Framework and Plugin
PEGASUS (Robo Taxi)XAVIER (L3/L4)XAVIER* (L2)
One AV Architecture for L2 up to Robo Taxi
ONE ARCHITECTURE
NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 4
DRIVE AGX XAVIERAV Software Platform
Hardware
Software
I/O
OS/3rd Party SoftwareNVIDIA SoftwareHardware
Hypervisor
Radar Lidar GPS IMU
XavierAurix/NXP uC
CANFlexRay 10G/1G/BR EthernetGPIO
DRIVE OS
CUDA
TensorRTcuDNN
QNX BSP
OS Drivers, USB,
File System, Network
NvM
edia
EG
L 1
.5
VPIAdaptive (ARA)
L1 S
afe
ty S
erv
ices
Foundati
on S
erv
ices
Securi
ty S
erv
ices
Com
munic
ati
on S
erv
ices
L3 S
afe
ty S
erv
ices
Securi
ty S
erv
ices
Com
munic
ati
on S
erv
ices
Dia
gnost
ics
Cameras
RS
232
GMSL
(via CSI-2)NOR UFSeMMCPCIe NVMEPMIC
Scalability w/dGPU
Open G
L
ES 3
.2
NVLINK
Scalability+Xavier
or Optional Processor
Pow
er
Managem
ent
L2 S
afe
ty S
erv
ices
DRIVE AV
Data
Dis
trib
uti
on (
DD
S)
SO
ME/IP
NVIDIA DRIVE SOFTWARE
DRIVE Software
DRIVE AV
DriveWorks SDK
DRIVE AR
DRIVE OS
NVMedia CUDA TensorRT
DRIVE Core
DRIVE CalibrationDRIVE Networks
DRIVE IX
Gaze Head Pose Face Recognition
Gesture Voice Recognition & Lip Reading
DRIVE Perception
DRIVE Mapping
DRIVE PlanningDRIVE AV Viz
DRIVE IX Viz
DRIVE AV Application
(Roadrunner)
SIMULATION SCENARIOS
NVIDIA DRIVE — WORLD’S FIRST AUTONOMOUS VEHICLE PLATFORM
Lip Reading
Gaze Tracking
XAVIER
DRIVE AV DRIVE IX
DRIVE AR
DRIVE OS
BB8
NG SoC
DRIVE PEGASUS
Head Tracking
Face Recognition
COMPUTER
CONSTELLATION HIL
CONSTELLATION IG
DRIVE SIM
WAKANDA
SIMULATION
COVERAGE MAP
SATURNV
DATA FACTORY
MAGLEV
TRAINING
SOTIFISO/WD PAS 21448
ASIL-DISO 26262
OPEN MAP
NCAP
SAFETY
NVIDIA DRIVE — WORLD’S FIRST AUTONOMOUS VEHICLE PLATFORM
DRIVE IXAvailable Now
DRIVE AGX XAVIER DEVELOPER KITAvailable Now
DRIVE AVAvailable Now
HIGH LEVEL ARCHITECTURE
DriveWorks
Drive AV
RoadRunner
MapWorks
DNNs
Self-Calibration
Obstacle Perception Path Perception Wait Perception
Egomotion Estimation
Localization
Sensor Abstraction
Low Level Processing
Actuation
Cloud Mapping
Map Perception
DriveNet
OpenRoadNet
Motion/Structure
PilotNet
LaneNet
PathNet
Frame Packaging and
Serialization
Map Creation
and updatesDriveNet
WaitNet
HD Map
MapNet
Route Planner
Control
World Model
Lane Planner
Behavior Planner
Safety Force Field
Search Based Planning AI Based Prediction
LSA ImageSafety Checks
Learned Planning
SubModule
MAIN FUNCTIONS ON-RAMP TO OFF-RAMP
Rich Perception and Control
Surround Perception and Lane Changes
Localization
Planning
ON-RAMP/OFF-RAMP
SURROUND
MYROUTE
SHUTTLE
SELF CALIBRATION
CAMERA PITCH/YAW/ROLL CALIBRATION
CAMERA HEIGHT CALIBRATION
EGOMOTION
PERCEIVED IMU/ODOMETRY EGOMOTION
PERCEIVED CAMERA EGOMOTION (SLAM)
OBSTACLE PERCEPTION
DRIVENET
OBSTACLE TRACKING
OPENROAD NET
MOTION DETECTION
LIDAR DETECTION AND TRACKING
RADAR DETECTION AND TRACKING
PATH PERCEPTION
FACTORED PILOTNET TURNS
LANE NET+TRACKING+FUSION
NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
PATH FUSION
LANE ASSIGNMENT (OIPA)
WAIT PERCEPTION
INTERSECTION DETECTION
MAP PERCEPTION
LOCALIZATION
CAMERA LOCALIZATION
NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
LIDAR LOCALIZATION
KEY TAKEAWAYS
One Architecture
From L2 to L5HW, OS, SWData CenterTest and Simulation
Safety
ECU, ChipHypervisor, OS ToolchainSoftware, Algorithm
Security
No-trust ModelDeep InspectionPrevention-DetectionAudit
Real Time
High PerformanceLow Latency
High Accuracy
DNNsMotion EstimationOnline Calibration…
Customizable
Framework and Plugin
Top Related