Nov 2018

NVIDIA AUTONOMOUS VEHICLE STACK

KEY TAKEAWAYS

One Architecture

From L2 to L5HW, OS, SWData CenterTest and Simulation

Safety

ECU, ChipHypervisor, OS ToolchainSoftware, Algorithm

Security

No-trust ModelDeep InspectionPrevention-DetectionAudit

Real Time

High PerformanceLow Latency

High Accuracy

DNNsMotion EstimationOnline Calibration…

Customizable

Framework and Plugin

PEGASUS (Robo Taxi)XAVIER (L3/L4)XAVIER* (L2)

One AV Architecture for L2 up to Robo Taxi

ONE ARCHITECTURE

NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE. 4

DRIVE AGX XAVIERAV Software Platform

Hardware

Software

I/O

OS/3rd Party SoftwareNVIDIA SoftwareHardware

Hypervisor

Radar Lidar GPS IMU

XavierAurix/NXP uC

CANFlexRay 10G/1G/BR EthernetGPIO

DRIVE OS

CUDA

TensorRTcuDNN

QNX BSP

OS Drivers, USB,

File System, Network

NvM

edia

EG

L 1

.5

VPIAdaptive (ARA)

L1 S

afe

ty S

erv

ices

Foundati

on S

erv

ices

Securi

ty S

erv

ices

Com

munic

ati

on S

erv

ices

L3 S

afe

ty S

erv

ices

Securi

ty S

erv

ices

Com

munic

ati

on S

erv

ices

Dia

gnost

ics

Cameras

RS

232

GMSL

(via CSI-2)NOR UFSeMMCPCIe NVMEPMIC

Scalability w/dGPU

Open G

L

ES 3

.2

NVLINK

Scalability+Xavier

or Optional Processor

Pow

er

Managem

ent

L2 S

afe

ty S

erv

ices

DRIVE AV

Data

Dis

trib

uti

on (

DD

S)

SO

ME/IP

NVIDIA DRIVE SOFTWARE

DRIVE Software

DRIVE AV

DriveWorks SDK

DRIVE AR

DRIVE OS

NVMedia CUDA TensorRT

DRIVE Core

DRIVE CalibrationDRIVE Networks

DRIVE IX

Gaze Head Pose Face Recognition

Gesture Voice Recognition & Lip Reading

DRIVE Perception

DRIVE Mapping

DRIVE PlanningDRIVE AV Viz

DRIVE IX Viz

DRIVE AV Application

(Roadrunner)

SIMULATION SCENARIOS

NVIDIA DRIVE — WORLD’S FIRST AUTONOMOUS VEHICLE PLATFORM

Lip Reading

Gaze Tracking

XAVIER

DRIVE AV DRIVE IX

DRIVE AR

DRIVE OS

BB8

NG SoC

DRIVE PEGASUS

Head Tracking

Face Recognition

COMPUTER

CONSTELLATION HIL

CONSTELLATION IG

DRIVE SIM

WAKANDA

SIMULATION

COVERAGE MAP

SATURNV

DATA FACTORY

MAGLEV

TRAINING

SOTIFISO/WD PAS 21448

ASIL-DISO 26262

OPEN MAP

NCAP

SAFETY

NVIDIA DRIVE — WORLD’S FIRST AUTONOMOUS VEHICLE PLATFORM

DRIVE IXAvailable Now

DRIVE AGX XAVIER DEVELOPER KITAvailable Now

DRIVE AVAvailable Now

HIGH LEVEL ARCHITECTURE

DriveWorks

Drive AV

RoadRunner

MapWorks

DNNs

Self-Calibration

Obstacle Perception Path Perception Wait Perception

Egomotion Estimation

Localization

Sensor Abstraction

Low Level Processing

Actuation

Cloud Mapping

Map Perception

DriveNet

OpenRoadNet

Motion/Structure

PilotNet

LaneNet

PathNet

Frame Packaging and

Serialization

Map Creation

and updatesDriveNet

WaitNet

HD Map

MapNet

Route Planner

Control

World Model

Lane Planner

Behavior Planner

Safety Force Field

Search Based Planning AI Based Prediction

LSA ImageSafety Checks

Learned Planning

SubModule

MAIN FUNCTIONS ON-RAMP TO OFF-RAMP

Rich Perception and Control

Surround Perception and Lane Changes

Localization

Planning

ON-RAMP/OFF-RAMP

SURROUND

MYROUTE

SHUTTLE

SELF CALIBRATION

CAMERA PITCH/YAW/ROLL CALIBRATION

CAMERA HEIGHT CALIBRATION

EGOMOTION

PERCEIVED IMU/ODOMETRY EGOMOTION

PERCEIVED CAMERA EGOMOTION (SLAM)

OBSTACLE PERCEPTION

DRIVENET

OBSTACLE TRACKING

OPENROAD NET

MOTION DETECTION

LIDAR DETECTION AND TRACKING

RADAR DETECTION AND TRACKING

PATH PERCEPTION

FACTORED PILOTNET TURNS

LANE NET+TRACKING+FUSION

NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.

PATH FUSION

LANE ASSIGNMENT (OIPA)

WAIT PERCEPTION

INTERSECTION DETECTION

MAP PERCEPTION

LOCALIZATION

CAMERA LOCALIZATION

NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.

LIDAR LOCALIZATION

KEY TAKEAWAYS

One Architecture

From L2 to L5HW, OS, SWData CenterTest and Simulation

Safety

ECU, ChipHypervisor, OS ToolchainSoftware, Algorithm

Security

No-trust ModelDeep InspectionPrevention-DetectionAudit

Real Time

High PerformanceLow Latency

High Accuracy

DNNsMotion EstimationOnline Calibration…

Customizable

Framework and Plugin