Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering...
Transcript of Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering...
![Page 1: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/1.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Self-driving Car From An Engineering Perspective
Chenggang Liu
January 31, 2018
![Page 2: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/2.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Outline
Introduction
Safety
AI or Not
System Design Consideration
Motion Planning Design Consideration
Summary
![Page 3: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/3.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Introduction
The following are my personal opinions, if you have comments, questionsor ideas, please feel free to send me Emails at cgliu2008 AT gmail.com
![Page 4: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/4.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Introduction
People don’t believe self-driving car is achievable argue that:• A computer system is not reliable and safe enough.• Bugs in software are inevitable.• A malicious attack can always cause trouble.• Collisions are inevitable.• There are a lot of unsolved AI problems, for example, human intentprediction.
However, I deeply believe self-driving car is a achievable goal inreasonable near future, here is why and how.
![Page 5: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/5.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Introduction
People don’t believe self-driving car is achievable argue that:• A computer system is not reliable and safe enough.• Bugs in software are inevitable.• A malicious attack can always cause trouble.• Collisions are inevitable.• There are a lot of unsolved AI problems, for example, human intentprediction.
However, I deeply believe self-driving car is a achievable goal inreasonable near future, here is why and how.
![Page 6: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/6.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
How to Design A Safety Critical System
Figure: Examples of safety critical systems
![Page 7: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/7.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
The 10−9 Challenge 1
Critical system services must be more reliable than any one of thecomponents: e.g., System Dependability 1 FIT–Component dependability1000 FIT (1 FIT: 1 failure in 109 hours)
• Architecture must be distributed and support fault-tolerance tomask component failures.
• A system as a whole is not testable to the required level ofdependability.
• The safety argument is based on a combination of experimentalevidence about the expected failure modes and failures rates offault-containment regions (FCR) and a formal dependability modelthat depicts the system structure from the point of view ofdependability.
• Independence of the FCRs is a critical issue
1From a federated to an integrated architecture for dependable embedded systems,H. Kopetz, TU Wien, September 2004
![Page 8: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/8.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Independence of FCRs
There are two basic mechanisms that compromise the independence ofFCRs
• Missing fault isolation among the FCRs• Error propagation–the consequences of a fault, the ensuing error,propagates to a healthy FCR by an erroneous message.
![Page 9: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/9.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Integrated Architecture
Figure: The transition from a federated architecture to an integratedarchitecture
![Page 10: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/10.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Safety Consideration for Integrated Architecture
A number of technical and economic advantages could be realized if thedifferent DASes were integrated into a single architecture
• Cost savings by the reduction of nodes, sensors and wiring points(results also in an increase in hardware reliability).
• Better integration of functions–more flexibility• Implementation of fault tolerance simplifiedBut
• Independence of individual DAS compromised–increased potential oferror propagation from one DAS to another DAS
• Integration increases complexity and diagnostics• Allocation of responsibility more difficult
![Page 11: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/11.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Platform Safety• DO-297 Integrated Modular Avionics (IMA) Development Guidanceand Certification Considerations
Figure: A380 Integrated Modular Avionics (IMA) system
![Page 12: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/12.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Platform Safety - OS
• Operating System ARINC 653 (Avionics Application StandardSoftware Interface) a software specification for space and timepartitioning in safety-critical avionics real-time operating systems(RTOS).
![Page 13: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/13.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Platform Safety - Network
• AFDX Avionics Full-Duplex Switched Ethernet (AFDX)• ARINC 664
![Page 14: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/14.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Platform Safety - Software
• Software DO-178B, Software Considerations in Airborne Systemsand Equipment Certification
Figure: DO-178B Software Development Processes Objectives
![Page 15: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/15.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Platform Safety - Hardware
• DO-254, Design Assurance Guidance For Airborne ElectronicHardware
Figure: DO-254 Hardware Control Category
![Page 16: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/16.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Integrated Modular Self-driving System
• Partitioning system, the performance of each system must beunaffected by any other
• To allow systems to be developed, tested and verified separately• To allow system faults to be contained• To allow new systems to be added post certification
• For self-driving platform, we need to have partitioned computing,communication, and interface resources.
Safety can’t be achieved by testing, but by a careful plan, design,implementation, and validation and verification process.For self-driving cars, it is impractical to follow the same process as whatin Aviation for now. But a minimal system engineering effort is stillrequired, which will save money and time.
![Page 17: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/17.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Integrated Modular Self-driving System
• Partitioning system, the performance of each system must beunaffected by any other
• To allow systems to be developed, tested and verified separately• To allow system faults to be contained• To allow new systems to be added post certification
• For self-driving platform, we need to have partitioned computing,communication, and interface resources.
Safety can’t be achieved by testing, but by a careful plan, design,implementation, and validation and verification process.For self-driving cars, it is impractical to follow the same process as whatin Aviation for now. But a minimal system engineering effort is stillrequired, which will save money and time.
![Page 18: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/18.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Integrated Modular Self-driving System
• Partitioning system, the performance of each system must beunaffected by any other
• To allow systems to be developed, tested and verified separately• To allow system faults to be contained• To allow new systems to be added post certification
• For self-driving platform, we need to have partitioned computing,communication, and interface resources.
Safety can’t be achieved by testing, but by a careful plan, design,implementation, and validation and verification process.
For self-driving cars, it is impractical to follow the same process as whatin Aviation for now. But a minimal system engineering effort is stillrequired, which will save money and time.
![Page 19: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/19.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Integrated Modular Self-driving System
• Partitioning system, the performance of each system must beunaffected by any other
• To allow systems to be developed, tested and verified separately• To allow system faults to be contained• To allow new systems to be added post certification
• For self-driving platform, we need to have partitioned computing,communication, and interface resources.
Safety can’t be achieved by testing, but by a careful plan, design,implementation, and validation and verification process.For self-driving cars, it is impractical to follow the same process as whatin Aviation for now. But a minimal system engineering effort is stillrequired, which will save money and time.
![Page 20: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/20.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?Driver-less has already been achieved during the DARPA RoboticsChallenges!
Figure: The robot drove a car by itself in the DARPA Robotics Challenge
![Page 21: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/21.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?
Figure: Tele-assistant system behind the scene
![Page 22: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/22.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?
• Autonomous 6= Driver-less
• ’AI problems are problems that haven’t been solved yet’.• Self-driving problem is not an ’AI’ problem
Machine learning methods are good ways to improve performance, but becareful when you decide to use it. They are promising but not magic andthe non-interpretative issues with the black-box learning approaches maytrap us before achieving acceptable performance. Pure data-drivenapproaches are expensive and hard to deliver on time.
![Page 23: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/23.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?
• Autonomous 6= Driver-less
• ’AI problems are problems that haven’t been solved yet’.• Self-driving problem is not an ’AI’ problem
Machine learning methods are good ways to improve performance, but becareful when you decide to use it. They are promising but not magic andthe non-interpretative issues with the black-box learning approaches maytrap us before achieving acceptable performance. Pure data-drivenapproaches are expensive and hard to deliver on time.
![Page 24: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/24.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?
• Autonomous 6= Driver-less
• ’AI problems are problems that haven’t been solved yet’.• Self-driving problem is not an ’AI’ problem
Machine learning methods are good ways to improve performance, but becareful when you decide to use it. They are promising but not magic andthe non-interpretative issues with the black-box learning approaches maytrap us before achieving acceptable performance. Pure data-drivenapproaches are expensive and hard to deliver on time.
![Page 25: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/25.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?
The system design should minimize open ’AI’ problems!
![Page 26: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/26.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?
How to minimize open AI problems?
• Limit scope by simplifying scenarios and operational conditions
• Use as much prior knowledge in the maps as possible
• Minimize system perception-reaction latency and take advantage offeedback control. The faster the system can respond to the dynamicenvironment, the less challenging are the AI problems.
• Have humans in the loop to solve the most challenging AI problem
• Take uncertainties into account during motion planning (robustmotion planning)
![Page 27: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/27.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?
How to minimize open AI problems?• Limit scope by simplifying scenarios and operational conditions
• Use as much prior knowledge in the maps as possible
• Minimize system perception-reaction latency and take advantage offeedback control. The faster the system can respond to the dynamicenvironment, the less challenging are the AI problems.
• Have humans in the loop to solve the most challenging AI problem
• Take uncertainties into account during motion planning (robustmotion planning)
![Page 28: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/28.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?
How to minimize open AI problems?• Limit scope by simplifying scenarios and operational conditions
• Use as much prior knowledge in the maps as possible
• Minimize system perception-reaction latency and take advantage offeedback control. The faster the system can respond to the dynamicenvironment, the less challenging are the AI problems.
• Have humans in the loop to solve the most challenging AI problem
• Take uncertainties into account during motion planning (robustmotion planning)
![Page 29: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/29.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?
How to minimize open AI problems?• Limit scope by simplifying scenarios and operational conditions
• Use as much prior knowledge in the maps as possible
• Minimize system perception-reaction latency and take advantage offeedback control. The faster the system can respond to the dynamicenvironment, the less challenging are the AI problems.
• Have humans in the loop to solve the most challenging AI problem
• Take uncertainties into account during motion planning (robustmotion planning)
![Page 30: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/30.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?
How to minimize open AI problems?• Limit scope by simplifying scenarios and operational conditions
• Use as much prior knowledge in the maps as possible
• Minimize system perception-reaction latency and take advantage offeedback control. The faster the system can respond to the dynamicenvironment, the less challenging are the AI problems.
• Have humans in the loop to solve the most challenging AI problem
• Take uncertainties into account during motion planning (robustmotion planning)
![Page 31: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/31.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What are we trying to solve?
How to minimize open AI problems?• Limit scope by simplifying scenarios and operational conditions
• Use as much prior knowledge in the maps as possible
• Minimize system perception-reaction latency and take advantage offeedback control. The faster the system can respond to the dynamicenvironment, the less challenging are the AI problems.
• Have humans in the loop to solve the most challenging AI problem
• Take uncertainties into account during motion planning (robustmotion planning)
![Page 32: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/32.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Lesson Learned from the DARPA Robotics Challenge
• Nimble robots win!• A hierarchical optimization architecturebecomes popular.
• High-speed feedback control is themost efficient way to handleuncertainties and model errors.
![Page 33: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/33.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Lesson learned from the egress task
• Get the robot outside of the car• Challenges:
• Keep balance• Maintain contacts• Highly constrained space• Uncertainties
High-speed feedback control is critical tothe success!
![Page 34: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/34.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Lesson learned from the egress task
• Get the robot outside of the car• Challenges:
• Keep balance• Maintain contacts• Highly constrained space• Uncertainties
High-speed feedback control is critical tothe success!
![Page 35: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/35.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What makes a system fragile?
• ’Perfectness’ assumption. Design a motion planning systemassuming that the perception and the prediction system are ’perfect’.
• A death trap• To improve the perception system, it runs slower.• To improve the prediction system, it runs slower.• Because the system runs slower, the motion planning requires abetter perception and prediction systems.
• Handle failure cases separately, case by case. The final system is notconsistent.
![Page 36: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/36.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What makes a system fragile?
• ’Perfectness’ assumption. Design a motion planning systemassuming that the perception and the prediction system are ’perfect’.
• A death trap• To improve the perception system, it runs slower.• To improve the prediction system, it runs slower.• Because the system runs slower, the motion planning requires a
better perception and prediction systems.
• Handle failure cases separately, case by case. The final system is notconsistent.
![Page 37: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/37.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
What makes a system fragile?
• ’Perfectness’ assumption. Design a motion planning systemassuming that the perception and the prediction system are ’perfect’.
• A death trap• To improve the perception system, it runs slower.• To improve the prediction system, it runs slower.• Because the system runs slower, the motion planning requires a
better perception and prediction systems.
• Handle failure cases separately, case by case. The final system is notconsistent.
![Page 38: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/38.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Self-driving Architecture
![Page 39: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/39.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Self-driving Architecture
• Each module shall be self-contained and fully functional.• Each module can be tested, independently.• The system shall be developed inside-out, not the opposite.• The response time shall decrease towards the kernel.• The inner high speed loops are critical to the system robustness.• The outer modules are important to the system performance (e.g.drive speed) and capabilities (e.g. scenario handling).
![Page 40: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/40.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Self-driving Architecture
The goal is not a sum of perfect subsystems, but a harmonious systemwhose parts are consistent with each other!
![Page 41: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/41.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Self-driving Architecture
The goal is not a sum of perfect subsystems, but a harmonious systemwhose parts are consistent with each other!
![Page 42: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/42.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Performance goals
When we design a self-driving system, We should consider the system asa whole and optimize its components all altogether.The following formula show the connects between perception,localization, prediction, and planning systems 2:
clearance = v0τ +v20
2a+ 2√σ2p(0) + T 2σ2
v (0)
Divide and conquer, but don’t design separately and try to achieveunrealistic goals!
2https://cgliu.github.io/posts/self-driving/speed.html
![Page 43: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/43.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Motion planning system design consideration
• One major reason for self-driving car becoming a realistic goal isthat the perception algorithm and systems make a lot of progress inrecent years. Compared with the perception system, the motionplanning system seems more mature. You probably think it is asolved problem.
• Yes, if we can get the ground truth information in the future and wehave enough time to do planning. However, we can never get theground truth information in the future and we have to handle thereal-time performance issue in practice.
• Therefore, motion planning is NOT a solved problem (exiting!).
The perception system will never be perfect and we can never predict thefuture. We have to design a motion planning system based on this fact.
![Page 44: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/44.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Motion planning system design consideration
• One major reason for self-driving car becoming a realistic goal isthat the perception algorithm and systems make a lot of progress inrecent years. Compared with the perception system, the motionplanning system seems more mature. You probably think it is asolved problem.
• Yes, if we can get the ground truth information in the future and wehave enough time to do planning. However, we can never get theground truth information in the future and we have to handle thereal-time performance issue in practice.
• Therefore, motion planning is NOT a solved problem (exiting!).
The perception system will never be perfect and we can never predict thefuture. We have to design a motion planning system based on this fact.
![Page 45: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/45.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Motion planning system design consideration
• One major reason for self-driving car becoming a realistic goal isthat the perception algorithm and systems make a lot of progress inrecent years. Compared with the perception system, the motionplanning system seems more mature. You probably think it is asolved problem.
• Yes, if we can get the ground truth information in the future and wehave enough time to do planning. However, we can never get theground truth information in the future and we have to handle thereal-time performance issue in practice.
• Therefore, motion planning is NOT a solved problem (exiting!).
The perception system will never be perfect and we can never predict thefuture. We have to design a motion planning system based on this fact.
![Page 46: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/46.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
How to drive if collisions are inevitable?
• Yes. according to the analysis 2, there is always a risk of collision aslong as the vehicle moves.
• However:• The self-driving car is not responsible for all collisions, for example,collisions by others’ faults.
• And the collision severity levels are different.
Therefore, the design goal is not to avoid all kinds of collisions but toavoid collision in a reasonable way and show due care to inevitablecollisions or collisions caused by others’ faults.
![Page 47: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/47.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
How to drive if collisions are inevitable?
• Yes. according to the analysis 2, there is always a risk of collision aslong as the vehicle moves.
• However:• The self-driving car is not responsible for all collisions, for example,collisions by others’ faults.
• And the collision severity levels are different.
Therefore, the design goal is not to avoid all kinds of collisions but toavoid collision in a reasonable way and show due care to inevitablecollisions or collisions caused by others’ faults.
![Page 48: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/48.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
How to drive if collisions are inevitable?
• Yes. according to the analysis 2, there is always a risk of collision aslong as the vehicle moves.
• However:• The self-driving car is not responsible for all collisions, for example,collisions by others’ faults.
• And the collision severity levels are different.
Therefore, the design goal is not to avoid all kinds of collisions but toavoid collision in a reasonable way and show due care to inevitablecollisions or collisions caused by others’ faults.
![Page 49: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/49.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
How to drive if collisions are inevitable?
• Yes. according to the analysis 2, there is always a risk of collision aslong as the vehicle moves.
• However:• The self-driving car is not responsible for all collisions, for example,collisions by others’ faults.
• And the collision severity levels are different.
Therefore, the design goal is not to avoid all kinds of collisions but toavoid collision in a reasonable way and show due care to inevitablecollisions or collisions caused by others’ faults.
![Page 50: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/50.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Motion planning system design consideration
Motion planning system’s functionalities:• Navigation: travel from A to B:• Guidance: obey traffic law• Control: avoid collisions
![Page 51: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/51.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Optimization-based motion planning
• Rather than designing control policy or rules, the developers designcost functions and then let optimization algorithms figure out thebest policy
• Pros:• More direct• Easy to get the system to work• Make it possible to build a harmonious system• Compatible with Reinforcement Learning framework• Better performance
• Cons• Hard to find a good cost function• Real-time performance issues• Robustness issues
![Page 52: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/52.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Driving problem formulation
The objectives:• Minimize the time to the destination• Minimize the risk of collision• Maximize ride quality
Subject to:• Dynamics constraints• Path, control, and other temporal constraints
![Page 53: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/53.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
The risk of collision
risk = severity × exposure× probability
The expectation of collision risk:
E(risk) =∫ t
0severity(τ)p(τ)dτ
![Page 54: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/54.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
The risk of collision
Figure: Probabilistic collision
The collision probability:
p(t) ≈∫S
pav (x , y |t)pobs(x , y |t)dxdy
![Page 55: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/55.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
The risk of collision
![Page 56: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/56.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
The risk of collision
The severity level at urban drive speed (< 50 mph):
severity ∝ v
Therefore,
E (risk) ≈∫ t
0
∫S
v(τ)pav (x , y |τ)pobs(x , y |τ)dxdydτ
![Page 57: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/57.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Optimal problem formulation
U = argminu(·)
{Lf (x , u, tf ) +
∫ tf
t0
L(x , u, t)dt}
and subject to:x(0) = x0
h(x , u) ≤ 0
E(risk) ≤ risk_level
The cost functions should take the collision risk, ride quality, the desireddriving path and other constraints into account.
![Page 58: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/58.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Optimization-based motion planning
• Navigation (long-range)• Ignore dynamic obstacles• Low resolution, such as at lane level• spatial and temporal constraints, e.g. time-based lane• Methods: A*, D*, PRM, and etc.
• Decision making (long-range and long-term)• Simple model, low quality, long-term• Method: Dynamic Programming
• Trajectory optimization (mid-range and mid-term)• Full model, high quality, mid-term• Methods: DDP, iLQR, Direct collocation, Pseudospectral methods,
or spline + differential fatness.• Control
• Full-model, high quality, short-term• Method: Finite-horizon LQR, LQR gain scheduling, QP, ADRC and
etc.
![Page 59: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/59.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Optimization-based motion planning
• Harmony• The cost functions shall be consistent with each other for each level
• Cost functions• Manually designed based on domain knowledge
• Real-time performance• Cache cost and avoid duplicate computation• Hessian matrix approximation,• Parallelism (e.g. multiple shooting)
• Robustness• Warm-start generation• Multiple shooting
![Page 60: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/60.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
From Excellent to Superb
• Cost function• Learning from imitation (Inverse Reinforcement Learning)
• Maximum Margin Planning• Maximum Entropy Inverse Reinforcement Learning
• Trial and error (Reinforcement Learning, e.g. trajectory-basedReinforcement Learning 3).
• Real-time performance• Cache motion planning priors, e.g. use a offline generated library 4, 5
• Hierarchical optimization architecture 6
• Long-term optimization optimizes for highly-likely, slowly-changingthings
• Short-term optimization optimizes for less-likely, fast-changing things
3Trajectory-based dynamic programming4Standing balance control using a trajectory library5Biped walking control using offline and online optimization6Optimization-based Full Body Control for the DARPA Robotics Challenge
![Page 61: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/61.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
From Excellent to Superb
• Robustness• High speed feedback control, 7
• Warm-start generation, e.g. using a non-parametric optimizer togenerate a warm-start for a parametric optimizer8.
• Plan for uncertainties• Hindsight optimization• Belief-space planning
7Full-body motion planning and control for the car egress task of the DARPArobotics challenge
8Biped walking control using a trajectory library
![Page 62: Self-driving Car From An Engineering Perspective · Self-driving Car From An Engineering Perspective Author: Chenggang Liu Created Date: 1/31/2018 9:11:48 AM ...](https://reader034.fdocuments.net/reader034/viewer/2022050220/5f65ed801b41b83d72018bf4/html5/thumbnails/62.jpg)
Introduction Safety AI or Not System Design Consideration Motion Planning Design Consideration Summary
Summary
• System design shall avoid or reduce AI problems• System development should follow a similar path as the naturalevolution.
• Hierarchical optimization architecture is an efficient way to handlereal-time performance issues
• High-speed feedback control is one of the mast efficient ways toimprove system robustness.
• Evaluate system safety as a risk probability and design for it• The motion planning system design should take the uncertainty intoaccount.