Validation & Verification€¦ · Design, Validation & Verification framework for ADAS and AV MiL /...
Transcript of Validation & Verification€¦ · Design, Validation & Verification framework for ADAS and AV MiL /...
Nico Nagl – Portfolio Development Autonomous Driving
Validation & Verification ADAS-Fahrfunktionen effizient
validieren und verifizieren
Where today meets tomorrow.Nico Nagl – Portfolio Development
Unrestricted © Siemens 2020
Page 2 Siemens Digital Industries Software
Connectivity Autonomous Vehicles
Shared MobilityVehicle Electrification
Disruptive InnovationKey to sustained business
Engineering the NEXT product not just the best product for the future
Unrestricted © Siemens 2020
Page 3 Siemens Digital Industries Software
Addressing challenges for autonomous driving
vehicle development
FROM ADAS TO AUTONOMOUS DRIVING
“+25% CAGR (through 2030) for Sensors”Roland Berger , on “Autonomous Driving”, 2014
…
“14.2 billion kilometers of testing is needed”Akio Toyoda, CEO of Toyota
Paris Auto Show 2016
“Design validation will be a major – if not the
largest – cost component”Roland Berger
“Autonomous Driving” 2014
Unrestricted © Siemens 2020
Page 4 Siemens Digital Industries Software
Engineering implications of the AV development challenge
Increasing software and hardware complexity
Massive validation and verification cycles
Growing number and variety of sensors
Complex interactions between systems
Rethink the vehicle development processes
While balancing safety, comfort and efficiency performances
Growing number and variety of sensors
Massive validation and verification cycles
Reconciling agility with better traceability
Increased hardware and software complexity
Unrestricted © Siemens 2020
Page 5 Siemens Digital Industries Software
ADAS/AV systems virtual V&V
Automotive industry needs
Virtual validation(MiL, SiL)
Semi virtual validation(HiL, DiL, VehiL)
Real validation vehicle testing
(proving ground, public road)
~106 test cases
~103 test cases
~102 test cases
~102 test cases
~102 test cases
~102 test casesSAE level 1 to 5
SAE level 1 to 5
SAE level 1 to 5
Need for efficient and automated
simulation orchestration
FAILING IN SIMULATION DOES NOT KILL PEOPLE
Unrestricted © Siemens 2020
Page 6 Siemens Digital Industries Software
Do Things Right
-
Doing the Right Things
Efficiency and Effectiveness
Unrestricted © Siemens 2020
Page 7 Siemens Digital Industries Software
ADAS/AV systems virtual V&VAutomotive industry needs
• Take not ideal world into account
• Need for realistic and non-ideal
environments
• Need for more vehicle physics than before
• Simulation of appropriate scenarios is
essential
Unrestricted © Siemens 2020
Page 8 Siemens Digital Industries Software
Design, Validation & Verification framework for ADAS and AV
MiL / SiL / Cluster
HiL / DiL / ViL
Proving ground /
field test
V&V environments
Digital Twin
“World”
Digital Twin
“Vehicle”
Design adaptations
(HW/SW)
1M – 10M scenariosRequirements
Multiple variants
Certification -
HomologationSimulation definition
Requirements &
system architectures
Real world
Vehicle under development
Unrestricted © Siemens 2020
Page 9 Siemens Digital Industries Software
Massive Verification and Validation of ADAS and AVs
Requirements
Certification -
HomologationSimulation definition
Requirements &
system architectures
Real world
Vehicle under development
Digital Twin
“World”
Digital Twin
“Vehicle”
Multiple variants
1M – 10M scenarios
MiL / SiL / Cluster
V&V environments
HiL / DiL / ViL
Proving ground /
field test
Unrestricted © Siemens 2020
Page 10 Siemens Digital Industries Software
Challenge:
From thousands of scenarios (or millions of
miles) to the relevant critical representation
Closed loop automated process
for generating critical scenarios
Orchestration of virtual test scenarios
“Falsification”
Identify critical scenarios
Digital Twin of the World
1000’s of scenarios(weather, light, road types, …) (sensors, controls,
powertrain, chassis)
…
Data Mining, Analytics
Optimize vehicle only against
relevant critical scenarios
Digital Twin of the
Test Vehicle
Unrestricted © Siemens 2020
Page 11 Siemens Digital Industries Software
Simcenter PrescanVirtual testing of autonomous driving functions
Complete sensor models library:
Camera, Radar, LIDAR, Ultrasone, Infrared, V2X, GPS
Scenario 1 - Adaptive Cruise Control ACCScenario 2 – Advanced Emergency Braking System
AEBS
Unrestricted © Siemens 2020
Page 12 Siemens Digital Industries Software
Unrestricted © Siemens 2020
Page 13 Siemens Digital Industries Software
Simcenter Prescan: camera simulationGround truth: depth camera example
Unrestricted © Siemens 2020
Page 14 Siemens Digital Industries Software
Simcenter PrescanWorld modelling solutions
Scenario import
Scripted scenario generation
Ready to use scenarios
GUI
Unrestricted © Siemens 2020
Page 15 Siemens Digital Industries Software
Simcenter PrescanWorld modelling: non-ideal environment
Realistic bumped asphalt Faded, dirty lane markers
Non-perfect lane markers Lane markers with snow Mud, water puddles on the road
Unrestricted © Siemens 2020
Page 16 Siemens Digital Industries Software
Simcenter Prescan – Scenario generationFrom real data to simulation
16
Unrestricted © Siemens 2020
Page 17 Siemens Digital Industries Software
Simcenter
Prescan
Simcenter PrescanWorld modelling: custom data source import
KITTI Dataset
Ego
GPS Data
Target
GPS Data
Ego state
Prescan API
Road
network
Target
states
Target types
Unrestricted © Siemens 2020
Page 18 Siemens Digital Industries Software
Simcenter PrescanWorld modelling: DataModel API
Explore critical scenarios Prepare for certification
• Prescan DataModel API
→ programmatic creation of scenarios
→ Repeatability
• All important assets can be created via
scripting:
• Roads
• Actors
• Traffic signs
• Nature elements
• Trajectories
• Environmental conditions
• Etc.
Parameter variation
Unrestricted © Siemens 2020
Page 19 Siemens Digital Industries Software
V2X
&
Ult
raso
nic
Rad
ar
&
Lid
ar
Cam
era
Simcenter PrescanReady to use sensor models
Sensor simulation
V2N
V2V
V2P V2I
Unrestricted © Siemens 2020
Page 20 Siemens Digital Industries Software
Simcenter PrescanSensors models: the right fidelity level for scaled-up simulation
Balancing accuracy and
computation time of
sensor simulations
Lidar (spinning and solid-state)
Example: during night-time driving Example: Realistic lighting conditions
Simcenter Prescan Physics Based Camera (PBC) simulation
Physics-based Radar simulation
Unrestricted © Siemens 2020
Page 21 Siemens Digital Industries Software
Radar simulation exampleDevelopment with model validation in mind
Two projects for radar models validation performed in close collaboration with
major Dutch Tier2 and Japanese Tier1
From a lab… To a test track… To the real world…
Unrestricted © Siemens 2020
Page 22 Siemens Digital Industries Software
Radar SimulationValidating simulation results against measured data
Real World Testing
• Vehicle with radar
• Range-doppler measurements
Simulation Testing
• Simulated vehicle using the
physics-based radar model
• Range-doppler data generated
based on the simulated scenario
Unrestricted © Siemens 2020
Page 23 Siemens Digital Industries Software
Simcenter AmesimWhen higher fidelity vehicle dynamics makes the difference!
For AEBS, ESC pump dynamics is
critical.
For level 4-5, redundancy will be
ensured by the ESC, the EPB and
the eBooster.
When level 4-5, we
will probably work with
steer by wire and
motor redundancy.Powertrain and braking systems
models for ACC cases
Pick the relevant fidelity level from
Simcenter Amesim
scalable modeling offer
Full vehicle dynamics models for
AEB safety cases
Unrestricted © Siemens 2020
Page 24 Siemens Digital Industries Software
Simcenter AmesimWhen higher fidelity vehicle dynamics makes the difference!
45 Libraries / 4,000 Multi-physics Models
• Validated and maintained
• Supporting multiple levels of complexity
• No need for details physics expertise
• Hydraulic, hydraulic component design
• Hydraulic resistance, filling
• Pneumatic, pneumatic component design
• Gas Mixture, moist airFluids
• Signal and control
• Engine signal generator
• Real time, MIL – SIL – HIL
Control
• 1D – 2D – 3D mechanical,
• Transmission, cam and followers
• Finite-elements import
• Vehicle dynamicsMechanics
• IFP drive, IFP engine
• IFP exhaust
• CFD-1D
IC Engine
• Electrical basics, electromechanical
• Electrical motors and drives
• Electrical static conversion
• Automotive electrics, electrochemicalElectrics
• Fuel cell
• Battery
• Power generation
Energy
• Thermal, thermal hydraulics
• Thermal-hydraulic component design
• Cooling, air-conditioning
• Two-phase flowThermal
Unrestricted © Siemens 2020
Page 25 Siemens Digital Industries Software
Simcenter Prescan360
Scenario authoring
Models integration environment
Sensors and environment simulationSimcenter Prescan
Process automation
Simulation plan orchestrator
Results analysis and reportingHEEDS
Third partyVehicle dynamics
Off-the-shelf validation
scenarios, metrics and
dashboards
Vehicle dynamicsSimcenter Amesim
OR
Unrestricted © Siemens 2020
Page 26 Siemens Digital Industries Software
Simcenter Prescan360
Simulation production: overall workflow and AEBS example
Numerous results analysis
Reporting
Simulation plan definition
Simulation plan execution
Scenarios Ego models
Azure
Kubernetes
Unrestricted © Siemens 2020
Page 27 Siemens Digital Industries Software
Scripted scenario generation automates the process of creating scenarios at scale
Test Automation /
Design
Optimization Tool
Orchestration
• HEEDS
• Prescan APIs
• 3rd party tooling
Scenario Change
• Parametric sweeping
• Design of Experiments
• Optimization
How to run
• Single machine
• Distributed
• Cloud and cluster
Results
• Local
• Cluster
• Test automation Interface
Simcenter Prescan360
Create wide variability with on cloud and cluster
Unrestricted © Siemens 2020
Page 28 Siemens Digital Industries Software
Simcenter Prescan360
Do not simulate any scenario.
Simulate critical scenarios related to
your application!
BUT
How do we identify critical scenarios? Should this scenario be simulated?
Unrestricted © Siemens 2020
Page 29 Siemens Digital Industries Software
Unrestricted © Siemens 2020
Page 30 Siemens Digital Industries Software
Simcenter Prescan360 Benefits
Plan Execute Report
• Process Automation: avoidance of manual errors
• No manual creation of scenarios saves time
• Multiple scenario testing for algorithms
• Identify critical scenarios for each individual application
• Deep insight in highly complex correlations
• Realistic simulations
• Verification traceability: ready for regulations
• SAFE TIME
• REDUCE COSTS
• ENSURE HIGHEST QUALITY
• BE INNOVATIVE
Speed Up the Development of
Autonomous Vehicles with
Simcenter Prescan360
Where today meets tomorrow.Nico Nagl - E-Mail: [email protected]