NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING
Transcript of NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING
![Page 1: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/1.jpg)
NEW MACHINE VISION SENSORS FOR
AUTONOMOUS DRIVING
RONALD MUELLER
VISION MARKETS
EMVA BUSINESS CONFERENCE 2018, DUBROVNIK
2018-06-09
![Page 2: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/2.jpg)
Trends
Full Screen image on trendsYour global growth in Imaging and Photonics
stronger
Brandmore
Salesbetter
Offeringsbigger
Company
![Page 3: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/3.jpg)
Trends
Full Screen image on trends
51 customers served
since 2015:
• MV OEMs
• Private Equity Firms
• MV-Users
• System Integrators
• Distributors
• EMSs
• Market researchers
• Mobile Phone OEM
![Page 4: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/4.jpg)
Trends
Full Screen image on trends
51 customers served
since 2015:
• MV OEMs
• Private Equity Firms
• System Integrators
• Distributors
• EMSs
• Market researchers
• Mobile Phone OEM
http://www.euclidlabs.it/event/
![Page 5: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/5.jpg)
TRENDS
full screen image on welcome and or first things first
Market Trends
![Page 6: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/6.jpg)
1893
175 15766 1,95
72
1855
177 161 982,25
72
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Mobile phones Tablets Notebooks SurveillanceCams
MV Cams Other
Million
units
Shipments of Devices with Image Sensors
2016 2017
IMAGE SENSOR MARKET
av. 2 image sensors / phone & tablet
![Page 7: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/7.jpg)
GROWTH IN SURVEILLANCE & MV
4092
1241
41997
5360
1340
604150
0
1000
2000
3000
4000
5000
6000
Hikvision FLIR Systems Cognex Basler
Turn
ove
r [m
€]
Surveillance & Machine Vision – Selected Players
2016 2017
![Page 8: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/8.jpg)
AUTOMOTIVE
73
77
80
68
70
72
74
76
78
80
82
Cars
Million
units
Global Car Production
2015 2016 2017
Every new car sold in
US is equipped with
a rear-view camera
Up to 12
image sensors/car
![Page 9: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/9.jpg)
AUTOMOTIVE VISION MARKET: ~25BN$ BY 2025
Image source: Autoliv
![Page 10: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/10.jpg)
VISION BOM IN L4/5 CARS: 385 $
Automotive
requirements are
driving sensor
technology
developments
![Page 11: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/11.jpg)
QUIZADAS & ADS
Image source: Youtube
Image source: Youtube
Image source: Youtube
Image source: Youtube
Image source: Youtube
Image source: Youtube
Image source: Youtube
![Page 12: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/12.jpg)
BASICS OF ADAS & ADS
• Standards
• SAE International – J3016:
Taxonomy and Definitions for Terms
Related to Driving Automation Systems
for On-Road Motor Vehicles
• Terminology
• ADAS: Advanced Driver Assistance System
• ADS: Autonomous Driving System
• DDT: Dynamic Driving Task
• Levels: 0 – 5
Image source: SAE International
![Page 13: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/13.jpg)
AUTONOMOUS DRIVING – SAE LEVELS 0 - 5Le
vel
NameLateral &
Longitudinal ControlMonitoring Fallback
Automated
Driving Modes
0 No Automation n/a
1 Driver Assistance Some
2 Partial Automation Some
3 Cond. Automation Some
4 High Automation Many
5 Full Automation All
![Page 14: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/14.jpg)
Level
NameLateral &
Longitudinal ControlMonitoring Fallback
Automated
Driving Modes
0 No Automation n/a
1 Driver Assistance Some
2 Partial Automation Some
3 Cond. Automation Some
4 High Automation Many
5 Full Automation All
AUTONOMOUS DRIVING – SAE LEVELS 0 - 5
![Page 15: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/15.jpg)
Level
NameLateral &
Longitudinal ControlMonitoring Fallback
Automated
Driving Modes
0 No Automation n/a
1 Driver Assistance Some
2 Partial Automation Some
3 Cond. Automation Some
4 High Automation Many
5 Full Automation All
AUTONOMOUS DRIVING – SAE LEVELS 0 - 5
![Page 16: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/16.jpg)
IMAGE SENSORS IN SAE LEVEL 1&2
•Machine Vision
• Driving lane recognition
(departure warning / automatic steering)
• 3D localization of other road users
(driver alerting / emergency breaking,
lane changes, adaptive front lights)
• Street sign recognition
• Driver monitoring
•Human vision (surround view)
•Night vision
https://www.sensorsmag.com/components/three-sensor-types-drive-autonomous-vehicles
![Page 17: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/17.jpg)
IMAGE SENSORS IN SAE LEVEL 3-5
Additional functions:
•3D sensing of the road
• Lanes, road curbs, parking cars,
junctions, construction work, etc.
•3D sensing of mobile objects
• Other vehicles, cyclists, pedestrians
• Pets and wildlife
• Obstacles
•2D recognition of traffic lights
Image source: Nvidia
![Page 18: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/18.jpg)
CHALLENGES FOR AUTOMOTIVE IMAGE SENSORS
• Sunlight, head lights, reflections
• Low light
• Compactness
• Car speed
• High spatial & depth resolution
• LED Flicker
• Temperatures
• Reliability
• Security
Image source: Sensormag
Source: Ni78.com
![Page 19: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/19.jpg)
SENSOR ARRAY IN MERCEDES S-CLASS
Image: https://blog.caranddriver.com/sensory-overload-how-the-new-mercedes-s-class-sees-all/
![Page 20: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/20.jpg)
SENSOR ARRAY IN MERCEDES S-CLASS
Image: https://blog.caranddriver.com/sensory-overload-how-the-new-mercedes-s-class-sees-all/
4 Wide angle
cameras for 360°
surround view
NIR camera
for night vision
display
Infra-red camera to spot
people and wildlife up to
160 m
Visible spectrum stereo camera
for object detection up to 400 m
3D sensing up to 50 m
![Page 21: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/21.jpg)
IMAGE SENSORS REQUIRED FOR ADS LEVEL 3 - 5
https://www.sensorsmag.com/components/three-sensor-types-drive-autonomous-vehicles
![Page 22: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/22.jpg)
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
Compact front-view stereo
camera modules
•3D sensing at up to 100 m at
35° FOV
•Traffic sign/light recognition
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
Image source: Denso
Image source: Subaru
![Page 23: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/23.jpg)
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
RCCC image sensors
•Max. sensitivity
•Color recognition for traffic
signs, traffic lights, lanes
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
Image source: Schematic Scholar
![Page 24: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/24.jpg)
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
Split pixels / Multi-gain readout / Super-Exposure
•Single-exposure HDR: 95 dB
•Multi-exposure HDR: 140 dB
•Full-well capacity of 100 ke-
Image source: Sony
![Page 25: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/25.jpg)
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
LED Flicker Mitigation in daylight
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
Image source: ON Semiconductor
![Page 26: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/26.jpg)
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
LiDAR
(Light Detection and
Ranging)
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
![Page 27: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/27.jpg)
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
SPADs:
Single Photon Avalanche Diodes
SiPM:
Silicon Photon Multipliers
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
![Page 28: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/28.jpg)
TECHNICAL DEVELOPMENTS IN AUTOMOTIVE
Solid state LiDAR at
905 or 1500 nm
•SLAM up to 300 m
•Costs down to
few hundred $
Image: http://www.safecar.info/early-sunsets-require-different-driving-techniques/
![Page 29: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/29.jpg)
ACHIEVEMENTS AT AUTOMOTIVE SENSORS
• Resolution: 2 MP stereo, 8 MP single
• Compactness: 2.25 – 3 µm
• Sensitivity: several ke-/lux s
• Full-well capacity: 100 ke-
• Dark noise: 1 e-
• High Dynamic Range:
multi-exposure: 140 dB
single-exposure: 95 dB
• Stacked sensors with on-chip processing,
e.g. motion compensation at ME-HDR
• Frame rate:
60 fps @ 1080p
40 fps @ 7.4 MP
• Heat resistance: 130°C
• Self-monitoring: ASIL B/C & IEC26262
• On-chip security features
• Solid State Lidar based on SPAD/SiPM
& MEMS
![Page 30: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/30.jpg)
SUMMARY
•Market potential of Automotive Vision
drives image sensor innovations
• Tough requirements and first advancements:
• Imaging performance (HDR, Sensitivity, …)
• Compactness
• Reliability
• Security
• Cost
• Solid State LiDAR about to be solved
![Page 31: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/31.jpg)
Tuesday, June 12, 2018© VISION MARKETS
Curious for more?
![Page 32: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/32.jpg)
NEW MARKET REPORT – AVAILABLE FROM Q3/2018
• Technology challenges deriving from application requirements for ADAS and
autonomous vehicles.
• Scoring and ranking based on technical difficulty and impact
• Detailed analysis of the current status of solutions for each top ranked technology
challenge, and forecast for the state-of-the-art until 2023
Top 20 Technology Challenges for
Image Sensors in Automotive to 2023
![Page 33: NEW MACHINE VISION SENSORS FOR AUTONOMOUS DRIVING](https://reader031.fdocuments.net/reader031/viewer/2022022219/6214dd006bed09360a440c4a/html5/thumbnails/33.jpg)
CONTACT
Ronald MüllerPresident, Vision Markets UG
t (DE): +49 89 21 553 665t (US): +1 302 231 1343
Your global growth in Imaging and Photonics