Davide Scaramuzza

Are We Ready for Autonomous Drones?

1

My lab homepage: http://rpg.ifi.uzh.ch/

Publications: http://rpg.ifi.uzh.ch/publications.html

Software & Datasets: http://rpg.ifi.uzh.ch/software_datasets.html

YouTube: https://www.youtube.com/user/ailabRPG/videos

Drone Market Today: 130B$

22

Inspection Agriculture

Transport Search and Rescue

Davide Scaramuzza University of Zurich

https://www.pwc.pl/pl/pdf/clarity-from-above-pwc.pdf

Swiss Drone Industry Map

3https://droneindustry.ch/swiss-drone-industry-map/

How are current commercial drones controlled?

Remote control

Requires line of sight or communication link

Requires skilled pilots

GPS-based navigation

Drone crash during soccer match, Brasilia, 2013

Flyability - Inspection drone (remote controlled)

5

Wingtra - Autonomous Mapping (GPS-based)

6

Cargo Delivery (GPS-based)

7

Matternet & Swiss Post delivering blood samples between two hospitals in Lugano

https://www.post.ch/en/companycars/post-company-cars-ltd/media/2017/swiss-post-drone-to-fly-laboratory-samples-for-ticino-hospitals

Does not work indoors

Even outdoors it is not a reliable service

Satellite coverage

Multipath problem

What is GPS if obstructed or just not available?

9

How do we localize without GPS?

11

Ultra-Wide-Band Localization (UWB)

12A Ledergerber, M Hamer, R D'Andrea,

A robot self-localization system using one-way ultra-wideband communication, IROS 2015

Ultra-Wide-Band Localization (Verity Studios AG)

13

A Ledergerber, M Hamer, R D'Andrea,

A robot self-localization system using one-way ultra-wideband communication, IROS 2015

How can Autonomous Drones fly without external sensors

(no GPS, no UWB)?

14

LIDAR-based Localization

15

Lidar-based Localization

Google car

Lidar-based Localization

17J. Zhang and S. Singh.

Laser-visual-inertial Odometry and Mapping with High Robustness and Low Drift, Journal of Field Robotics. 2018

Vision-based Localization

18

Vision-based Localization

19

Camera

Phone PC

How Does it Work ?

Image 1Image 2

Scaramuzza, Fraundorfer, Pollefeys, Lee, Meier, Siegwart, Chli, Achtelick, Weiss, et al., Vision-Controlled Micro Flying Robots, IEEE RAM’14

Visual Simultaneous Localization and Mapping(VSLAM)

DARPA FLA Program (2015-2018)

21Mohta, Watterson, Mulgaonkar, Liu, Qu, Makineni, Saulnier, Sun, Zhu, Delmerico, Karydis, Atanasov, Loianno, Scaramuzza, Daniilidis, Taylor, Kumar

Fast, Autonomous Flight in GPS‐denied and Cluttered Environments, Journal of Field Robotics, 2018. PDF

Is then Autonomous Navigation Solved?

22

Davide Scaramuzza – University of Zurich – sdavide@ifi.uzh.ch

My Vision (10 years?): Fast, Lightweight, Autonomous

WARNING! There are 50 drones in this video but 40 are CGI and

10 are controlled via a Motion Capture System

LEXUS commercial, 2013

23

24

Perception is the biggest challenge:

Challenging illumination

Motion blur at high speeds

Open Challenges for Vision-based Drones

Artificial Intelligence

• Currently solved by Machine Learning, i.e., programming by examples: Show the computer what to do, without explaining how to do it!

• The robot programs itself to imitate the examples at best.

• Continuous self improvement via more data/experience.

25

Deep Networks Can Help Cope with Inaccurate Models

[Giusti, RAL’16], [Smolyanskiy, IROS 2017], [Loquercio, RAL’18], [Kaufmann, CORL’18, 4% Oral Acceptance Rate]

[Giusti, RAL’16], [Smolyanskiy, IROS 2017], [Loquercio, RAL’18], [Kaufmann, CORL’18]

Learning to Fly by Mimicing Cars and Bicycles

Reconfigurable Drones

Falanga, Kleber, Mintchev, Floreano, Scaramuzza, An Autonomous Vision-based Foldable Quadrotor that can Squeeze and Fly, IEEE Robotics and Automation Letters 2018, under review

Davide Scaramuzza – University of Zurich – sdavide@ifi.uzh.ch

Future: Autonomous Drone Racing?

WARNING! There are 50 drones in this video but 40 are CGI and

10 are controlled via a Motion Capture System

LEXUS commercial, 2013

WARNING! This drone flown is NOT autonomous; it is operated by a human pilot.

Human pilots take years to acquire the skills shown in this video.29