Mission planning of autonomous quadrotors
-
Upload
ivano-malavolta -
Category
Technology
-
view
51 -
download
0
Transcript of Mission planning of autonomous quadrotors
Ivano MalavoltaAssistant professor
Vrije Universiteit Amsterdam
Mission planning of autonomous quadrotors
VRIJEUNIVERSITEITAMSTERDAM
Hello
Empirical software engineering+ Software Architecture + MDE
+applied to
Complex systemsAutonomous robots
Mobile-enabled systems
If you think good architecture is expensive, try bad architecture.
... Brian Foote and Joseph Yoder
Roadmap
Background
The platform
Flight plan synthesis
Adaptation at run-time
Support for any kind of robot
Conclusions
Background
High coststeam training and transportation
operating costs
Safetysignificant risks (e.g., fire, earthquake, etc.)
Timing and enduranceexhausting shiftsactivities stopped at night
Civilian missions today
INTUITION
A team of autonomous quadrotors might work
together with staff
Special kind of helicopter with:
• high stability
• omni-directional
• smaller fixed-pitch rotors
à safer than classical helicopters
• simple to design and construct
• relatively inexpensive
http://goo.gl/FJFS5l
What is a drone for us?
Many civilian missions can be executed either by flying, ground or water robots
Using robots for civilian missions
Civilian missions can be executed by multiple robots
à lower mission completion timeà fault-toleranceà highly-specialized robots
All the robots perform their actions to fulfil the common goal of the mission
however...
common goal
Multi-robot missions
On-site operators must be expert of all the types of used robots
in terms of dynamics, hardware capabilities, etc.
On-site operators have to simultaneously control a large number of robots during the mission execution
Robots provide very low-level APIs and very basic primitives
error-prone development task-specific robotsno reuse
These issues ask for • abstraction• automation
Challenges
Application scenario
Mission
to monitor the CO2 level in a geographical area by monitoring it along a grid composed of cells of size 12x12 meters
Contextual entities
obstacles
emergency areas
no fly zones
home NO FLY ZONE
MDE allows all stakeholders to focus on models of the mission with concepts that are:
• closer to the application domain • independent from the specific robot technologies
• enabling automation à autonomous robots
http://mdse-book.com
Model-driven engineering for robotics
The platform
Mission modeling
Mission
Context
Map
MML
BL
Behavior specification
Detailed flight plan synthesis
Robot typemodels
MissionExecution
Engine
this layer is extensible
RL
Mask complexity• usable by non-technical experts• domain-specific concepts• separation of concerns
Independence w.r.t. the types of robots• behaviour generation capability
Reuse• context models• robot models
Robots must be autonomous
Principles Mission
Context
Map
MML
BL
Behavior specification
Detailed flight plan synthesis
Robot typemodels
MissionExecution
EngineRL
Modeling languages Mission
Context
Map
MML
BL
Behavior specification
Detailed flight plan synthesis
Robot typemodels
MissionExecution
EngineRL
Operator
in-the-field stakeholder specifying the mission
Robot engineermodels a specific kind of robot
develops the controller that instructs the robot on how to perform BL basic operations
Platform extenderextends the MML metamodel with new kinds of tasks
develops a synthesizer for transforming each new task to its corresponding BL operations
MML
RL + software controller
MML + synthesizer
Involved stakeholders
Mission layer
sequence of tasks executed by a team of robots
Extension point
Monitoring mission language (MML)
Photogrid task
• identifies a virtual grid within an area• each cell of the grid has a size of n meters
• each drone flies over each cell of the grid at an altitude of z meters and takes a picture of the ground
Example of extension
Context layer
geographical areas that can influence the execution of the mission
The focus is on spatial context
Monitoring mission language (MML)
Hardware and low-level configuration of each type of robot
Robot language (RL)
This makes RL models reusable and shared across missions, projects, and organizations
Atomic movements
and actions performedby each robot of the
swarm
Behaviour language (BL)
Example (1)MML model (in the tool)
NF1
NF2
home
RT
PGT
Example (2)
Robot model (Parrot)
Example (3)
Behavioural model
Drone&D1&
Drone&D2&
Drone&D3&
Start&(ε,&ε)& Start&(ε,&ε)& Start&(ε,&ε)&
TakeOff&(ε,&ε)& TakeOff&(ε,&ε)& TakeOff&(ε,&ε)&
GoTo&(ε,&ε)&GoTo&(ε,&ε)& GoTo&(ε,&ε)&
GoTo&(ε,&{Photo})&GoTo&(ε,&{Photo})& GoTo&(ε,&{Photo})&
GoTo&(ε,{Photo,BroadCast(D3.R1.Done)})&
GoTo&(ε,&ε)&
Land&(ε,&ε)&
Stop&(ε,&ε)&
GoTo&(ε,&ε)&
Land&(ε,&ε)&
Stop&(ε,&ε)&
0GoTo&(ε,&{Photo,&&BroadCast&(D2.PG1.Done)})&
0GoTo&(ε,&ε)&
Land&(ε,&ε)&
Stop&(ε,&ε)&
GoTo(ε,&{Photo,&&BroadCast&(D1.PG1.Done)})&
PG1 PG1 R1
Tool support…
ROS
HTTPS and web sockets
Web interface
Editor for MML models
M2M transformation+ models validation+ mission workflow
manager
Layer of controllers that interpret BL models at run-time
HTML5, CSS3, JavaScript Java + OCL + Rosbridge
Any technology supported by ROS
Drone driver
any
C1
C2
C3
…
Any protocol
Web UI
Mission Design and
BehaviourSynthesis
request MML &drones configurations
1
2 MML concepts & drones
3 mission design
4 submit MML models
6 BL model and analysis results
5behaviour analysisand synthesis
7 start mission
Mission ExecutionEngine
Mission modeling workflow
Mission execution
Mission ExecutionEngine
6 send status*(continuous flow)
start mission
setup drone controllers
* steps executed in parallel
MonitoringEngine
tasks executioncommands*
feedback*
feedback*
25
1
3
4 C1
D1
Mission simulation
Simulation-specific controller • Software-In-The-Loop (SITL) platform• Same components of real deployment Mission Execution
Engine
navigationcommands(MAVLink)
telemetry(MAVLink)
C1
MAVProxy
DroneKit
Arducopterphysics
simulator
physics data
Google Earth
drones positions at run-time
DEMO
Flight plan synthesis
OVERVIEW OF THE SYNTHESIS APPROACH
BL model automatic synthesis
BL
Mission entering
Mission leaving
Mission tasks execution
……
dn
…
d2
…
d1
State transitionState
Synchronization and communication message
Overview of the synthesis approach
MML
Each MML task is constrained by:• geometry• behavioural strategy• actions
pointline polygon
volume
Geometries supported by MML
STRATEGIES AND ACTIONS SUPPORTED BY MML
sweep(d) – full coverage of an area discretized by a grid whose cells dimension is d
search(target, d) – visit performed on a d-sized grid towards the discovery of target (e.g., represented by a PNG image)
track(m, m’, d) – it is like sweep(d) where m and m’arethe restart and stop messages, respectively
Actions:• taking a picture• making a video• detecting the presence of CO2• etc.
Strategies supported by MML
SYNTHESIS METHOD AT WORK ON THE PEM SCENARIOLet’s go back to the example…
NF1
NF2
home
RT
PGT
SYNTHESIS METHOD AT WORK ON THE PEM SCENARIO
Context = ( (NF1,NF2), (OB) )
PGT = ( (p1,p2,p3,p4,p1), sweep(10), (doPhoto(2480×3508), i) )
Swarm ={ (d1, {PGT}), (d2, {PGT}), (d3, {RT}) }
Task dependency graph( (t0, PGT, RT, t3), ∅ ∪ { (t0, {PGT, RT}) } ∪ { ({PGT, RT}, t3) } )
t0
PGT RT
t3
Synthesis method on the example
q4= p2
q1= p1
q3
q14
q5
q6
q15
q16
q2
q13
q12
q11
q7
q8
q9
q10
p3
p4
c1
c2
p1
Synthesis method on the example
SYNTHESIS METHOD AT WORK ON THE PEM SCENARIO
s10 TakeOff(p1.z) s1
Mission entering
s2s'''2
Goto(c1)
s''2
Goto(c2)s'2
Goto(p1)Goto(home1)Land
s1f land1
Mission leaving
non-fluid transition fluid transition Transition label syntax = <OP> |<ACT> | <OP> / <ACT>
s'1 Goto(c2) s''1 Goto(p1) v1
v11
Goto(q1) / DoPhoto(…)
v21v161
…
NoOp
Mission tasks execution
Goto(q16) / DoPhoto(…)
NoOpr1
NoOp
Goto(c1)
v171
Synthesis method on the example
SYNTHESIS AUXILIARY FUNCTIONS
Divide – distributes the geographical area of a task into a set of (sub-)areasoverlapping between no-fly zones, obstacle, and areasno “cross-cutting” no-fly zonessmallest distance assignment criterion1 point to 1 drone, 1 line to n drones (l/n)polygon/volume partitioning algorithm (rif. H. Bast and S. Hert. The area partitioning problem. In Proceedings of the 12th Canadian
Conference on Computational Geometry, 2000)
Appr – generates the obstacle- and collision-free path that a drone d must travel to reach a geographical area a
path planning problem in 3-dimensional world (rif. S. S. Skiena. The algorithm design manual, 1997. Stony Brook, NY: TelosPr, 504)
spherical obstacle enlargementtarget points identification (vertexes of the sub-area)trajectory definition = well-known visibility graph
Cover – takes a starting position s of a drone d, a geographical area a, and a real number rrepresenting the resolution of the grid that implicitly discretizes a
returns a set of <point,angle>coverage path planning problem (rif. R. Mannadiar and I. Rekleitis. Optimal coverage of a known arbitrary environment. In Robotics
and Automation (ICRA), 2010 IEEE International Conference on, pages 5525–5530. IEEE, 2010)
minimum visit planbased on the Boustrophedon cellular decomposition
Auxiliary functions
PROPERTIES OF THE SYNTHESIZED QBL MODEL
Safety• avoidance of collisions (P1), and of traversing no-fly
zones (P2)• no concurrency issues (P3)
P1&P2• correct and complete specification• Appr• Cover
P3• Divide• no message lost• sound mechanisms for sequencing, join, and fork• a drone cannot be involved in concurrent tasks
Properties of the synthesized BL model
Adaptation at run-time
• Drone d1 of the team T must reach a target geographical position p
• Drone d1 identifies an obstacle along its trajectory towards p
• if the obstacle is avoidable (e.g., a tree):
• then d1 adapts its trajectory so that it avoids the obstacle to reach p
• otherwise (e.g., a large building):
• the behaviour of d1 and other drones in T are adapted so that:
• the position p is still covered by another drone di∈ T
• d1 can cover some other points within the area
Scenario
• Safety as first-class element in the design of the system
• Clear separation of concerns between the generic safety-specific mechanisms and the functional behavior of the robots while defining the mission
• Decentralized and collective adaptation
• Multiple entities are adapted simultaneously so that
• critical runtime condition are properly addressed
• the working consistency and the collaboration of the ensemble arepreserved
Overview
Sustainable safety via collective adaptationGOAL
Software architecture Key features:• collective adaptation in a
decentralized fashion • managed at run-time• new solvers can be
introduced at any time• separation of concerns
• Entities - basic building blocks representing the actors and components ofthe system
• Ensembles - set of roles that can be played by participating entities
Ensemble role specification
• Issue - definition of a critical situation that can happen to a role of an ensemble
• Solver - ability of a role to handle certain types of issues
Issues resolution
Support for any kind of robot
• Modeling languages• special care has been put in MML and BL
• Modeling infrastructure• overall software architecture• model transformations
What is independent of the used robots?
• Definition of a generic editing environment• flexible w.r.t. the use of geographical concepts
à we need to relax the constraint of always referencing to a map
• Evaluate alternative architectures• decentralized• no assumption of continuous connectivity with ground station• P2P communication between robots
• Experimentation with other kinds of physical robots• e.g., underwater autonomous robots
What needs to be done?
Step 1 - check the expressiveness of the languages
• we reverse engineered the AUV used in the RALF3 project
Step 2 - extend the modelling languages
• Mission tasks
• Context
• Behaviour
• Robot
Extension for underwater robots
no extension needed Winning features:• geo and relative coordinates• behavioural movements driven by
specific conditions (e.g., vision)• objects as behavioural targets
Extended robot language for UAVs
Conclusions
EXPERIMENTATION
models@runtime to control the mission execution and manage adaptation
Support other types of robots
We need you!Mission ExecutionEngine
navigationcommands(MAVLink)
telemetry(MAVLink)
C1
MAVProxy
DroneKit
Arducopterphysics
simulator
physics data
Google Earth
drones positions at run-time
Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, and Massimo Tivoli. Automatic Gen-eration of detailed Flight Plans from High-level Mission Descriptions. In ACM/IEEE 19thInternational Conference on Model Driven Engineering Languages and Systems (MODELS),pages 45–55. ACM/IEEE, Oct 2016.
Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione (2016) Adopting MDEfor Specifying and Executing Civilian Missions of Mobile Multi-Robot Systems IEEE AccessJournal. http://dx.doi.org/10.1109/ACCESS.2016.2613642
Darko Bozhinoski, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Massimo Tivoli (2015).FLYAQ: Enabling Non-Expert Users to Specify and Generate Missions of AutonomousMulticopters. In 30th IEEE/ACM International Conference on Automated Software Engineering(ASE 2015).
Darko Bozhinoski, Ivano Malavolta, Antonio Bucchiarone, Annapaola Marconi (2015). SustainableSafety in Mobile Multi-Robot Systems via Collective Adaptation. In Ninth IEEE InternationalConference on Self-Adaptive and Self-Organizing Systems, SASO 2015, Cambridge,Massachusetts, USA, September 21-25, 2015,
Darko Bozhinoski (2015). Managing Safety and Adaptability in Mobile Multi-Robot Systems. InProceedings of the 11th International ACM SIGSOFT Conference on Quality of SoftwareArchitectures, pp. 135–140.
References
Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione (2014). A family of Domain-SpecificLanguages for specifying Civilian Missions of Multi-Robot Systems. In Proceedings of the 1stInternational Workshop on Model-Driven Robot Software Engineering (MORSE), pp. 13–26.
Davide Di Ruscio and Ivano Malavolta and Patrizio Pelliccione (2014). The Role of Parts in theSystem Behaviour. In Software Engineering for Resilient Systems - 6th International Workshop,SERENE 2014, Budapest, Hungary, October 15-16, 2014. Proceedings, pp. 24–39.
Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione (2013). Engineering a Platform for MissionPlanning of Autonomous and Resilient Quadrotors. In Software Engineering for ResilientSystems - Fifth International Workshop, SERENE 2013, pp. 33–47.
References
ContactIvano Malavolta |
Assistant professorVrije Universiteit Amsterdam
iivanoo
www.ivanomalavolta.com