IR RoboticSoccer Coordinationlpreis/ir2007_08/documents/IR_RoboticSoccer... · Intelligent Robotics...

Intelligent Robotics Intelligent Robotics –– Coordination in MultiCoordination in Multi--Robot Systems: Applications in Robotic SoccerRobot Systems: Applications in Robotic Soccer

1

IEEE ISIE 2007

Coordination in Multi-Robot Systems: Applications in Robotic Soccer

Luís Paulo Reis [email protected]

LIACC – Artificial Intelligence and Computer Science Lab.Faculty of Engineering of the University of Porto

Rua Dr. Roberto Frias, s/n 4200-465 Porto, Portugalhttp://www.fe.up.pt/~lpreis

Intelligent Robotics Intelligent Robotics –– Coordination in MultiCoordination in Multi--Robot Systems: Applications in Robotic Soccer Robot Systems: Applications in Robotic Soccer 22

Presentation Outline

Introduction

RoboCup Initiative

RoboCup Leagues

Simulator – Soccer Server

FC Portugal Project

Coordination Methodologies

Experimental Results

Research Areas and Conclusions

Introduction | RoboCup | Rob.Leagues | Simulator | FCPortugal | Coordination Meth. | Results | Conclusions


IntroductionArtificial IntelligenceIntelligent AgentsEnvironment and Its ComplexityAgent Architectures and ApplicationsMulti-Agent Systems

RoboCup InitiativeRoboCup LeaguesSimulator – Soccer ServerFC Portugal ProjectCoordination Methodologies and ResultsResultsResearch Areas and Conclusions




Artificial Intelligence

Intelligence“Capacity to solve new problems through the use of knowledge”

Artificial Intelligence“Science concerned with building intelligent machines, that is, machines that perform tasks that when performed by humans require intelligence”



Autonomous Agents

Traditional Definition:“Computational System, situated in a given environment, that has the ability to perceivethat environment using sensors and act, in an autonomous way, in that environment using its actuators to fulfill a given function.”



Autonomous Agents

Information AgentElectronic Commerce Agent Air Traffic ControllerAutonomous PilotPersonal AssistantMeeting SchedulerGame Playing AgentAutonomous Robot



Intelligent Agents?!

Are these Agents? Are they Intelligent?Thermostat?Answering Machine?Phone?Pencil?Java or C++ Object?

Intelligence:Is this a binary (yes/no) scale?Is this a continuous scale (0-infinit)?



Agent Requisites

Traditional definition include to much or leaves “holes”!Requisites:

Perceive its environment (sensors)Decide actions to execute (“think”)Execute actions in environment using its actuatorsCommunicate?Perform a complex function?



Intelligent AgentsAgent:

Perceive its environment using sensors and executes actions using its actuatorsSensors:

Eyes, ears, nose, touch, …Actuators:

Legs, Arms, hands, vocal cords, …

Robotic Agent:Sensors:

Cameras, sonar, infra-red, microphoneActuators:

Motors, wheels, manipulators, speakers



Agent’s Proprieties

Main Proprieties:Autonomy

Pro-Activity

Reactivity

Social Ability

Other Proprieties:Mobility

Truth and Benevolence

Knowledge and Believes

Intentions, Obligations

Rationality

Intelligence

Learning

Temporal Continuity

Character



Agents vs. Objects

Essential Differences:Agents decide what to do

Object methods are called externally

Agents react to sensors and control actuators

“Objects do it for free; agents do it for money”



Agent Architectures

Physical Agents vs. Software AgentsAgent’s Structure:

Simples Reflex Agents (reactive) Agents with world representationAgents Based on ObjectivesAgents Based on UtilityBDI Agent – Believes, Desires and IntentionsAgents with complex architectures(learning, decision, planning, cooperation, etc.)



Agent Architectures

Characteristics:Simplicity FunctionalityExpansibility Portability

Types:DeliberativeReactiveHybrid

Difficult to Balance:

ReactivityDeliberationSociability

How to do it?



Agent Architectures - HybridRaciocínio

AcçõesPossíveis

DecisãoDeliberativa

Actualização doEstado do

Mundo

Conhecimento /Estado do

Mundo

Objectivos

Comunicação

Percepção

Mundo Acção

Decisão Reactiva

Execução daAcção

Regras SimplesSituação/Acção

(comportamento)

Interpretação daPercepção

Fusão dasDecisões

ComponenteReactiva

ComponenteDeliberativa



Decisão(resolução deconflitos entre

objectivos)

Execução daAcção

Actualização doEstado doMundo e

Modelos dosoutros Agentes

Conhecimento /Estado do

Mundo - Próprio

Comunicação

Percepção

Mundo

Comunicação

Interpretação daPercepção e

Comunicação ConhecimentoGeral

Agenda paraCoordenação de

Actividades

Planos dosOutros Agentes

Intenções dosOutros Agentes

Meta-Regras deRaciocínio

Modelos dosoutros Agentes

Raciocínio deModelização deoutros Agentes

Modelos doAmbiente

Construção dePlanos

(resolução deinteracções)

Acção

Execução daComunicação

Conhecimentode Aprendizagem

PlanosPredefinidos

Preferências dosAgentes

Activação dePlanos

Agenda deObjectivos

RecursosDisponíveis

Espaço deDecisões

Espaço deAcções

Planos Activados

Novos Planos

Execução dosPlanos Activados

Resultados daSimulação

Simulação dePlanos e dasReacções de

outros Agentes

Aprendizagem

Selecção eAdaptação dePlanos Pré-Definidos



Agent Applications

Industrial ApplicationsInformation SearchElectronic CommerceEntertainment Applications and GamesMedical ApplicationsIntelligent SimulationRobot ControlAutonomous DrivingIntelligent BuildingsResearch Competitions



Environment

Accessible vs. Inaccessible

Static vs. Dynamic

Discrete vs. Continuous

Deterministic vs. Non Deterministic

Single Agent vs. Multi-Agent



Multi-Agent System (MAS)

Composed by multiple agentsthat:

Exhibit autonomous behavior

Interact with the other agents in the system



MAS Motivation

Problem DimensionsLegacy SystemsNatural Solution (distributed problems)Distributed knowledge or informationHuman-machine interfaceProject Clarity and simplicityEfficiencyRobustness and ScalabilityProblem divisionInformation privacy



Agents and Multi-Agent Systems

To build individual autonomous intelligent agents is important

However:Agents don’t leave alone…

Necessary to work in group

Multi-Agent Applications

Robotic Agents: Body, complex environment

Coordination in necessary: “to work in harmony in a group”



Reasons for Coordination

Dependencies in agent actionsNeed to respect global constraintsNo agent, individually has enough resources, information or capacity to execute the task or solve the complete problemEfficiency:

Information exchange or tasks division

Prevent anarchy and chaos:Partial vision, lack of authority, conflicts, agent’s interactions



Cooperative vs Competitive MAS

Cooperative MAS:Usually projected by a single entityGlobal utility and global performance

Competitive MAS (“self-interested agents”): Each agent has a distinct designerAgents have their own motivation and agendaAgents are interested in their own utility Usual in negotiation, electronic commerce, internet




IntroductionRoboCup Initiative

Objectives of the RoboCup InitiativeRoboSoccer - Global PerspectiveComplexity of RoboSoccerRoboCup FederationRoboCup Official Competitions

RoboCup LeaguesSimulator – Soccer ServerFC Portugal ProjectCoordination Methodologies ResultsResearch Areas and Conclusions



RoboCupObjectives of the RoboCup Initiative

Joint International Project: (Distributed) Artificial Intelligence

Intelligent Robotics

Soccer – Central Research Topic:Very complex collective game

Huge amount of technologies involved:Autonomous Agents, Multi-Agent Systems, Cooperation, Communication, Robotics, Sensor Fusion, Real-Time Reasoning, Machine Learning, etc



RoboCup Objectives of the RoboCup InitiativeMain Goal of the RoboCup Initiative:

“By 2050, develop a team of fully autonomous humanoid robots that may win against the human world champion team in soccer!”



RoboCup RoboSoccer – Global Perspective

Soccer LeaguesSimulation – SoccerServer (2D, 3D, Coach)Simulation New: PV-League, Nanogram, Microsoft Robotics Soccer Challenge Robots Small-SizeRobots Medium-SizeLegged Robots (Aibo Dogs - Sony)Humanoid Robots

RoboCup RescueSimulation, Virtual, Robotic

RoboCup JúniorRoboCup @ Home



RoboCup The Complexity of RoboSoccer

DistributedCentralizedControl

Non SymbolicSymbolicSensor Reading

IncompleteCompleteWorld State Accessibility

Real-TimeIn TurnsState Change

DynamicStaticEnvironment

RoboSoccerChessComparison with Chess



RoboCup The RoboCup Federation

President: Minoru Asada (Japan) -> Manuela Veloso (USA)

Trustees

Executive Committee

Technical Committees for each League

Maintenance Committees

Local Federations/Committees



RoboCup RoboCup Official Competitions

RoboCup 1997 – Nagoya (Japan)RoboCup 1998 – Paris (France)RoboCup 1999 – Stockholm (Sweden)RoboCup 2000 – Melbourne (Australia)RoboCup 2001 – Seattle (USA)RoboCup 2002 – Fukuoka (Japan)RoboCup 2003 – Padua (Italy)RoboCup 2004 – Lisbon (Portugal)RoboCup 2005 – Osaka (Japan)RoboCup 2006 – Bremen (Germany)RoboCup 2007 – Atlanta (USA)Local Championships:

German Open (European), Japanese Open, Australian Open, American Open, Portuguese Open, Iranian Open, China Open!



RoboCup - Participants

Awarded Countries:Japan

Germany

USA

Australia

China

Iran

Portugal

Holland



RoboCup 2002 – Fukuoka Dome



RoboCup 2002 – Fukuoka Dome

1500 Researchers and over 150000 spectators



Presentation Structure RoboCup Leagues

IntroductionRoboCup InitiativeRoboCup Leagues

Simulation League (2D, 3D, Coach, PV, MRSC, Nanogram)Small-Size LeagueMiddle-Size LeagueLegged LeagueHumanoid LeagueRescue Leagues

Simulator – Soccer ServerFC Portugal ProjectCoordination Methodologies ResultsResearch Areas and Conclusions



RoboCup Leagues Simulation League 2D

Virtual Robots (software agents)

105*68m Virtual Field

Agents controlled by different computers (or processes)

Simulator sends perception and receives actions from agents

Teams of 11 players plus a coach



RoboCup Leagues Simulation League




Most Significant Teams:FC Portugal (Portugal) – Univ. of Porto and Aveiro

Tactics, Formations, RolesFlexible Teamwork, Configurable StrategyStrategic vs Active BehaviorSBSP - Situation Based Strategic PositioningDPRE - Dynamic Positioning and Role ExchangeADVCOM and SLM - Strategic Looking MechanismVisual DebuggerCoach UnilangRelease of FCPAgent Source Code

UVA Trilearn (Holland)Coordination GraphsRelease of UVA Source Code




Most Significant Teams:CMUnited (USA) - Carnegie Mellon University

Formations, SPAR and Roles

Layered Learning

Release of the low-level Source Code

Karlsruhe Brainstormers (Germany) - Univ. KarlsruheLearning of Skills - Reinforcement Learning

Tsinghuaeolus (China) - Tsinghua UniversityUse of Windows OS, Windows Monitor

Individual Marking, Dribbling Skills

Wright Eagle (China)



RoboCup Leagues Simulation 3D League

Started in 2004

Objectives of 3D Simulation:Replace the 2D environment of previous simulator with a 3D environment

New, more realistic, physics model

Simulation results should not be dependent on available computational power or on the quality of network resources




Complexities fromreal robots

Third dimensionadds complexity

Future evolution to legged andhumanoid robots




Humanoids startingin 2007

Very realistic physics

2 vs 2 games



RoboCup Leagues Microsoft Robotics Soccer Challenge



RoboCup Leagues Physical Visualization League

two step motors (1), a li-ion battery (2), a control board (8bit PIC18 processor) (3), Infrared Sensor (4) to receive its commands and aluminium body (5)

Augmented reality setupSmall real robots, Eco-Bes, play soccer on top of a virtual field with a virtual ball

Several research challenges included:

Vision Based Self Localization, Data Fusion, Real-Time Control, Decision and Cooperation



RoboCup Leagues Physical Visualization League


Camera is calibrated to recognize the field and the robots’ markers.

Each robot identified by an individual marker recognized by the vision system

Server application responsible to control socket connections with clients, monitor, camera and communication with infrared USB transmitter.

Server, compiles all information and, sends relative positions of all elements in the field

Monitor application uses the same information to draw the field and project it and shows the virtual ball


RoboCup Leagues Nanogram Competition


Microscopic robots that compete against each other in soccer-related agility challenges

Robots measure a few tens of micrometers to a few hundred micrometers

Masses ranging from a few nanograms to a few hundred nanograms

Playing field is a set of insulated interdigitatedelectrodes, across which an AC waveform can be applied


RoboCup Leagues Nanogram Competition

Challenges:2 Millimeter Dash

Slalom Drill

Ball-Handling Drill



RoboCup Leagues Small-Size League

15cm Robots

Field of 4*3m table

Robots controlled by radio by a single remote computer

Vision given by a camera placed on top of the field

Teams of 5 Robots







Most Significant TeamsCornell (USA) - Cornell University

Omni directional drive

Dribbling mechanism

Trajectory generation

Cooperation mechanisms: Roles, Passes

Fu-Fighters (Germany) - University of BerlinPowerful kicking device

Heterogeneous robots

Hierarchical generation of reactive behaviors




Lucky Star (Singapore) - Ngee Ann PolytechnicStable hardware platform, Good Basic skillsSimple cooperation mechanisms

5DPO (Portugal) - Faculty of Engineering of Univ. PortoModular Flexible SoftwareCooperation mechanisms (roles, tasks and actions)Vision (2 cameras)Tracking using circular bar codesInfrared (IR) communication system

CMU – Carnegie Mellon UniversityPowerful KickerVery good Control

Local Vision Teams!



RoboCup Leagues Middle-Size League

80 cm Robots

5x10m Field

Autonomous Robots (local vision and decision)

Teams of 4/5 Robots




Most Significant TeamsBrainstormers Tribots

Learning of SkillsPerfect Robot controlWorld and European Champions

CS-Freiburg (Germany) - University of Freiburg Multi-Agent Coordination: Role assignment, positioningRich set of Basic Skills, Behavior NetworksLaser range finder for Self Localization

Sharif CE (Iran) - Sharif UniversityOmnidirectional Castor Wheel, Robust platform, Simple vision




Most Significant TeamsGOLEM Team (Italy) - University of Padua

Omni directional drive and omni directional visionFlexible Roles: Homogeneous Robots

ART - Azzurra Team (Italy) - Univ. Parma, Milano, Padua, Genova and Rome

Heterogeneous robots from different universitiesCooperative behavior: Passes

Portuguese Teams: IsocRob5DPOMinho Team ISEPortoCambada



RoboCup Leagues Results League

30cm Robots

Field of 5*4m

Autonomous Robots

Teams of 4 Robots – Sony AIBO dogs

Emphasis on Computer Vision and Legged motion



RoboCup Leagues Results League



RoboCup Leagues Four Legged League



RoboCup Leagues Legged Robots League

Most Significant TeamsGerman Team – Germany

4 German UniversitiesCoordinationLegged robots Simulation

UNSW (Australia) - University of New South WalesBall control and fast locomotion (supported on elbows)Low-level skillsTeammate recognition and LocalizationSimple high-level strategy

CMPack (USA) - Carnegie Mellon UniversityEmphasis on visionSensor Resetting Localization



RoboCup Leagues Humanoid League

Penalties / Challenges2 vs. 2 games (two sizes)



RoboCup Leagues Humanoid League



RoboCup Rescue is an expansion of RoboCup soccer competitions aiming at researching in socially useful areas

RoboCup RescueIntroduction | RoboCup | Rob.Leagues | Simulator | FCPortugal | Coordination Meth. | Results | Conclusions


Rescue Simulation

Agents: Police Force

Ambulances

FireFighters

Centers

Objectives:Save Buildings

Rescue Civilians



RoboCup RescueOne of the 3 major research lines in RoboCup initiative

Main Challenge: Development of an heterogeneous team capable of saving people and limiting damage in a city after a major earthquake

Rescue Simulation league challenges: Urban virtual scenario, partially unknown, dynamic environment

Global strategy for a team, coordination

Partial hierarchical, low bandwidth communication

Heterogeneous agents in MAS system

Prediction of fire spreading and civilians health



RoboCup RescueSimulated Environment

City Virtual Model (Roads, Buildings and Agents)

Simulation of 72 hours after a major natural catastrophe

Emergency and security forces are mobilized in order to minimize life losses and material damages



Partial 3D View of Kobe City (Japan)

Partial 2D View of Foligno City (Italy)

RoboCup RescueVisualization



RoboCup RescueSimulated Environment

Agent Types:Fire Brigades, Police Forces and Ambulances

Command Centers (superior hierarchical level)

Simulator:Perception/Decision/Communication/Action

Fire Spreading and extinguishing

Building Collapses

Civilian Movement and health evolution

Traffic jams



Division into several simulation sub-modules

Simulation in discrete steps

Permanent changes, development and growing

RoboCup Rescue Simulated Environment



RoboCup RescueSimulated World Objects

Building with increasing level of damage (1 to 4)

Roads with obstructions: (1) Partial (2) Total

Refuge

Refuge (2D and 3D view)

a) Healthy

b) hounded

c) Dead

Civilians with different health levels



RoboCup RescueAmbulances

Ambulance rescuing a civilian

Buried Civilian Ambulances

Healthy Civilian

Function:

• Rescue civilians and take them to the refuges

Associated Challenges:• Rescue scheduling

• Buried agents life time prediction

• Ambulance coordination

• Coordination with agents of different type



Buildings in fire

Fire Brigades

Building being extinguished

Spring of water

Function:

• Extinguish fires and control its expansion

Associated Challenges:• Choose the best region to extinguish

• Choose the best building (preemptive?, direct?)

• Anticipate fire spreading (buildings and human lives)

• Collective and individual management of water in tanks

• Fire Brigades coordination

•Coordination with agents of different type

RoboCup RescueFire Brigades



Policeman approaching new

obstruction

Policeman clearing a road

Obstructed Roads

Function:

• Road clearance

Associated Challenges:

• Scheduling of the roads to clear considering: trapped emergency vehicles, principal routes, paths to refuges, fires and

trapped civilians

• Coordination between police forces

• Coordination with agents of different type

RoboCup RescuePolice Forces



RoboCup RescueCenter Agents and Communication

Function:

• Interface between different agent types

• Agent Coordination

Associated Challenges:

• Use global vision to improve high-level decisions

• Coordination of different agent types

• Communication limitation management (low-bandwidth)



RoboCup RescueRobot’s Perception and Action

Robot’s Sensorial information:

Visual

Aural

Radio receptionRobot’s Agent actions:

Move

Speak (voice and radio)

Specific agent type actionsExtinguish fire

Clear road

Unbury civilian



RoboCup RescueExample of a Simulation



Presentation Structure Simulator – Soccer Server

IntroductionRoboCup InitiativeRoboCup LeaguesSimulator – Soccer Server

Global PerspectiveThe Soccer MonitorThe Game, Field and ObjectsPerception and Action of the AgentsAgent ConstructionSimulation Major Difficulties

FC Portugal ProjectCoordination MethodologiesResultsResearch Areas and Conclusions



How the Simulator Works?Client-Server System

Agents (player’s brains) control a single player:

UDP sockets/Linux

Server:Receives agent commands

Simulates the movement of objects

Sends perceptions to the agents

Two teams with 11 players try to score goals!

Server Architecture

Simulator - Soccer ServerGlobal Perspective



Simulation CharacteristicsReal-Time - Human

Distributed – 24 Processes

Inaccessible (hidden), Continuous and Dynamic World

Errors in: Perception, Movement e Action

Limited Resources: Energy and Recovery

Limited Communication

Multi-Objective, Cooperative and Adverse Environment

Simulator - Soccer ServerGlobal Perspective



Soccer MonitorLinux or Windows

Simulator - Soccer ServerSoccerMonitor for Linux and Windows



2D SimulationField of 105*68m

22 Players and 1 BallCircles

Marking Flags and Lines

Automatic RefereeRules Reinforcement

Online CoachInstructions (stopped game)

Human RefereeComplex situations

Simulator - Soccer ServerThe Game, Field and Objects



Visual PerceptionLimited vision (angle and distance) with errors

Aural PerceptionLimited bandwidth communication

Physical Perceptionenergy, recovery, speed, …

----------Physical

Adversaries MessagesLimited DistanceAural

ErrorsLimited Distance/AngleVisual

NoiseSpatial LimitationsType

Simulator - Soccer ServerAgent’s Perceptions – Global Perspective



Agent Vision

Simulator - Soccer ServerAgent’s Perceptions - Visual



Agent Aural Perception (communication characteristics):

Not reliable!

No guaranty that the message is send or heard!

22 Agents (including opponents) use the same channel!

Limited communication capacity and range

No information about the sender!

Simulator - Soccer ServerAgent’s Perceptions - Aural



Agent’s ActionsTurn (Angle) - Rotate

Dash (Power) - Accelerate

Kick (Power, Angle) - Kick the Ball

Catch (Angle) - Catch the Ball (goal keeper)

Turn_Neck (Angle) - Rotate Neck

ChangeView (VType) - Vision Control

Say (Message) - Speak

Tackle (Power) - Tackle Opponent

PointTo (Dist,Angle) - Point to a Point

AttentionTo (Unum) - Give attention to Teammate

Simulator - Soccer ServerAgent’s Actions



How to Build Basic Agents for RoboSoccer?

Perception

World State Update

High-Level Decision

Action Execution

PerceptionInterpretation

ActionExecutionState of the

World

World - Soccer Server

Perception Action Soccer Monitor

AGENT

ENVIRONMENT

Simulator - Soccer ServerAgent Construction



Simulator - Soccer ServerAgent Construction

Main Questions:What Info in the World State?

How to Update the World State?

How to Build Complex Actions?(dribble, pass, move, mark, intercept, etc.)

How to use the Limited Resources (Energy)?

How to Create Collective Actions?

How to Really Create a Team? (defending/attacking, tactics, formations, player behaviors, etc.)




IntroductionRoboCup InitiativeRoboCup LeaguesThe Simulator – Soccer ServerFC Portugal Project

Global Vision of the ProjectAgent Architecture and World StateLow-Level SkillsActive Behavior with/without BallAssociated Tools

Coordination MethodologiesResultsResearch Areas and Conclusions



FC Portugal ProjectOverview

Project Leaders: Luis Paulo Reis (LIACC/FEUP) and Nuno Lau (IEETA/UA)

Approach: Multi-Agent System with Simple Coordination Techniques

CMU99 Low-level source code used as a starting pointResults:

European Champion - 2000 in Amsterdam (86-0)!World Champion – 2000 in Melbourne (94-0)!European Champion - 2001 in Paderborn (56-4)!3rd World Championship - 2001 in Seattle! (150-5)!Coach Champions 2002 – Fukuoka, JapanCoach 2nd Place – Padova, Italy and Lisbon, Portugal (2003,2004)Euro Champions (3D league) - 2006 in Eindhoven! (17-0)!World Champions (3D league) - 2006 in Bremen! (78-0)!



FC Portugal ProjectAgent’s Architecture

TeamStrategy

ADVCOMStrategies

PerceptionInterpretation

ActionPrediction

Action

Tactics

World State

ActionSelection Info

World - Soccer Server

Action

Soccer Monitor

Human Coach

Player TypesSLM

Strategies

High-LevelDecisionModule

Opp. ModStrategy

Formations

Situations

Situation Info

Game InfoUpdateMulti-LevelWorld State

Perception

Multi-Level World State



FC Portugal ProjectLow-Level Skills – Overview

Low-Level Skills:Movement in the Field (with obstacle avoidance)Ball InterceptionStop and Hold the BallKick the BallDribble

Innovation:Optimization Kick – A kick based on online optimization techniques!

Interception

Kick



FC Portugal Project

Decision Module Control Flux

STRATEGY – Strategic Analysis

DPRE – Positioning Exchange

ADVCOM – Intelligent Communication

SLM –Intelligent perception

SBSP – Strategic Positioning

Decisions: stopped game, with and without ball

Execution and prediction of Actions

World State and Situation Update

Initialization

DPRE

ADVCOM

SBSP

STRATEGY

CriticalSituation?

Ball Posses-sion Decision

NoYes

YesStoppedGame?

Stopped GameDecision

ActionExecution

World StateUpdate

No

SLM

Has Ball?Yes

Ball RecoveryDecision

No



FC Portugal Project Active Behavior with Ball - Overview

Ball Possession Actions:Shoot to the Goal

Pass (rapidly) the Ball

Forward the Ball (to a given point)

Dribble with the Ball

Hold the Ball

Decision Matrixes used to Select the Best Action!



FC Portugal ProjectActive Behavior with Ball - Passing

Pass Evaluation:Positional Value

Pass Cone

Initial Conditions

Final Congestion

Out Probability

Distance

Position Confidence

Reception Confidence

Reception Type

G.K. Interception

Shoot Possibility



FC Portugal Project Tools and Debugging

Debugging methodology principles:Offline debuggingVisual debuggingSuperimposed real environment and agent physical knowledgeFeature-focused debuggingInformation structured in layers of abstraction with different detail levels

Development tools implemented:Visual Debugger used to analyze the reasoning of agentsTeam Designer for graphical definition of soccer strategiesOffline client methodology3D Visualization Tools with Intelligent Camera ControlWstateMetrics that evaluates the accuracy of world states



FC Portugal Project Visual Debugger

Testing and tuning of the team

Shows what the agents see, hear, feel, think and do!



Project FC Portugal3D Viewer



Presentation Outline IntroductionRoboCup InitiativeRoboCup LeaguesThe Simulator – Soccer ServerFC Portugal ProjectCoordination Methodologies

Strategical CoordinationSBSP – Situation Based Strategic PositioningDPRE – Dynamic Positioning and Role ExchangeADVCOM – Advanced CommunicationSLM – Strategic Looking MechanismMM – Mutual ModelingCoaching – Coach Unilang

ResultsResearch Areas and Conclusions



Coordination MethodologiesFormalization of the Team Strategy



Coordination Methodologies SBSP - Situation Based Strategic Positioning

Strategic Situation: SBSP – Strategic Positioning

Active Situation (with/without Ball): Active Behavior



Coordination MethodologiesSBSP - Situation Based Strategic Positioning





Coordination MethodologiesSBSP - Situation Based Strategic PositioningALGORITHM SituationBasedStrategicPositioning(Tactic, Situation, Player) RETURNS PositionPARAMETERS Tactic, Situation, Player{

Formation = SituationFormation(Tactic, Situation)FormHeight = SituationFormationHeight(Tactic, Situation)FormWidth = SituationFormationWidth(Tactic, Situation)Positioning = PlayerGetPositioning(Player)Position = HomeSBSPPosition(Formation, FormWidth, FormHeight, Positioning)Role = FormationGetRole(Formation, Positioning)RoleStrategic = RoleGetStrategicRole(Role)BallAdjPos = AdjustedBallPosition(BallPosition)Position = BasicSBSPPosition(Position, Formation, RoleStrategic, Positioning,

BallAdjPos)Position = RegionalAdjustSBSPPosition(Position, Formation, RoleStrategic,

Positioning, BallAdjPos)Position = LegalAdjustSBSPPosition(Position, BallAdjPos)Position = DomainAdjustSBSPPosition(Position, BallAdjPos)RETURN Position

}



Coordination Methodologies DPRE - Dynamic Positioning and Role Exchange

Dynamic Exchange of Positionings and Behaviors based on utility:

Distances from players positions to its strategic positions

Positioning importance and adequacy of agents

DPRE improves the robotic team collective performance

Important against opponents with similar collective capabilities





Coordination Methodologies DPRE - Dynamic Positioning and Role ExchangeALGORITHM DynamicPositioningExchange(WorldState, Situation, Positionings)RETURNS Positionings(TeamSize)PARAMETERS WorldState, Positionings[TeamSize], Situation{FOR PL1 = 2 TO TeamSize-1 DOFOR PL2 = PL1+1 TO TeamSize DO IF PositionValid(PL1) AND PositionValid(PL2) THEN {Dist11 = Distance(Position(Pl1),SBSPPosition(Pl1))Dist22 = Distance(Position(Pl2),SBSPPosition(Pl2))Dist12 = Distance(Position(Pl1),SBSPPosition(Pl2))Dist21 = Distance(Position(Pl2),SBSPPosition(Pl1))Adeq11 = PosAdequacy(Pl1, Positioning[Pl1])Adeq22 = PosAdequacy(Pl2, Positioning[Pl2]) Adeq12 = PosAdequacy(Pl1, Positioning[Pl2]) Adeq21 = PosAdequacy(Pl2, Positioning[Pl1]) Util = ExchangePositions(DPREMode, Situation, Dist11, Dist22, Dist12, Dist21,

Adeq11, Adeq22, Adeq12, Adeq21, PosImportance(Positioning[Pl1]),PosImportance(Positioning([Pl2])

IF Util > ThresUtil(Situation) THEN exchange(Positionings[Pl1], Positionings[Pl2]) }RETURN Positionings

}



Coordination Methodologies ADVCOM –Intelligent Communication

Challenges: What and When to communicate?

What to Communicate?Individual World State – to improve world state accuracy

Useful Events – for coordination

When to Communicate?Communication utility is very high or is greater then communication utility (modeled) of teammates!

Creation of a Communicated World State!



Coordination Methodologies SLM – Strategic Looking Mechanism

Motivation:Intelligent Sensor utilization!

Different World State object’s importance

Look always to the ball? No…

What is it?Looking Direction decided based on the Expected Utility to look on that direction!

Possible looking direction with greater utility is chosen!



Coordination Methodologies MM - Mutual Modeling

Motivation and principle:Use Teammate and Opponent Models!

Estimate their positions and actions!

Fuse this information with visual, sensorial, aural information

Enables more accurate world states

Very useful when vision and sensors is not of good quality



Coordination Methodologies Online Coach and Game Statistics

Game Statistics and Opponent Modeling InformationTime and ResultIndividual Action with/without ballBall losses and Ball recoveriesAttacks and AssistancesBall PossessionBy:

PeriodRegion (from and to)TeamPlayeretc.

Coach

Assistant Coach

Players

Gam

e S

tatis

tics

Definitions

Opp

onen

t M

odel

ing

Field Information Field

Act

ions

Instructions

Principal Coach



Coordination MethodologiesCoach Unilang – Language to Coach a (Robo)Soccer Team

Base Concepts: Time Periods, Regions, Tactics, Formations, Situations, Player Types

Language Defined in BNF

Examples:<MESSAGE> ::= (<TIME> <ID> <MESSAGE PART> {<MESSAGE PART>})

<MESSAGE PART> ::= <DEFINITION_MESS> |<STATISTICS_MESS> | <OPP_MOD_MESS> | <INSTRUCTION_MESS>

TACTIC_DEFINITION> ::= <TEAM_MENTALITY> <GAME_PACE> <TEAM_PRESSURE> <FIELD_USE> <PLAYING_STYLE> <RISK_TAKEN> <OFFSIDE_TACTIC> <POSITIONING_EXCHANGE_USE> <FORMATIONS_USED>

<FORMATION> ::= <PREDEFINDED_FORMATION> <FORMATION_NAME> | <FORMATION_DEFINITION>

<PREDEFINED_FORMATION> ::= 433 | 433att | 442 | 343 | 4123 | 352 …

<FORMATION_DEFINITION>::= {(<PLAYER> <POS_NUMBER> <PLAYER_POSITIONING> <PLAYER_TYPE>

<PLAYER_POSITIONING> ::= <VERTICAL_POSITIONING> <HORIZONTAL_POSITIONING>




IntroductionRoboCup InitiativeRoboCup LeaguesSimulator – Soccer ServerFC Portugal ProjectCoordination MethodologiesResults

Results in Controlled Experiments Results in Official Competitions

Research Areas and Conclusions



Results – Competition and Controlled Experimental Results

Competition Results

Controlled Experiments Results

Competition Local Goals Shoots Scor. Chances Atack Midle Defense

Euro 2000 Amsterdam 87 - 0 136 - 7 194 - 16 53% 35% 12%

RoboCup 2000 Melbourne 94 - 0 144 - 3 201 - 7 59% 33% 8%

German Open 2001 Paderborn 56 - 4 79 - 17 128 - 25 37% 44% 19%

RoboCup 2001 Seattle 150 - 5 195 - 8 281 - 14 60% 32% 8%

Teams GM GS RM RS OM OS Atack Middle Def. Vict. Draw Def.

Very Good 2 0,4 3,1 1 9,8 2,9 40,0% 42,0% 18,0% 70% 20% 10%

Good 8,5 0,1 12,5 0,5 17,1 1,4 56,0% 32,5% 11,5% 100% 0% 0%

Average 21,1 0 27 0 36,7 0,1 74,7% 23,6% 1,7% 100% 0% 0%

ResultsGoals Shoots Scoring Chances Ball Possession




With SBSP – Situation Based Strategic Positioning

Without SBSP (using SPAR – Atractions and Repulsions)

Normal

Teams GM GS RM RS AT MF DEF V E DVery Good 2 0,4 3,1 1 40,0% 42,0% 18,0% 70% 20% 10%

Good 8,5 0,1 12,5 0,5 56,0% 32,5% 11,5% 100% 0% 0%Average 21,1 0 27 0 74,7% 23,6% 1,7% 100% 0% 0%

Goals Shoots Ball Possession Results

SPAR

Teams GM GS RM RS AT MF DEF V E DVery Good 0,1 3,0 0,7 5,2 14,0% 49,0% 37,0% 3% 13% 83%

Good 0,7 2,0 1,1 4,4 20,5% 43,0% 36,5% 10% 20% 70%Average 2,8 0,2 4,4 0,9 45,7% 38,7% 15,7% 80% 17% 3%





With DPRE – Dynamic Positioning and Role Exchange

Without DPRE – Dynamic Positioning and Role Exchange

Normal


Good 8,5 0,1 12,5 0,5 56,0% 32,5% 11,5% 100% 0% 0%Average 21,1 0 27 0 74,7% 23,6% 1,7% 100% 0% 0%


NO DPRE

Teams GM GS RM RS AT MF DEF V E DVery Good 1,4 0,8 2,3 1,4 33,7% 41,7% 24,0% 47% 30% 23%

Good 7,1 0,3 9,9 0,9 50,0% 33,5% 15,5% 95% 0% 5%Average 17,9 0,0 21,5 0,0 69,7% 27,3% 3,0% 100% 0% 0%





With ADVCOM, SLM and MM

Without ADVCOM, SLM and MM

Normal


Good 8,5 0,1 12,5 0,5 56,0% 32,5% 11,5% 100% 0% 0%Average 21,1 0 27 0 74,7% 23,6% 1,7% 100% 0% 0%


No AdvCom

No SLM/MM GM GS RM RS AT MF DEF V E DVery Good 0,4 2,4 0,8 3,8 14,0% 43,0% 43,0% 10% 33% 57%

Good 1,8 1,0 3,7 2,0 24,0% 42,5% 33,5% 70% 20% 10%Average 11,4 0,0 14,9 1,1 38,3% 47,0% 14,7% 100% 0% 0%




Coordination Methodologies Results: European 2000 - Amsterdam

First Stage - Group A

FC Portugal 3 - 0 Essex Wizards (England)FC Portugal 13 - 0 Lucky Luebeck (Germany)FC Portugal 4 - 0 Cyberoos (Australia) FC Portugal 22 - 0 Pizza Tower (Italy) FC Portugal 19 - 0 Polytech (Russia) FC Portugal 6 - 0 PSI (Russia)

Quarters and Semi-Finals

FC Portugal 13 - 0 Wroclaw (Poland) FC Portugal 5 - 0 Essex Wizards (England)

Final

FC Portugal 2 - 0 Karlsruhe Brainstormers (Germany)



Coordination Methodologies Results: RoboCup 2000 - Melbourne

First Stage - Group E

FC Portugal 33 – 0 Oulu (Finland) FC Portugal 18 – 0 Zeng (Japan) FC Portugal 20 – 0 Robolog (Germany)

Second Stage

FC Portugal 7 – 0 Essex Wizards (England)FC Portugal 3 – 0 Karlsruhe Brainstormers (Germany)FC Portugal 6 – 0 YowAI (Japan)

Semi-Finals

FC Portugal 6 – 0 ATT-CMU (USA)Final

FC Portugal 1 – 0 Karlsruhe Brainstormers (Germany)



Coordination Methodologies Results: European 2001 - Paderborn

First Stage - Group A

FC Portugal 2 – 0 Sharif Arvand (Iran) FC Portugal 3 – 1 Aras (Iran) FC Portugal 26 – 0 Osnabruck (Germany) FC Portugal 5 – 2 UvaTrilearn (FC Portugal 3 – 1 Lucky Luebeck (Germany)

Second Stage

FC Portugal 6 – 0 Dr Web (Russia)FC Portugal 9 – 0 Robolog (Germany)FC Portugal 1 – 0 Karlsruhe Brainstormers (Germany)

Final

FC Portugal 1 – 0 Karlsruhe Brainstormers (Germany)



Coordination Methodologies Results: RoboCup 2001 - Seattle

First Stage – Group AFC Portugal 29 – 0 11Monkeys (Japan) FC Portugal 9 – 0 TUT-Grove (Japan) FC Portugal 32 – 0 RMIT Goannas (Australia) FC Portugal 8 – 0 Robolog (Germany)

Second Stage – Group CFC Portugal 5 – 0 Helli Respina (Iran)FC Portugal 16 – 0 UtUtd (Iran)FC Portugal 4 – 0 FC Tripletta (Japan)FC Portugal 13 – 0 AT Humboldt (Germany) FC Portugal 22 – 0 ATTUnited (USA)

Final StageFC Portugal 8 – 0 YowAI (Japan)FC Portugal 4 – 1 UvaTrilearn (Netherlands)FC Portugal 0 – 3 Tsinghuaeolos (China)FC Portugal 0 – 1 Karlsruhe Brainstormers (Germany)



Results

FC Portugal -TsinghuAeolus (China) 5:0

FC Portugal - AllemaniACs3D (Germany) 5:0

FC Portugal - MRL (Iran) 2:0

FC Portugal - RoboLog3D (Germany) 5:0

FC Portugal - Aria (Iran) 4:0

FC Portugal - MainzRollingBrains (Germany) 8:0

FC Portugal - JU-TsubameGaeshi (Japan) 9:0

FC Portugal - WrightEagle (China) 3:0

FC Portugal - MainzRollingBrains (Germany) 8:0

FC Portugal - AmoiensisNQ (China) 5:0

FC Portugal - Arman (Iran) 1:0

FC Portugal - SEU (China) 0:0 FC Portugal - MRL (Iran) 3:0 FC Portugal - Rezvan (Iran) 1:0 FC Portugal - Brainstormers (Germany) 5:0 FC Portugal - Caspian (Iran) 1:0 FC Portugal - CZU2006 (China) 5:0 FC Portugal - Arman (Iran) 2:0 FC Portugal - Virtual Werder (Germany) 3:0 FC Portugal - Aria (Iran) 1:0

Final

FC Portugal - WrightEagle (China) 1:0

Coordination Methodologies Results: RoboCup 2006 – Bremen




IntroductionRoboCup InitiativeRoboCup LeaguesSimulator – Soccer ServerFC Portugal ProjectCoordination MethodologiesResultsResearch Areas and Conclusions

RoboCup Research TopicsApplications of RoboCup ResearchAdditional InformationConclusions



Research AreasRoboCup Research Topics

Coordination of Teams of Homogeneous / Heterogeneous Agents in Adversarial Environments: SBSP and DPREConcept of Strategy for a Competition against other Team with Opposite Goals - STRATEGYCommunication in MAS - ADVCOMIntelligent Perception – SLMOptimization Techniques – Optimization Kick, Smart DribbleLearning and Opponent Modeling in Adversarial Environments Soccer – Individual Decision



Research AreasApplications of Research

Real/Simulated Sport Competitions

RoboCup Rescue

War Scenarios

Mine Clearance and Land Exploration

Control of Hospital Robots

Public Transports Coordination

Satellite Control and Nuclear Weapon Management

Cleanup of Radioactive and Toxic Contamination

Implementation of AI Opponents for Simulation Games



Research AreasAdditional Information

Web Pages:

RoboCup Federation - www.robocup.org

Luis Paulo Reis – www.fe.up.pt/~lpreisFC Portugal - www.ieeta.pt/robocup ; www.fe.up.pt/~rescue

5DPO - www.fe.up.pt/~robosoc

IsocRob - http://socrob.isr.ist.utl.pt

Minho Team - www.robotica.dei.uminho.pt

ISEPorto Team - www.isep.ipp.pt/iseporto.html

Contacts:Luís Paulo Reis ([email protected])

A. P. Moreira ([email protected])

F. Ribeiro ([email protected])

José Almeida ([email protected])

Nuno Lau ([email protected])

Paulo Costa ([email protected])

Pedro Lima ([email protected])

José L. Azevedo ([email protected])



ConclusionsNew Projects in RoboCup

Common framework for cooperative robotics

Small and Middle-Size league (5DPO, Cambada) - Mixed Teams

RoboCup Rescue and other domains

Real Soccer Intelligent Game Analysis System

Present Simulators:

General Strategy for a Competition

General Coaching – Coach Unilang

Opponent Modeling and Game Analysis

Learning high-level skills

Heterogeneous Players

Extension of SBSP, DPRE, ADVCOM and SLM

3D Visualizer with Camera Intelligent control

Simulator Extensions and Changes – 3D Simulator



ConclusionsRoboCup in the Future

In the Future? Is it really possible, by 2050, to build a team of fully autonomous humanoid robots that may win against the human world champion team in soccer?

Major Difficulties?Robotic Sensors?

Robotic Actuators?

Artificial Brains?

Social Behavior?

RoboCup is not only about Competition!Universities and Companies!

Research applied in other areas


IR RoboticSoccer Coordinationlpreis/ir2007_08/documents/IR_RoboticSoccer... · Intelligent Robotics...

Documents

Transcript of IR RoboticSoccer Coordinationlpreis/ir2007_08/documents/IR_RoboticSoccer... · Intelligent Robotics...