First International Workshop on

Agent Technology for Disaster Management

Foreword

In the light of recent events throughout the world, ranging from natural disasters such as the Asian Tsunami and hurricane Katrina in New Orleans, to the man-made disasters such as the 7/7 terrorist attacks in London and 9/11 attacks in New York, the topic of disaster management (also known as emergency response) has become a key social and political concern. Moreover, the evidence from these and many other similar disasters is that there is also an overwhelming need for better information technology to help support their efficient and effective management. In particular, disaster management requires that a number of distinct actors and agencies, each with their own aims, objectives, and resources, be able to coordinate their efforts in a flexible way in order to prevent further problems or effectively manage the aftermath of a disaster. The techniques involved may necessitate both centralized and decentralized coordination mechanisms, which need to operate in large-scale environments, which are prone to uncertainty, ambiguity and incompleteness given the dynamic and evolving nature of disasters.

Against this background, we initiated this first international workshop on Agent Technology for Disaster Management (ATDM). Its aim is to help build the community of researchers working on applying multi-agent systems to disaster management, through either designing, modeling, implementing, or simulating agent-based disaster management systems. In this context, this collection consists of the papers accepted at the ATDM workshop. The collection is organized into four main sections, namely (i) Coordination Mechanisms (ii) Agent-based simulation: agent models and teamwork (iii) Agent-based simulation: tools and experiments (iv) Agent-based architectures and position papers, each of which aims to focus on particular issues arising in the theme common to all papers in each section. This collection represents the first contribution to support agent-based researchers in organising themselves to deal with this challenging, and high-impact field of disaster management.

Nicholas R. Jennings, Milind Tambe, Toru Ishida, Sarvapali D. Ramchurn

8th May 2006

Hakodate, Japan

i

Organising Committee Prof. Nicholas R. Jennings (University of Southampton, UK) Prof. Milind Tambe (University of Southern California, USA) Prof. Toru Ishida (Kyoto University, Japan) Dr. Sarvapali D. Ramchurn (University of Southampton, UK) Programme Committee

Prof. Austin Tate (AIAI, University of Edinburgh, UK)

Dr. Alessandro Farinelli (Università di Roma ''La Sapienza', Italy)

Dr. Frank Fiedrich (George Washington University, USA)

Dr. Alex Rogers (University of Southampton, UK)

Prof. H. Levent Akin (Bogaziçi University, Turkey)

Prof. Hitoshi Matsubara (Future University, Japan)

Dr. Itsuki Noda (AIST, Ibaraki, Japan)

Dr. Jeff Bradshaw (IHMC, USA)

Dr. Lin Padgham (RMIT, Australia)

Dr. Partha S. Dutta (University of Southampton, UK)

Dr. Paul Scerri (Robotics Institute, CMU, USA)

Dr. Ranjit Nair (Honeywell, USA)

Dr. Stephen Hailes (University College London, UK)

Prof. Victor Lesser (University of Massachusetts, USA)

Prof. Tomoichi Takahashi (Meijo University, Japan)

ii

Table of contents1 Section 1: Coordination Mechanisms

1

Gerhard Wickler, Austin Tate, and Stephen Potter Using the <I-N-C-A> constraint model as a shared representation of intentions for emergency response

2

Doran Chakraborty, Sabyasachi Saja, Sandip Sen, and Bradley Clement Negotiating assignment of disaster monitoring tasks

10

Joshua Reich and Elizabeth Sklar Toward automatic reconfiguration of robot-sensor networks for urban search and rescue

18

Jean Oh, Jie-Eun Hwang, and Stephen F. Smith Agent technologies for post-disaster urban planning

24

Alessandro Farinelli, Lucia Iocchi, and Daniele Nardi Point to point v/s broadcast communication for conflict resolution

32

Nathan Schurr, Pratik Patil, Fred Pighin, and Milind Tambe Lessons learnt from disaster management

40

Section 2: Agent-based Simulation (Agent Models & Teamwork) 48

Paulo R. Ferreira Jr. and Ana L. C. Bazzan Swarm-Gap: A swarm based approximation algorithm for E-GAP

49

Kathleen Keogh and Liz Sonenberg Agent teamwork and reorganisation: exploring self-awareness in dynamic situations

56

Hiroki Matsui, Kiyoshi Izumi, and Itsuki Noda Soft-restriction approach for traffic management under disaster rescue situations

64

Vengfai Raymond U and Nancy E. Reed Enhancing agent capabilities in a large rescue simulation system

71

Tomoichi Takahashi Requirements to agent-based disaster simulations from local government usages

78

Utku Tatlidede and H. Levent Akin Planning for bidding in single item auctions

85

1 Equivalent to workshop programme

iii

Section 3: Agent-based Simulation (Tools & Experiments) 91

Jijun Wang, Michael Lewis, and Paul Scerri Cooperating robots for search and rescue

92

Yohei Murakami and Toru Ishida Participatory simulation for designing evacuation protocols

100

Venkatesh Mysore, Giuseppe Narzisi, and Bud Mishra Agent modeling of a Sarin attack in Manhattan

108

Alexander Kleiner, Nils Behrens, and Holger Kenn Wearable computing meets multiagent systems: a real-world interface for the RobocupRescue simulation platform

116

Daniel Massaguer, Vidhya Balasubramanian, Sharad Mehrotra, and Nalini Venkatasubramanian Multi-agent simulation of disaster response

124

Magnus Boman, Asim Ghaffar, Fredrik Liljeros Social network visualisation as a contract tracing tool

131

Section 4: Agent-based Architectures and Position Papers 134

Yuu Nakajima, Hironori Shiina, Shohei Yamane, Hirofumi Yamaki, and Toru Ishida Protocol description and platform in massively multiagent simulation

135

J. Buford, G. Jakobson, L. Lewis, N. Parameswaran, and P. Ray D-AESOP: A simulation-aware BDI agent system for disaster situation management

143

Juan R. Velasco, Miguel A. López-Carmona, Marifeli Sedano, Mercedes Garijo, David Larrabeiti, and María Calderón Role of Multiagent system on mimimalist infrastructure for service provisioning in ad-hoc networks for emergencies

151

Márton Iványi, Lászlo Gulyás, and Richárd Szabó Agent-based simulation in disaster management

153

Dean Yergens, Tom Noseworthy, Douglas Hamilton, and Jörg Denzinger Agent based simulation combined with real-time remote surveillance for disaster response management

155

Nicholas R. Jennings, Sarvapali D. Ramchurn, Mair Allen-Williams, Rajdeep Dash, Partha Dutta, Alex Rogers, and Ioannis Vetsikas The ALADDIN Project: Agent technology to the rescue

157

iv

1

Section 1

Coordination Mechanisms

2

Using the <I-N-C-A> Constraint Model as a Shared Representation of Intentions for Emergency Response

Gerhard Wickler AIAI, University of Edinburgh

Edinburgh, Scotland, UK

g.wickler@ed.ac.uk

Austin Tate AIAI, University of Edinburgh

Edinburgh, Scotland, UK

a.tate@ed.ac.uk

Stephen Potter AIAI, University of Edinburgh

Edinburgh, Scotland, UK

s.potter@ed.ac.uk

ABSTRACT The aim of this paper is to describe the I-X system with its underlying representation: <I-N-C-A>. The latter can be seen as a description of an agent’s intentions, which can be shared and communicated amongst multiple I-X agents to coordinate activities in an emergency response scenario. In general, an <I-N-C-A> object describes the product of a synthesis task. In the multi-agent context it can be used to describe the intentions of an agent, although it also includes elements of beliefs about the world and goals to be achieved, thus showing a close relationship with the BDI agent model which we will explore in this paper. From a user’s perspective, I-X Process Panels can be used as an intelligent to-do list that assists emergency responders in applying pre-defined standard operating procedures in different types of emergencies. In particular, multiple instances of the I-X Process Panels can be used as a distributed system to coordinate the efforts of independent emergency responders as well as responders within the same organization. Furthermore, it can be used as an agent wrapper for other software systems such as web-services to integrate these into the emergency response team as virtual members. At the heart of I-X is a Hierarchical Task Network (HTN) planner that can be used to synthesize courses of action automatically or explore alternative options manually.

Categories and Subject Descriptors I.2.4 [Artificial Intelligence]: Knowledge Representation Formalisms and Methods – Representation languages; I.2.8 [Artificial Intelligence]: Problem Solving, Control Methods, and Search – Plan execution, formation, and generation; I.2.11 [Artificial Intelligence]: Distributed Artificial Intelligence – Multiagent systems.

General Terms Human Factors, Standardization, Languages, Theory.

Keywords HTN planning, agent capabilities and coordination, agent modelling.

1 INTRODUCTION There are a number of tools available that help people organize their work. One of these is provided with virtually every organizer, be it electronic or paper-based: the “to-do” list. This is because people are not very good at remembering long lists of potentially unrelated tasks. Writing these tasks down and ticking them off when they have been done is a simple means of ensuring

that everything that needs to be done does get done, or at least, that a quick overview of unaccomplished tasks is available. In responding to an emergency this is vital, and the larger the emergency is, the more tasks need to be managed. The I-X system provides the functionality of a to-do list and thus, it is a useful tool when it comes to organizing the response to an emergency. The idea of using a to-do list as a basis for a distributed task manager is not new [9]. However, I-X goes well beyond this metaphor and provides a number of useful extensions that facilitate the finding and adaptation of a complete and efficient course of action. The remainder of this paper is organized as follows: Firstly, we will describe the model underlying the whole system and approach: <I-N-C-A>. This is necessary for understanding the philosophy behind I-X Process Panels, the user interface that provides the intelligent to-do list. Next, we will describe how the intelligence in the to-do list part is achieved using a library of standard operating procedures, an approach based on Hierarchical Task Network (HTN) planning [14,20]. The HTN planning system built into I-X is seamlessly integrated into the system. I-X is not meant to only support single agents in responding to an emergency, but it also provides mechanisms for connecting a number of I-X Process Panels and supporting a coordinated multi-agent response. The key here is a simple agent capability model that automatically matches tasks to known capabilities for dealing with these tasks. Finally, we will discuss <I-N-C-A> as a generic artifact model for a synthesis task and show how its components relate the BDI model in the context of planning agents.

2 USING I-X PROCESS PANELS I-X Process Panels constitute the user interface to the I-X system. They more or less directly reflect the ontology underlying the whole I-X system, the <I-N-C-A> ontology [23], which is a generic description of a synthesis task, dividing it into four major components: Issues, Nodes, Constraints, and Annotations. Of these, nodes are the activities that need to be performed in a course of action, thus functioning as the intelligent to-do list. The other elements contain issues as questions remaining for a given course of action, information about the constraints involved and the current state of the world, and notes such as reports or the rationale behind items in the plan.

2.1 The <I-N-C-A> Ontology In <I-N-C-A>, both processes and process products are abstractly considered to be made up of a set of “Issues” which are associated with the processes or process products to represent potential requirements, questions raised as a result of analysis or critiquing,

3

etc. They also contain “Nodes” (activities in a process, or parts of a physical product) which may have parts called sub-nodes making up a hierarchical description of the process or product. The nodes are related by a set of detailed “Constraints” of various kinds. Finally there can be “Annotations” related to the processes or products, which provide rationale, information and other useful descriptions. <I-N-C-A> models are intended to support a number of different uses:

• for automatic and mixed-initiative generation and manipulation of plans and other synthesized artifacts and to act as an ontology to underpin such use;

• as a common basis for human and system communication about plans and other synthesized artifacts;

• as a target for principled and reliable acquisition of knowledge about synthesized artifacts such as plans, process models and process product information;

• to support formal reasoning about plans and other synthesized artifacts.

These cover both formal and practical requirements and encompass the requirements for use by both human and computer-based planning and design systems.

2.1.1 Issues The issues in the representation may give the outstanding questions to be handled and can represent decisions yet to be taken on objectives to be satisfied, ways in which to satisfy them, questions raised as a result of analysis, etc. Initially, an <I-N-C-A> artifact may just be described by a set of issues to be addressed (stating the requirements or objectives). The issues can be thought of as implying potential further nodes or constraints that may have to be added into the specification of the artifact in future in order to address the outstanding issues. In work on I-X until recently, the issues had a task or activity orientation to them, being mostly concerned with actionable items referring to the process underway – i.e., actions in the process space. This has caused confusion with uses of I-X for planning tasks, where activities also appear as “nodes”. This is now not felt to be appropriate, and as an experiment we are adopting the gIBIS orientation of expressing these issues as questions to be considered [15,3]. This is advocated by the Questions – Options – Criteria approach [10] – itself used for rationale capture for plans and plan schema libraries in earlier work [12] and similar to the mapping approaches used in Compendium [16].

2.1.2 Nodes The nodes in the specifications describe components that are to be included in the design. Nodes can themselves be artifacts that can have their own structure with sub-nodes and other <I-N-C-A> described refinements associated with them. The node constraints (which are of the form “include node”) in the <I-N-C-A> model set the space within which an artifact may be further constrained. The “I” (issues) and “C” constraints restrict the artifacts within that space which are of interest.

2.1.3 Constraints The constraints restrict the relationships between the nodes to describe only those artifacts within the design space that meet the objectives. The constraints may be split into “critical constraints”

and “auxiliary constraints” depending on whether some constraint managers (solvers) can return them as “maybe” answers to indicate that the constraint being added to the model is okay so long as other critical constraints are imposed by other constraint managers. The maybe answer is expressed as a disjunction of conjunctions of such critical or shared constraints. More details on the “yes/no/maybe” constraint management approach used in I-X and the earlier O-Plan systems are available in [21]. The choices of which constraints are considered critical and which are considered as auxiliary are decisions for an application of I-X and specific decisions on how to split the management of constraints within such an application. It is not pre-determined for all applications. A temporal activity-based planner would normally have object/variable constraints (equality and inequality of objects) and some temporal constraints (maybe just the simple before {time-point1, time-point-2} constraint) as the critical constraints. But, for example in a 3D design or a configuration application, object/variable and some other critical constraints (possibly spatial constraints) might be chosen. It depends on the nature of what is communicated between constraint managers in the application of the I-X architecture.

2.1.4 Annotations The annotations add additional human-centric information or design and decision rationale to the description of the artifact. This can be of assistance in making use of products such as designs or plans created using this approach by helping guide the choice of alternatives should changes be required.

2.2 I-X Process Panels: Intelligent To-Do Lists The user interface to the I-X system, the I-X Process Panel, shows four main parts that reflect the four components of the <I-N-C-A> ontology just described. They are labeled “Issues”, “Activities”, “State”, and “Annotations”, as shown in figure 1.

Figure 1. An I-X Process Panel, shown here addressing a

simulated oil spill incident. In the case of the artifact to be synthesized being a course of action, the nodes that will eventually make up the artifact are activities, and these play the central role in the view of an I-X panel as an intelligent to-do list. Users can add an informal

4

description of a task to be accomplished to the activities section of the panel where it will appear as the description of that activity. Each activity consists of four parts listed in the four columns of the activities part of the panel:

• Description: This can be an informal description of a task such as “do this” or it can be a more formal pattern consisting of an activity name (verb) followed by a list of parameters such as: (deploy ?team-type) where the words preceded by a question mark are variables that need to be bound before the task can be dealt with.

• Annotation: This can be used to add arbitrary pieces of information to a specific activity.

• Priority: This defines the priority of the activity. Possible values are Highest, High, Normal, Low, or Lowest.

• Action: This field contains a menu that gives the various options that are available to deal with the activity.

It is the last field that allows the user to mark the task as “Done”, which corresponds to ticking off an item in a to-do list. Other options that are always available are “No action”, the default value until the task has been dealt with, or “N/A” if the activity does not make sense and is “not applicable” in the current context. The entries in the action menu related to an activity are determined by the activity handlers. These are modules that can be plugged into the I-X system and define ways in which activities can be dealt with. If an activity handler matches an activity it can add one or more entries to the according action menu. The most commonly used activity handler in the context of HTN planning adds “Expand” items to this menu, and this is the point where the to-do list becomes intelligent. Instead of just being able to tick off an activity, users can use the knowledge in a library of standard operating procedures to break an activity down into sub-activities that, when all performed, accomplish the higher-level task. Of course, sub-activities can themselves be broken down further until a level of primitive actions is reached, at which point the library of procedures no longer contains any refinements that mach the activities. This mechanism supports the user in two ways:

• The library of standard operating procedures may contain a number of different refinements that all match the present activity. All of the applicable procedures are added to the action menu by the activity handler, thus giving the user a comprehensive and quick overview of all the known standard procedures available to deal with this task.

• When a refinement for an activity is chosen, the I-X Process Panel shows all the sub-activities as new items in the to-do list. This ensures that users do not forget to include sub-activities, a common problem especially for infrequently applied procedures.

Both of these problems become only more severe when the user is under time pressure and lives depend on the decisions taken. Note that the intelligence of the to-do list comes in through the underlying HTN planner that finds applicable refinements in the

library and, on demand, can complete a plan to perform a given task automatically, propagating all constraints as it does so. Equally important, however, is the knowledge contained in the library of standard operating procedures.

2.3 Other Features As activities are the nodes that make up a course of action, it is only natural that the activity part of the I-X Process Panel forms the centre of attention for our view of I-X as an intelligent to-do list. In fact, we have implemented a cut-down interface called Post-IX which only shows this part of the panel (and so provides a minimal or ‘entry level’ interface to the system). We shall now briefly describe the other parts of a panel and how they are used. World state constraints are used to describe the current state of the world. Essentially, these are a state-variable representation of the form “pattern = value” allowing the user to describe arbitrary features of the world state. They are displayed in the I-X Process Panel in the constraints section. However, it is not expected that users will find this list of facts about the world style representation very useful. Thus, I-X allows for the registration of world state viewers that can be plugged into the system. For example, BBN Openmap [11] has been used in a number of applications to provide a 2D world map with various features. Most importantly, it can be automatically synchronized with the world state constraints such that icons in the map always represent current positions of the entities they represent. Constraints are propagated and evaluated by constraint managers that are plugged into the I-X system. Issues can be seen as a meta to-do list: instead of listing items that need to be done to deal with an emergency in the real world, they list the questions or outstanding items that need to be dealt with to make the current course of action complete and consistent. Often, these will be flaws in the current plan, but they can also be opportunities that present themselves, or simply facts that need to be verified to ensure a plan is viable. Issues can be either formal, in which case registered issue handlers can be used to deal with them just like activity handlers deal with activities, or they can be informal. Annotations are used for arbitrary comments about the course of action as a whole, stored as “keyword = value” patterns.

3 STANDARD OPERATING PROCEDURES As outlined above, standard operating procedures describe the knowledge underlying the intelligent to-do list. The formalism is based on refinements used in HTN planning and will be explained next. However, users are not expected to learn this formalism, but they can use a domain editor and its graphical user interface to define the library of procedures.

3.1 Activity Refinements in HTN Planning What are known as standard operating procedures to domain experts are called methods in HTN planning [5]. Methods formally describe how a task can be broken down into sub-tasks. The definition of a method consists of four main parts:

• Task pattern: an expression describing the task that can be accomplished with this method;

5

• Name: the name of this method (there may be several for the same task);

• Constraints: a set of constraints (e.g. on the world state) that must hold for this method to be applicable; and

• Network: a description of the sub-tasks into which this method refines the given task.

The task pattern of a method is used for matching methods to items in the activity list. If the task pattern matches the activity the method will appear in the action menu of the activity in the panel as a possible expansion. This is also where the name of the method will be used: the menu displays an entry “Expand using <name>” where name is the name of the method. In this way, the user can easily distinguish the different options available. The constraints are used to decide whether the method is applicable in the current world state. If they are satisfied, the method can be selected in the action menu, otherwise the unsatisfied constraints can be seen as issues, namely sub-goals that need to be achieved in some way. Finally, the network contains the list of sub-tasks that will be added as activities to the panel when the method is selected. The ordering constraints between sub-tasks are used to show in the interface those sub-tasks that are ready for tackling at any given time.

Figure 2. The I-X Domain Editor, here shown modelling an oil

spill response standard operating procedure.

3.2 The I-X Domain Editor Figure 2 shows an example of the I-X Domain Editor for defining standard operating procedures. The panel on the left lists all the currently defined procedures by name, and the task pattern they match. One, called “Oil Spill Response (General)”, is shown being edited. There are a number of views available to edit a refinement. The one shown is the graphical view which shows all the direct sub-tasks with their begin and end time points. Arrows between these activities indicate temporal ordering constraints, for example, the activity “Control source of spill” cannot be started before “Ensure safety of public and response personnel” has been completed. However, the activities “Control source of spill” and “Manage coordinated response effort” can then be

performed in parallel. Other views show the conditions and effects that can be defined for refinements.

4 AGENT COORDINATION WITH MULTIPLE PANELS

So far we have described I-X as a tool for assisting a single person in organizing and executing the response to an emergency. However, I-X is also a tool that supports the coordination of the response of multiple agents. I-Space is a tool in which users can register the capabilities of other agents. These capabilities can then be used from an I-X panel through inter-panel communication. Augmented instant messaging can be used to directly communicate with other responders via their panels.

Figure 3. The I-Space Tool. The agents’ relations to each other governs the nature of interactions between them.

4.1 I-Space Every I-X panel can be connected to a number of other I-X agents. Each I-X agent represents an agent that can potentially contribute to the course of action taken to respond in an emergency. The I-Space holds the model of the other agents and can be managed with a simple tool as shown in figure 3. Associated with each agent are one or more communication strategies which define how messages can be sent to this agent. By default, a built-in communication strategy simply sends XML-formatted messages to a given IP-address and socket. Alternatively, a Jabber-strategy [7] is available for using a chat-based mechanism for communication. New communication strategies can be added to communicate with agents implemented using different frameworks. Usually users will not be concerned with the question of how communication takes place as long as the system can find a way, but more with the relationships between the different agents in the I-Space. Within an organization a hierarchical structure is common, so collaborating agents are usually either superiors or subordinates. They can also be modelled as peers, which is also how agents from other organizations can be described. If the agent to be integrated into the virtual organization is a software agent it is described as a (web-)service. Finally, a generic relation “contact” is available, but it does not specify what exactly the relationship to this agent is.

4.2 Agent Capabilities At present there is only a relatively simple capability model implemented in I-X. The idea behind this model is that activities

6

are described by verbs in natural language and thus, a task name can be used as a capability description. Parameter values are currently not used to evaluate a capability. Each agent is associated with a number of capabilities that can be called upon. In the future it will be possible to use a much more sophisticated model. The problem with more complex representations is often that matching capabilities to tasks can be computationally expensive, and when the number of known capabilities becomes large, this can be a problem, which is why the current model is so simple. On the other hand, capabilities can often only be distinguished by a detailed description. One approach to this trade-off is to provide a representation that is flexible, allowing for a more powerful representation where required, but retaining efficiency if the capability description is simple [24]. Conceptually, the description of a capability is similar to that of an action, which is not surprising as a capability is simply an action that can be performed by some agent. A capability description essentially consists of six components:

• Name: The name of a capability corresponds a the verb that expresses a human-understandable description of the capability.

• Inputs: These are the objects that are given as parameters to the capability. This may be information needed to perform the capability, such as the location of a person to be recovered, objects to be manipulated by the capability, such as paper to be used in a printing process, or resources needed to perform the capability.

• Outputs: These are objects created by the capability. Again, this can be information such as references to hospitals that may have been sought, or they can be new objects if the capability manufactures these.

• Input constraints: These are effectively preconditions, consisting of world state constraints that must be true in the state of the world just before the capability can be applied. Usually, they will consist of required relations between the inputs.

• Output constraints: These are similar to effects, consisting of world state constraints that are guaranteed to be satisfied immediately after the capability has been applied. Usually, they will consist of provided relations between the outputs.

• I-O constraints: These cross constraints link up the inputs with the outputs. For example, a prioritization capability might order a given list of options according to some set of criterions. A cross constraint, referring to both the situation before and after the capability has been applied is necessary to say that the given list of options and the prioritized list contain the same elements.

This capability model can be used to describe the abilities of real-world agents that ultimately must be deployed to do things, or for software agents that provide information that can be used to guide the activity in the physical world.

4.3 Handling Activities through Task Distribution

From a user’s perspective, task distribution is integrated into the user interface through the “action” menu in the activities part of

the panel as just another option available to deal with an activity. The agent relationship is used to determine in which way the activity can be passed to another agent, for example, if the other agent is a subordinate the activity can simply be delegated to the agent. The capability model is used to filter the options that are listed in the action menu. Currently there is the option of specifying no capabilities for an agent in which case the agent will always be listed. If there is a list of capabilities associated with an agent than these options will only be listed if there is an exact match of the verb capability.

4.4 Structured Instant Messaging Another tool that is widely used for the coordination of efforts in response to an emergency is instant messaging. Like a to-do list, it is very simple and intuitive, but it lacks the formal structure that is needed when the scale of the event that needs to be addressed increases. As for the to-do list, I-X builds on the concept of instant messaging, extending it with the <I-N-C-A> ontology, but also retaining the possibility of simple and informal messages. Thus, users can use structured messaging when this is appropriate, or continue to use unstructured messaging when this is felt to be more useful. The structured version can be activated by selecting a message type: issue, activity, constraint or annotation, rather than a simple chat message. An <I-N-C-A> object with the content of the message will then be created and sent to the receiving I-X agent. Since all messages between agents are <I-N-C-A> objects, the receiving agent will treat the instant messenger generated message just like any other message from an I-X panel, e.g. the message generated when a task is delegated to a subordinate agent. In this way, structured instant messaging can be seamlessly integrated into the I-X framework without loosing the advantages of informal communications.

5 I-X/<I-N-C-A> AND THE BDI MODEL The idea behind <I-N-C-A> is that it can be used as a generic representation for any synthesized artifact. The nodes are the components that make up the artifact and the constraints restrict the ways in which the components may be synthesized for the design to be successful, i.e. they give relations between the components of the artifact as well as objects in the environment, The issues are the questions that need to be answered before the design is complete and the annotations hold background information of any kind. In the context of planning nodes are actions that need to be synthesized, constraints restrict the way actions can be related to each other, e.g. using the before relation to define a partial order, or what needs to be true in the environment for a plan to be applicable, issues are the items that still need to be worked on before the plan achieves its objective, and annotations hold background information about the plan such as rationale or assumptions. Thus, the task of planning can be described as synthesizing an <I-N-C-A> object, namely a plan which is just an instance of a synthesized artifact. In classical AI planning, a plan is considered to be a solution for a given planning problem if it achieves a goal, i.e. if the performance of the actions in the plan makes the goal condition come true. Two of the properties that are often associated with intelligent agents, amongst others, are that they are situated and that they should exhibit a goal-directed behaviour [13,6]. By “situatedness”

7

we mean that an agent exists in and acts upon some environment. The agent may be able to sense the environment and therefore hold some beliefs about the state of its environment. A goal is a condition that an agent desires to hold in its world, and if it is not believed to be true already, the agent may be able to act towards achieving. The (goal-directed) behavior of an agent is made up of the actions it performs and their performance is not just by accident but because it intends to do these actions. Beliefs, desires and intentions are the three cognitive primitives that form the basis for the BDI model of agency [19]. At present, the BDI model is probably the most widely used formal model for describing agents. <I-N-C-A> is the model underlying the I-Plan planner in I-X that is based on decades of planning research. Despite the difference in origin, the two models are closely related and we shall now explore this relation in more detail, by comparing a BDI agent with an I-X agent. We model an I-X agent by its current (possibly partial) plan (an <I-N-C-A> object) and its world state constraints (as described on the I-X panel). We can relate this to the beliefs, desires and intentions of a BDI agent as described below. The task-oriented nature of I-X means that intentions naturally become most prominent, and it is with these that we begin.

5.1 Intentions Essentially, I-X agents are focused on intentions. In BDI intentions can be considered to be relationships between an agent and a (again, possibly partial) plan; in the I-X ‘world’ a plan is the principal <I-N-C-A> object. Specifically, the nodes in an <I-N-C-A> plan are the intended actions; the activity constraints in <I-N-C-A> arrange these actions into a plan; the world state constraints in <I-N-C-A> correspond to that subset of the BDI beliefs that must be held if the plan is to be applicable. <I-N-C-A> issues are related to desires as described below.

5.2 Beliefs Beliefs are relationships between agents and statements about the world. An I-X agent maintains only specific beliefs, namely: ‘facts’ about the world that are believed to be true, modeled as constraints in the panel; capability descriptions of other agents in the world; and beliefs about how activities affect the state of the world. Note that the task-centric view of I-X agents means that the knowledge of other agents cannot be easily represented.

5.3 Desires Desires are not explicitly represented in <I-N-C-A>, but we can say there is a function that can map a given set of BDI desires and an intended partial plan to a set of unresolved or outstanding issues. This means that, in a given context, we can take a BDI description and map it to an <I-N-C-A> object. Correspondingly, given a set of issues and a partial plan, we can derive a super-set of the agent's desires. Initially, when there are no activities then the set of issues correspond to the desires, and eventually, when the plan is complete (and hence, will fulfill the agent's desires), the set of issues will be empty. At any intermediate point, the set of issues will correspond to those desires that the current partial plan will not, as yet, fulfill. Annotations can be used to capture the relationship between satisfied desires and the elements of the plan that satisfy them.

5.4 Summary This shows that the I-X model of agency and the BDI model are quite similar in many respects. The main difference is rooted in the task-centric view taken by the I-X agent. The <I-N-C-A> model is more specific when it comes to representing plans and activities, but focuses on activity-related beliefs. While this is not a restriction imposed by the <I-N-C-A> model, it is so in the I-X architecture with its specific syntax for representing world state constraints. This is of course necessary to build practical planners for efficient problem solving in real world applications.

6 APPLICATIONS I-X has been applied to a number of application scenarios in the area of emergency response. In this section we survey some of the current applications.

6.1 Co-OPR Personnel recovery teams operate under intense pressure, and must take into account not only hard logistics, but "messy" factors such as the social or political implications of a decision. The Collaborative Operations for Personnel Recovery (Co-OPR) project has developed decision-support for sensemaking in such scenarios, seeking to exploit the complementary strengths of human and machine reasoning [2,22]. Co-OPR integrates the Compendium sensemaking-support tool for real-time information and argument mapping, using the I-X framework to support group activity and collaboration. Both share a common model for dealing with issues, the refinement of options for the activities to be performed, handling constraints and recording other information. The tools span the spectrum, with Compendium being very flexible with few constraints on terminology and content, to the knowledge-based approach of I-X, relying on rich domain models and formal conceptual models (ontologies). In a personnel recovery experimental simulation of a UN peacekeeping operation, with roles played by military planning staff, the Co-OPR tools were judged by external evaluators to have been very effective.

6.2 I-Rescue Siebra and Tate [18] have used I-X to support coordination of rescue agents within the RoboCup Rescue simulation [8]. Strategic, Tactical and Operational levels of decision-making were modelled. Their work shows the integration of an activity-oriented planner with agent collaboration using the <I-N-C-A> framework, enabling the easy development of activity handlers that are customized according to the tasks of each decision-making level.

6.3 FireGrid FireGrid [1,4] is a multi-disciplinary UK project to address emergency response in the built environment, where sensor grids in large buildings are linked to faster-than-real-time grid-based simulations of a developing fire, and used to assist human responders to work with the building’s internal response systems and occupants to form a team to deal successfully with the emergency. The goal of FireGrid is to integrate several technologies, extending them where necessary:

• High Performance Computing applied to the simulation of fire spread and structural integrity.

8

• Sensors in extreme conditions with adaptive routing algorithms, including input validation and filtering.

• Grid computing including sensor-guided computations, mining of data streams for key events and reactive priority-based scheduling.

• Command and control using knowledge-based planning techniques with user guidance. The I-X technology is to be applied at this level.

This command and control element essentially provides an integrating ‘knowledge layer’ to the system. By using <I-N-C-A> to formalize the interactions between the various participating agents (which, as can be seen from the above description, are drawn from quite different fields and cultures) we hope to harness their various capabilities to provide a seamlessly integrated, response-focused system from the perspective of the human controller.

6.4 AKT e-Response The Advanced Knowledge Technologies (AKT – see www.actors.org) project is an inter-disciplinary applied research project involving a consortium of five UK universities, concentrating on ‘next generation’ knowledge management tools and techniques, particularly in the context of the semantic web. Emergency response has been chosen as an appropriate task to act as a focus for an integrated demonstrator of a number of AKT technologies. To this end, we are currently developing a scenario that builds upon the RoboCup-Rescue project “Kobe earthquake” simulator [8]. This project was begun in the wake of the devastating 1995 earthquake to promote applied research to address the inadequacies of the then available IT systems to cope with the demands of the situation. The Kobe simulator was developed to provide a focus to this effort; it models the immediate aftermath of the earthquake, with fires spreading across a district of the city, injured and trapped civilians, and blocked roads hindering response units. Researchers from various fields are invited to participate in the project as they see fit; for instance, the ideas of multi-agent systems researchers can be applied to the coordination of the available (firefighter, police, ambulance) rescue units to attempt to produce an effective response to the disaster. Indeed, this task has become something of a test-piece for researchers interested in agent coordination, with regular competitions to evaluate the relative success (in terms of minimizing overall human and material cost) of different strategies. However, since the AKT project is focused less on multi-agent systems than on more ‘semantic’ open systems centred on and around humans, for the purposes of the integrated demonstrator we are addressing the task of supporting the high-level strategic response to the emergency. In particular, we aim to provide an ‘intelligence unit’ for the strategy-makers that maintains an overview of the current state of the emergency and the response to it; allows them to access relevant ‘real’ information about the affected locations; lets them explore available options and revise the strategy; and provides a means by which to enact this strategy by relaying orders, reports and other information up and down the chain of command. Since we are looking beyond the simulated world and aim to exploit existing resources and information to guide the response, we have taken the pragmatic decision to

relocate the emergency to London, and in particular the central City of London region, because a number of the AKT technologies are geared towards mining English-language WWW resources for information. (Furthermore, the earthquake has now become a civilian aircraft crash affecting the area, earthquakes of destructive magnitude being rare in the UK.) The demonstrator is to be underpinned by semantic web technologies. The intelligence unit is supported by a ‘triple-store’ database of RDF ‘facts’ described against OWL ontologies describing types of buildings, medical resources, agents, events, phenomena, and so on. This database is to be populated in part by mining WWW pages. A semantic web service-based architecture [17] will be used to provide a flexible and open framework by which, for example resource management, expertise location, situation visualization and matchmaking services can be invoked. Compendium will again be used as the principal interface to the system, providing an ‘information space’ in which the state of the response is described as it evolves, and from which the various services can be invoked. Alongside this, and building on the I-Rescue work, I-X will be used to provide a process-oriented view of the response, with calls to libraries of standard operating procedures providing plans for dealing with archetypal tasks, and activities delegated to agents further down the command-chain, down to and including rescue units ‘on the ground’, also modelled as I-X agents. <I-N-C-A> will be used to formalize the information passed between the agents, and allow it to be located appropriately within the information space. Looking beyond AKT, we aim to make the modified simulation and the associated semantic resources available to the wider research community, the intention being to provide a test-bed for (and challenge to) semantic web and knowledge management researchers. By engaging these researchers in this manner, we hope to contribute to the RoboCup-Rescue project and its laudable aim of advancing the state-of-the-art in disaster management and response technologies.

7 CONCLUSIONS In this paper we have described the I-X system which can be seen as a distributed and intelligent to-do list for agent coordination in emergency response. In this view, the system can be used as an extension of a familiar and proven concept, integrating new technologies in a seamless way. Most importantly, it provides an HTN planner that uses methods (standard operating procedures) to define ways in which tasks can be accomplished, and a capability model that describes other agents in a virtual organization. Together these technologies are used to effectively support emergency responders in organizing a collaborative response quickly and efficiently. A fundamental conceptualization underlying the I-X architecture is the <I-N-C-A> model of a synthesized artifact. This shows up in the internal representation used by I-Plan, in the structure of messages exchanged between I-X agents, and in the user interface, the I-X Process Panels. <I-N-C-A> was developed in the context of AI planning as plan representation but can be generalized to generic synthesis tasks. Furthermore, we have shown that it is closely related to the BDI model of agency, thus providing further evidence that <I-N-C-A> is indeed a good basis for the I-X agent architecture which combines AI planning technology with agent-based system design into an practical

9

framework that has been and is being applied to several emergency response domains.

8 ACKNOWLEDGMENTS The I-X project is sponsored by the Defense Advanced Research Projects Agency (DARPA) under agreement number F30602-03-2-0014. Parts of this work are supported by the Advanced Knowledge Technologies (AKT) Interdisciplinary Research Collaboration (IRC) sponsored by the UK Engineering and Physical Sciences Research Council by grant no. GR/N15764/01. The University of Edinburgh and research sponsors are authorized to reproduce and distribute reprints and on-line copies for their purposes notwithstanding any copyright annotation hereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of other parties.

9 REFERENCES [1] Berry, D., Usmani, A., Terero, J., Tate, A., McLaughlin, S.,

Potter, S., Trew, A., Baxter, R., Bull, M. and Atkinson, M. (2005) FireGrid: Integrated Emergency Response and Fire Safety Engineering for the Future Built Environment, UK e-Science Programme All Hands Meeting (AHM-2005), 19-22 September 2005, Nottingham, UK.

[2] Buckingham Shum, S., Selvin, A., Sierhuis, M., Conklin, J., Haley, C. and Nuseibeh, B. (2006). Hypermedia Support for Argumentation-Based Rationale: 15 Years on from gIBIS and QOC. In: Rationale Management in Software Engineering (Eds.) A.H. Dutoit, R. McCall, I. Mistrik, and B. Paech. Springer-Verlag: Berlin

[3] Conklin J. (2003) Dialog Mapping: Reflections on an Industrial Strength Case Study. In: P.A. Kirschner, S.J. Buckingham Shum and C.S. Carr (eds.) Visualizing Argumentation: Software Tools for Collaborative and Educational Sense-Making. Springer-Verlag: London, pp. 117-136.

[4] FireGrid (2005) FireGrid: The FireGrid Cluster for Next Generation Emergency Response Systems. http://firegrid.org/

[5] Ghallab M., Nau D., and Traverso P. (2004) Automated Planning – Theory and Practice, chapter 11. Elsevier/Morgan Kaufmann.

[6] Huhns M., Singh M. (1998) Agents and Multi-Agent Systems: Themes, Approaches, and Challenges. In: Huhns M., Singh M. (eds) Readings in Agents, pp. 1-23, Morgan Kaufman.

[7] Jabber (2006) Jabber: Open Instant Messaging and a Whole Lot More, Powered by XMPP. http://www.jabber.org/

[8] Kitano H., and Tadokoro S. (2001) RoboCup Rescue: A Grand Challenge for Multiagent and Intelligent Systems, AI Magazine 22 (1): Spring 2001, 39-52

[9] Kreifelts Th., Hinrichs E., and Woetzel G. (1993) Sharing To-Do Lists with a Distributed Task Manager. In: de Michelis G. and Simone C. (eds.) Proceedings of the 3rd European Conference on Computer Supported Cooperative

Work, pp 31-46, Milano, 13-17 September 1993, Kluwer, Dordrecht.

[10] MacLean A., Young R., Bellotti V. and Moran T. (1991) Design space analysis: Bridging from theory to practice via design rationale. In Proceedings of Esprit '91, Brussels, November 1991, pp 720-730.

[11] Openmap (2005) Open Systems Mapping Technology. http://openmap.bbn.com/

[12] Polyak S. and Tate A. (1998) Rationale in Planning: Causality, Dependencies and Decisions. Knowledge Engineering Review, Vol.13(3), pp 247-262.

[13] Russell S. and Norvig P. (2003) Artificial Intelligence—A Modern Approach, 2nd edition, Prentice Hall.

[14] Sacerdoti E. (1975) The Nonlinear Nature of Plans. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp 206-214.

[15] Selvin A.M. (1999) Supporting Collaborative Analysis and Design with Hypertext Functionality, Journal of Digital Information, Volume 1 Issue 4.

[16] Selvin A.M., Buckingham Shum S.J., Sierhuis M., Conklin J., Zimmermann B., Palus C., Drath W., Horth D., Domingue J., Motta E. and Li G. (2001) Compendium: Making Meetings into Knowledge Events. Knowledge Technologies 2001, Austin TX, USA, March, pp 4-7.

[17] Shadbolt N., Lewis P., Dasmahapatra S., Dupplaw D., Hu B. and Lewis H. (2004) MIAKT: Combining Grid and Web Services for Collaborative Medical Decision Making. In Proceedings of AHM2004 UK eScience All Hands Meeting, Nottingham, UK.

[18] Siebra C. and Tate A. (2005) Integrating Collaboration and Activity-Oriented Planning for Coalition Operations Support. In Proceedings of the 9th International Symposium on RoboCup 2005, 13-19 July 2005, Osaka, Japan.

[19] Singh M., Rao A. and Georgeff M. (1999) Formal Methods in DAI: Logic-Based Representation and Reasoning. In: Weiss G. (ed) Multiagent Systems, pp. 331-376, MIT Press.

[20] Tate A. (1977) Generating Project Networks. . In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp 888-893.

[21] Tate A. (1995) Integrating Constraint Management into an AI Planner. Journal of Artificial Intelligence in Engineering, Vol. 9, No.3, pp 221-228.

[22] Tate A., Dalton J., and Stader J. (2002) I-P2- Intelligent Process Panels to Support Coalition Operations. In Proceedings of the Second International Conference on Knowledge Systems for Coalition Operations (KSCO-2002). Toulouse, France, April 2002.

[23] Tate A. (2003) <I-N-C-A>: an Ontology for Mixed-initiative Synthesis Tasks. Proceedings of the Workshop on Mixed-Initiative Intelligent Systems (MIIS) at the International Joint Conference on Artificial Intelligence (IJCAI-03), Acapulco, Mexico, August 2003, pp 125-130.

[24] Wickler G. (1999) Using Expressive and Flexible Action Representations to Reason about Capabilities for Intelligent Agent Cooperation. PhD thesis, University of Edinburgh.

10

Negotiating assignment of disaster monitoring tasks

Doran Chakraborty, Sabyasachi Sahaand Sandip Sen

MCS Dept.University of Tulsa

Tulsa, OK

{doran,saby,sandip}@utulsa.edu

Bradley ClementJet Propulsion Laboratory

Pasadena, California

bclement@aig.jpl.nasa.gov

ABSTRACTWe are interested in the problem of autonomous coordina-tion of ground-based sensor networks and control stationsfor orbiting space probes to allocate monitoring tasks foremerging environmental situations that have the potentialto become catastrophic events threatening life and property.We assume that ground based sensor networks have recog-nized seismic, geological, atmospheric, or some other nat-ural phenomena that has created a rapidly evolving eventwhich needs immediate, detailed and continuous monitor-ing. Ground stations can calculate the resources neededto monitor such situations, but must concurrently negoti-ate with multiple orbiters to schedule the monitoring tasks.While ground stations may prefer some orbiters over oth-ers based on their position, trajectory, equipment, etc, or-biters too have prior commitments to fulfill. We evaluatethree different negotiation schemes that can be used by thecontrol station and the orbiters to complete the monitor-ing task assignment. We use social welfare as the metricto be maximized and identify the relative performances ofthese mechanisms under different preference and resourceconstraints.

Categories and Subject DescriptorsI.2.11 [Artificial Intelligence]: Distributed Artificial In-telligence—Coherence and coordination, Multiagent systems,Intelligent agents

General TermsAlgorithms, Performance, Experimentation

Keywordstask allocation, scheduling, negotiation, disaster manage-ment

1. INTRODUCTION

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.Copyright 200X ACM X-XXXXX-XX-X/XX/XX ...$5.00.

An interesting problem addressed by NASA researchers isto use sensor nodes to inform satellites or orbiters1 aboutnatural events that are difficult to predict accurately, e.g.,earthquakes, forest fires, flash floods, volcanic eruptions, etc.A sensor network is a network of sensor nodes distributedover a region [2]. In each sensor network, there exists somebase stations which are typically more powerful than or-dinary sensor nodes. Sensor nodes communicate with thebase stations in their range. We assume that base stationsin turn are connected to a ground control station that cancommunicate with orbiters. Base stations can use aggre-gated information from sensor nodes to provide dynamicupdates on the monitored area. Such updates can be usedby the control station to identify emerging situations whichnecessitate a host of different high-level responses from theNASA orbiters.

Sensor network applications are gaining recognition atNASA. Already many Earth orbiter missions collaborateon taking joint measurements based on unpredictable atmo-spheric and geological events in order to increase the valueof each mission’s data. While this coordination currentlyrequires much human activity, there is a research initiativethat has demonstrated the coordination of a satellite andEarth-based sensors (such as a video camera or devices onan ocean buoy) to work together to monitor and investigatea large variety of phenomena [3]. When these sensors havedifferent modes of operation and can be controlled, thereis an opportunity to automate operation to more quicklyrespond to urgent events, such as forest fires or volcaniceruptions. In the case where the controllable sensor is aspacecraft, the decisions are not easy to make since thereare many competing objectives. Many scientists competefor spacecraft resources because there are typically five ormore instruments that have constraints on power, energy,temperature, pointing, etc.

Not only do scientists within a mission negotiate, butwhen there are multiple interacting spacecraft, they mustnegotiate with other mission teams. Creating operationplans is especially difficult when so many individuals haveinput. Currently, the activities of a spacecraft are oftenplanned weeks or months in advance for Earth orbiters; thus,these missions are practically unable to respond to events inless than a week. By automating the operation of a space-craft, one spacecraft may be able to respond in minutes.However, if it involves coordinating with other missions, theresponse time depends on the time that the missions take to

1Now onwards we use the term satellite and orbiter inter-changeably.

11

reach agreement on the response. If automated, the missionscould reach consensus quickly as long as they can commu-nicate. Currently, much of this negotiation could be donevia spacecraft operation centers on Earth, but spacecraftneed to be able to participate in coordination when a time-sensitive event is detected, and they need to communicate toreceive new commands or goals as a result of coordination.While some spacecraft are able to send and receive trans-missions at any time, others may only be able to commu-nicate for a few minutes once or twice a day. Coordinationof this kind has been demonstrated in simulation for Marsmissions [4]. Other work has looked into offline schedulingof a group of orbiters [1, 7], but they are centralized andignore the negotiation problem.

In this paper we study the problem of fully autonomousresponse to emerging, potential natural disasters that re-quire coordination of ground stations and earth orbiters foradequate monitoring. We are interested in expediting theresponse time and accuracy to different rapidly evolving nat-ural phenomenon including both ground conditions like for-est fires, earthquakes, volcanic eruptions, floods, and atmo-spheric events like hurricanes, tornadoes, etc. We assumethat ground based sensor networks and other monitoringunits have identified the onset of a rapidly evolving nat-ural event of possibly disastrous proportions and that theground control station responsible for tracking and moni-toring the event has to allocate the monitoring task by as-signing subtasks to orbiters with the requisite monitoringcapabilities. While ground stations have preference for allo-cating subtasks to particular orbiters based on their sched-uled trajectories, on-board equipment, etc., space orbitersare autonomous and have prior commitments and resourceconstraints which may or may not allow them to take on ad-ditional monitoring load at short notice. We assume that or-biters can negotiate with different ground control centers atdifferent times and can evaluate the utility of an announcedmonitoring task based on its current schedule and resourceconstraints, the priority of the task being negotiated and ex-pectations about future task arrivals. In general, the currentschedule and resource conditions of an orbiter is consideredprivate information. A given division of the monitoring taskbetween multiple orbiters will have different utilities fromthe perspective of each of the orbiters and a ground-basedcontrol station. Our research goal is to use distributed nego-tiation mechanisms that can maximize a utilitarian metricof social welfare.

Maximizing social welfare with distributed negotiation ona large solution space is a hard problem. In this paper,we evaluate three distinct negotiation mechanisms: (a) asequential auction scheme, (b) a monotonic concession pro-tocol based negotiation scheme, and (c) a simulated anneal-ing based distributed optimization scheme. While we didnot expect any one scheme to guarantee maximum socialwelfare, it is instructive to use careful experimentation totease out the relative strength of these approaches and iden-tify and characterize situations where it might be preferableto choose each of the approaches. We discuss our moti-vation between choosing these three distinct classes of ne-gotiation approaches and present preliminary experimentalresults with some summary observations.

2. COORDINATION VIA NEGOTIATIONA number of NASA orbiters (e.g., Earth orbiters or Mars

orbiters) are used by different space missions. These mis-sions compete for spacecraft and ground station resources,such as power or energy, orientation, memory storage, an-tenna tracks, etc. [5]. It is a significant challenge to au-tomate this process so that spacecraft resources are effi-ciently allocated. While plans developed offline can sched-ule resource usage for normal operations, system failures,delays, or emerging situations routinely require re-planningand rescheduling on short notice. Of particular relevanceis opportunistic scheduling mechanisms that create planscapable of accommodating high priority tasks at short no-tice [18]. Additionally, sophisticated automated negotiationmechanisms are required to ensure an efficient response tosuch dynamic contingencies.

In automated negotiation, autonomous agents representnegotiating parties [8]. In our formulation each ground sta-tion and orbiter is represented by an agent, and they cannegotiate to coordinate the use of available resources to ful-fill monitoring task requirements. We assume that theseagents are semi-cooperative, i.e., even though their primaryinterest lies in serving their own interests, they will coor-dinate to optimize social welfare. In case of negotiationswith multiple resources, if the priorities of individual agentsare not common knowledge, the rational agents can oftenreach inefficient solutions. The goal of this work is to ex-plore the possible avenues to ensure that the orbiters canrespond rapidly to emerging situations detected by ground-based sensors while ensuring efficient sharing of such addi-tional processing loads and satisfying, to the extent feasible,preferences of the ground station responsible for managingthe monitoring task.

For most of this paper, we restrict our discussion to oneground station negotiating with two orbiters for allocatinga fixed unit of monitoring tasks given an impending emer-gency detected by a network of sensors. Usually, the orbitershave a schedule to complete the preassigned tasks. When-ever any incident takes place which requires monitoring bythe orbiters, the orbiters have to reschedule its preassignedtasks if the incident has a high priority. We assume thatthe base station announces n time periods of monitoringrequirements as a task. The overall task can be dividedamong the two orbiters by partitioning the time periods intonon-overlapping sets. Each orbiter has some utility valueattached to each allocation of the new task based on itsprevious schedule, remaining energy, etc. The intention isto distribute the tasks among the orbiters in such a waythat the total utility of the entire system is maximized. Inthis paper, we have considered three representative negotia-tion mechanisms: sequential auction, multi-issue monotonicconcession protocol, and mediator-based simulated anneal-ing. In the following we present these alternative negotiationmechanisms and briefly discuss their merits and demerits.

Sequential auction: Auction mechanisms [11, 12] can beused to find subtask allocations to maximize social wel-fare. One option would be for the control station tohold a combinatorial auction where each time unit isviewed as an item to be auctioned. Each orbiter bidsfor every subset of time units that it can schedule,and then the control station chooses the social wel-fare maximizing allocation. Unfortunately, both thebid and the optimal task allocation computations areexponential in this case and hence this approach is notfeasible. A feasible, simplified, auction scheme can be

12

to auction each of the n time units sequentially. Theentire task is divided into unit time tasks and they areauctioned sequentially. The orbiters then need to onlysubmit bids for the current time unit under consid-eration and having knowledge of the outcome of theprevious auctions. For each time unit, the auctioneerchooses the allocation, i.e., assigns that time unit toan orbiter, which maximizes the sum of its own utilityand that of the orbiter to which the task is allocated.Suppose the utility of orbiter i for the jth unit task isuij . If jth unit task is done by orbiter i, the utility ofthe control station is denoted by uc

ij . Now, the control

station will award the jth unit task to the orbiter k,

where k = arg maxi∈I{uij + ucij}, I = {1, 2} is the set

of negotiating orbiters.

But this sequential allocation process, though compu-tationally efficient, is not guaranteed to maximize so-cial welfare. Also, there is no consideration of fairnessof the allocation, e.g., an orbiter may be assigned atask for which it has a low utility of such an allocationmay result in a large utility for the control station.

Multi-issue monotonic concession protocol (MC): Awell-known approach to negotiation between two par-ties is the monotonic concession protocol, where eachparty concedes slowly to reach a mutually acceptableagreement. In the current context, we use an exten-sion of this bilateral, single-issue monotonic concessionprotocol [17]. We use a multi-party, multi-issue mono-tonic concession protocol. The single-issue negotiationscenario is completely distributive where a decrease inthe utility of one party implies an increase in the utilityof the other. For multi-party, multi-issue negotiation,this is not the case, and negotiators can find win-winsolutions. But unlike monotonic concession in bilat-eral, single-issue negotiation, it is not clear what con-cession an agent should make [6]. In the protocol usedhere, both the orbiters and the control station partici-pate in the negotiation process. They can arrange thepossible agreements in decreasing order based on thecorresponding utilities and propose allocations in thatorder. If one party finds that the utility of the alloca-tion it is going to propose is as good as any proposal itis already offered, it accepts that proposal, Otherwiseit proposes the allocation which is next in its prefer-ence order. The negotiation will terminate when eachof the agents agree to accept an allocation. It canbe shown that this process will eventually terminate,and the negotiated solution would be Pareto optimalfor the three parties. A disadvantage of this protocolis the relatively slow exploration of different possibili-ties. This can, however, be improved by increasing theamount of concessions made at each step.

Mediator-based simulated annealing: Another distributedapproach to task allocation is proposed by Klein etal. [10], where the negotiating parties try to improve onthe current proposal by using simulated annealing us-ing current proposal as the starting point in its utilityspace of proposals. This is a mediator based approachthat can focus the search for an acceptable proposalin the search space. They have used a mediated singletext negotiation scheme suggested by Raiffa [16]. We

have used their approach for three negotiating parties.In this approach, a mediator proposes2 an allocationoffer, and the negotiating parties either accept, or re-ject the offer. If all of the parties accept the media-tor generates a new proposal by mutating the currentoffer. If any one of them rejects the offer, the medi-ator generates a new proposal by mutating the mostrecently accepted offer. The search terminates if anymutually acceptable proposal is not generated by themediator over a fixed number of proposals. It has beenshown that if all the participants use simulated anneal-ing to decide whether to accept or reject the proposal,they can reach an acceptable solution in reasonabletime. While this method tries to improve fairness, nei-ther Pareto-efficiency nor social welfare maximizationis guaranteed.

From the above discussion we can see that the three pro-tocols have different strengths and weaknesses and it wouldbe instructive to evaluate their efficacy in the context of taskallocation in our application domain.

3. TASK ALLOCATION FRAMEWORKTo evaluate the efficiency of the three schemes discussed

earlier, we have used a simulation environment with a sin-gle control station and two orbiters negotiating for the tasksannounced by the control station. The model assumes thatground sensors entrusted with the job of monitoring specificzones have reported to the control station some data sug-gesting an emerging disaster situation. The onus thereonlies on the control station to distribute the surveillance jobto the two satellites so that proper surveillance of the dis-aster area is achieved. The orbiters have a current sched-ule which is private information. The task, t, announcedby the control station is a surveillance task for a period ofl(t) units. Each satellite is capable of sending pictures ofdifferent quality (q). Quality can be either of high or lowresolution. The utility received by a satellite for sending apicture of high resolution is double that of sending a pictureof low resolution. So for a task of l(t) units, each orbiter

has 3l(t) proposals with the possible value of any unit beingeither :

• 0, signifying that the satellite does not want to dosurveillance for that time unit.

• L, signifying that the satellite is ready to do surveil-lance for that time unit but can only send low resolu-tion pictures.

• H, signifying that the satellite can send high resolutionpictures for that time unit.

A proposal is a vector x ∈ {0, H, L}l(t). Depending on theirapriori schedule, remaining energy level, task and quality,each orbiter has a utility of allocating given portion of thetask. We denote the utility function of an orbiter Ai as ui =ui(Si, ei, q, t), where Si, and ei are the current schedule andremaining energy of Ai respectively. Note that an orbitercan opt to do a part of the task and it has a correspondingutility for doing the subtask.

2The mediator initially generates this offer randomly.

13

There is another important factor taken into account inthe utility calculation. At times, there is a high probabil-ity of occurrence of another such event with higher prior-ity/emergency in the near future. For example, in the rainyseason local flooding is a very probable and frequent event.We assume that all the orbiters have some resource con-straints, and they have to spend some energy for doing atask. So, when responding to a task announced by the con-trol station, an orbiter should consider whether the task isworth doing or if it should preserve energy for a likely highpriority emergency task that would require immediate at-tention. The risk of the orbiter performing this task is thatit may not have enough energy left to serve the next eventof higher priority and the more important event may not getthe service it requires from the satellites. So, the orbitersconsider future expected gain (FEG) defined as,

FEGi = ui(Si, ei, q, t∗) × pr

it∗ − {ui(Si, ei, q, t)

+ui(S∗

i , e∗

i , q, t∗) × pr

it∗} (1)

where S∗i and e∗i are respectively the schedule and remaining

energy of orbiter Ai if it performs the task t, and prit∗ is the

probability of occurrence of another task t∗ of higher priorityfor orbiter i. The first term is the expected utility of doing afuture task t∗ without doing the current task t. The secondterm is the utility of doing the current task, and the thirdterm is the expected utility of doing the future task t∗ afterdoing the current task. Note that after doing the currenttask the schedule and energy level will change and that inturn will affect the utility. If the FEGi value is positive,then Ai would do better by not doing the current task andpreserving energy for future.

The control station can prefer one satellite over anotheron grounds such as:

• geographical proximity.

• quality of service.

• network traffic etc.

Thus the control station maintains a tuple V =< p1, p2 >

where pi denotes the preference of control station for satelliteAi. The utility of the control station depends on the finaldivision of the task between the satellites. The more thepreferred satellite gets the share of the job, the greater is theutility to the control station. The best utility for the controlstation corresponds to the task division when its preferredsatellite performs the entire monitoring task and decides tosend high resolution pictures for the entire time interval.For maintaining uniformity we have normalized the utilityrange of all the satellites and control station in the range[1..10]. The task assigned by the control station can be oftype low priority or high priority. The utility received bythe concerned parties (the control station and the orbiters)for doing a task of high priority is twice than that of doinga task of low priority.

4. EXPERIMENTAL RESULTSWe ran experiments on the above model with the orbiters

and the control station negotiating over tasks using threedifferent mechanisms presented in the previous section. Inall our experiments, we have assumed that the two orbiters

have the same amount of energy to start with. All the ex-perimental results presented here have been averaged over10 runs.

In the first experiment, we observe the variation of thesocial welfare of the system with a higher preference of thecontrol station for the second satellite (p2) while keeping theprobability of an impending high priority task for both satel-lites to a very low value of 0.05. We set p1 to 1 throughoutthe experiment. Figure 1, shows that the auction mechanismdominates the other two negotiation based mechanisms. Inthe auction mechanism, the control station allocates moretasks to the second satellite (because of a high value of p2),hence significantly increasing the satellite’s utility and alsokeeping its own utility high at 10 throughout (refer Fig-ure 2). Lesser amount of task is allocated to satellite 1and hence its utility remains very low. In the monotonicconcession protocol, initially, when both the satellites havethe same priority to the control station, the control stationobtains a high utility of 10 as it does not matter, whichsatellite does the job as long as the job is done in the mostefficient way. However, with an increase in priority of thesatellite 2 to the control station, the control station utilityshows a sharp fall. The reason for this being, the mono-tonic concession in an attempt to ensure fairness, deprivesthe control station from using its preferred satellite. Theutility of satellite 2 remains constant at a medium value of5.5, showing that the protocol prevents the control stationor the preferred satellite (in this case satellite 2) from bene-fiting unfairly at the expense of others. In the auction mech-anism the control station selects a winner in each auction foreach slot to maximize social welfare. Though such indepen-dent optimization is not guaranteed to maximize the overallsocial welfare, it provides a good heuristic approach in cer-tain situations including the current case where the priorityassigned to p2 is significantly increased. In the monotonicconcession technique, the three parties (the two satellitesand the control station) monotonically concede their pref-erences until none has any other proposal to offer which isbetter than the current proposal. Though it ensures fairnessto a certain degree, it does not ensure a high social welfareThe simulated annealing technique is more difficult to an-alyze, as each agent tries to maximize its utility over theirown utility spaces and there is no coordination between theparties to reach a high social welfare.

Next, we ran a similar set of experiments but with theprobability of an impending high priority task (pr1

t∗) forsatellite 1 to 0.95 (see results in Figure 3). The impendinghigh priority task probability of satellite 2, pr2

t∗ remains at0.05. Here we observe that monotonic concession producesa social welfare almost as high as the auction mechanism.For p2 >= 2 , monotonic concession allocates the entiretask to satellite 2 which maximizes the utilities of all theentities in the system. Satellite 2 receives high utility fordoing the entire job as it has no impending task. Satellite 1,anticipating a heavy schedule is inclined to conserve energyfor future high priority impending tasks. The control stationis also satisfied as its favored satellite is doing the entire job.

From the above results we can infer that under conditionswhere a control station has higher priority for one satelliteover another, the auction scheme ensures the highest socialwelfare compared to the other two negotiation mechanisms.However, if fairness is a criterion, then monotonic concessionshould be the preferred negotiation mechanism.

14

0

5

10

15

20

25

30

35

40

1 1.5 2 2.5 3 3.5 4 4.5 5

Soc

ial W

elfa

re

Priority of satellite 2

Auction

MC

SimulatedAnnealing

Figure 1: Social Welfare of the negotiated outcomeswith varying p2 pr1

t∗ = pr2t∗ = 0.05.

0

2

4

6

8

10

1 1.5 2 2.5 3 3.5 4 4.5 5

Util

ity

Priority of satellite 2

CS(MC)

CS(Auction)

S2(MC)

S2(Auction)

Figure 2: Utility Vs Priority of satellite 2 to thecontrol station. pr1

t∗ = pr2t∗ = 0.05. CS denotes the

control station, S2 denotes satellite 2. We followthis abbreviation scheme in the rest of our figures.

In the second set of experiments, we used p1 = 1 and p2

= 2. We recorded the variation in social welfare by increas-ing pr2

t∗ starting from a very low value, while keeping pr1t∗

constant at a very low value of 0.05 (see results in Figure 4and Figure 5). As shown in Figure 4, Initially the auctionscheme shows a higher social welfare than the other two ne-gotiation schemes. However when pr2

t∗ crosses 0.2, it takesa sudden drop. The reason for this is that the control sta-tion in an attempt to contribute to social welfare, starts toallocate more work to satellite 1. This can be verified by thedrop in utility value of both the control station and satel-lite 2 in Figure 5. However for pr2

t∗ values 0.45 and higher,the social welfare curve for auction picks up (refer Figure 4)as now satellite 2 is happy of being relieved of all its loadthereby achieving a high utility of 10 (refer Figure 5). Themonotonic concession shows a medium social welfare valueof about 18 throughout the experiment. The utilities ofthe satellites and the control station (refer Figure 5) remainfairly constant, thus showing that the protocol adapts itselfto changing scenarios to ensure fairness amongst the agents.Simulated annealing once again offers a fairly constant so-

0

5

10

15

20

25

30

35

40

1 1.5 2 2.5 3 3.5 4 4.5 5

Soc

ial W

elfa

re

Priority of satellite 2

Auction

MC

SimulatedAnnealing

Figure 3: Social Welfare of the negotiated outcomes.pr1

t∗ = 0.95 and pr2t∗ = 0.05.

0

5

10

15

20

25

30

35

40

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Soc

ial W

elfa

re

Emergency Probability of satellite 2

Auction

MC

SimulatedAnnealing

Figure 4: Social Welfare of the negotiated outcomeswith varying pr2

t∗ . pr1t∗ = 0.05, p1 = 1 and p2 = 2.

cial welfare value around 15, showing that agents climbingin their respective utility spaces seldom contribute to a highsocial welfare.

In the next scenario, we ran similar experiment againwhile increasing the impending high priority task probabil-ity of satellite 1 (pr1

t∗) to 0.95 (see Figure 6). The mono-tonic concession and auction protocol gives similar resultsunder such a scenario. For lower values of pr2

t∗ , monotonicconcession allocates most of the job to satellite 2. This inturn favors the control station too, so there is a high so-cial welfare value. Auction also does the same by allocatingmore task to satellite 2. Figure 7 shows the utility curves ofboth control station and satellite 2 for the two mechanismschemes. From Figure 7, it is clear, that satellite 2 is allo-cated lesser work for pr2

t∗ > 0.4, resulting in a decrease inits utility value. With an increase in involvement of satellite1 (the less preferred satellite to control station), the utilityof the control station falls too. For monotonic concessionthese utilities converge to a fixed value for probabilities ≥0.65. This happens because, from probabilities ≥ 0.65, thetask distribution between the satellite gets fixed, therebystabilizing the utility values of all the entities in the system.In case of auction, the situation is a little bit more compli-

15

0

2

4

6

8

10

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Util

ity

Emergency Probability of satellite 2

CS(MC)

CS(Auction)

S2(MC)

S2(Auction)

CS(MC)

CS(Auction)

S2(MC)

S2(Auction)

Figure 5: Utility obtained while varying pr2t∗ . Here,

pr1t∗ = 0.05, p1 = 1 and p2 = 2.

cated. For 0.4 ≤ pr2t∗ ≤ 0.5, control station in an attempt

to keep its utility high, overburdens satellite 2. This is re-flected in the sharp drop of the utility value for satellite 2in this region shown in Figure 7. But for pr2

t∗ > 0.5, thecontrol station has to sacrifice some of its utility to keep thesocial welfare of the system high. In this period, satellite2 shows a sharp rise of utility as it is relieved of some ofthe burden assigned to it before. Finally their utilities sta-bilize at pr2

t∗ ≥ 0.75. However from the social welfare pointof view, at higher values of p2, all the three curves convergethereby suggesting that there is not much to gain by choos-ing a specific negotiation technique when both the satellitesare extremely resource constrained (see Figure 6).

Finally we ran an experiment to compare the relative per-formances of the three negotiation techniques with increas-ing l(t), the number of time slots required for surveillance,keeping the resource constraints same for both satellites andp2 > p1 (see Figure 8). We see that both auction and mono-tonic concession performs better than simulated annealing.Thus under fairly similar states of the two satellites, auctionand monotonic concession should be the preferred negotia-tion techniques to divide the labor. If social welfare maxi-mization is the main criteria, then sequential auction shouldbe the preferred mechanism while if fairness is the chief cri-terion, then monotonic concession should be the preferrednegotiation mechanism.

5. RELATED WORKExtreme environmental events, like tsunami, tropical storms,

flooding, forest fires, etc. can lead to widespread disastrouseffects on our society. The frequency of such incidents in therecent past have focused the urgency of developing techno-logical solutions to mitigate the damaging effects of naturaldisasters [15, 19].

Multiagent systems are successfully deployed in diverseapplications for complex and dynamic environments [14].We believe it can be beneficial to apply the potential ofmultiagent systems research to minimize the effects of suchdisasters. Schurr et al. presents a large-scale prototype,DEFACTO, that focuses on illustrating the potential of fu-ture agent-based response to disasters. Robocup rescue [9] isanother effort to build robust disaster-response techniques.

0

5

10

15

20

25

30

35

40

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Soc

ial W

elfa

re

Emergency Probability Of satellite 2

AuctionMC

SimulatedAnnealing

Figure 6: Social Welfare of the negotiated outcomefor different values of pr2

t∗ . Here, pr1t∗ = 0.95, p1 = 1

and p2 = 2.

0

2

4

6

8

10

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Util

ity

Emergency Probability of satellite 2

CS(MC)

CS(Auction)

S2(MC)

S2(Auction)

CS(MC)

CS(Auction)

S2(MC)

S2(Auction)

CS(MC)

CS(Auction)

S2(MC)

S2(Auction)

Figure 7: Utility obtained for different pr2t∗ . Here,

pr1t∗ = 0.95, p1 = 1 and p2 = 2.

The agents need to find out optimal or near optimal searchstrategies after a large scale disaster. A significant challengeis to coordinate the actions and positions of the agents inthe team. All of these applications of multiagent systemsfocus on the coordination among the agents to improve theresponse to any environmental or technological disasters.

Satellite applications are very useful for monitoring disas-ters, like flood, volcanic eruption, forest fire, etc. Recentlya sensor network based application has been employed [3],where low-resolution, high-coverage, ground based sensorstrigger the observation by satellites. In this paper, we havediscussed a similar problem but our focus is to efficientlyand autonomously allocate the monitoring tasks among thesatellites. Negotiation is the most well-known method forefficient allocation of tasks among a group of agents [8, 13].The agents can search the solution space in a distributedway to reach optimal solution. Here we have compared somerepresentative negotiation strategies used in multiagent ne-gotiation [10, 17].

6. CONCLUSION

16

0

5

10

15

20

25

30

35

40

3 3.5 4 4.5 5 5.5 6 6.5 7

Soc

ial W

elfa

re

Task size

Auction

MC

SimulatedAnnealing

Auction

MC

SimulatedAnnealing

Figure 8: Social Welfare vs Task size. pr1t∗ = pr2

t∗ =0.05, p1 = 1 and p2 = 2.

In this paper we have studied the problem of fully au-tonomous response to emerging, potential natural disastersthat require coordination of ground stations and earth or-biters for adequate monitoring. The satellites can autonomouslydistribute the load of monitoring of any unprecedented event.We have compared three different negotiation mechanismsused by the orbiters and the control station to reach an ef-ficient agreement on the allocation of the task. We havefound the sequential auction to be the most effective mech-anism amongst them. But this mechanism also has somelimitations. Our objective is to find a robust, fast and effi-cient negotiation mechanism that enables the orbiters andthe control station to quickly reach an efficient agreement.We would also like to explore if the negotiating parties canadaptively choose the most suitable negotiation mechanismfor different emergencies.

This paper addresses the problem of coordinating Earthorbiters in the context of a sensor web when communica-tion opportunities are limited for some. Each spacecraft hasview-periods with different measurement targets based on itsorbit. For some of these view-periods, measurements havelower quality than others depending on the angle from thetarget to the spacecraft. Spacecrafts have overlapping anddifferent capabilities, so certain events and targets will re-quire some measurement types more than others, and somesubsets of spacecraft will be able to fulfill them. While wediscuss this problem in the specific context of Earth orbiters,other Earth-based sensors may also require similar sophisti-cated planning operation, and our techniques would applyto them. In addition, the Mars network of spacecraft androvers continues to grow, and the algorithms we present willbe of even greater significance to those missions since humaninvolvement is difficult when communication delay is tens ofminutes, and rovers are not in view half of the time.

Acknowledgments: This work has been supported by aNASA EPSCoR RIG.

7. REFERENCES[1] M. Abramson, D. Carter, S. Kolitz, J. McConnell,

M. Ricard, and C. Sanders. The design andimplementation of draper’s earth phenomenaobserving system (epos). In AIAA Space Conference,

2001.

[2] Ian F. Akyildiz, Wilian Su, YogeshSankarasubramaniam, and Erdal Cayirci. A survey ofsensor networks. IEEE Communications Magazine,40(8):102–114, 2002.

[3] S. Chien, B. Cichy, A. Davies, D. Tran, G. Rabideau,R. Castano, R. Sherwood, D. Mandl, S. Frye,S. Shulman, J. Jones, and S. Grosvenor. Anautonomous earth-observing sensorweb. IEEEIntelligent Systems, 20(3):16–24, 2005.

[4] B. J. Clement and A. C. Barrett. Continualcoordination through shared activities. In Proceedingsof the Second International Conference onAutonomous Agents and Multi-Agent Systems, pages57–64, 2003.

[5] B. J. Clement and M. D. Johnston. The deep spacenetwork scheduling problem. In Proceedings of theSeventeenth Innovative Applications of ArtificialIntelligence Conference, pages 1514–1520, 2005.

[6] U. Endriss. Monotonic concession protocols formultilateral negotiation. In AAMAS-06: Proceedingsof the fifth international joint conference on Autonomous agents and multiagent systems, 2006. to appear.

[7] J. Frank, A. Jonsson, and R. Morris. Planning andscheduling for fleets of earth observing satellites, 2001.

[8] N. Jennings, P. Faratin, A. R. L. S. Parsons, C. Sierra,and M. Wooldridge. Automated negotiation:prospects, methods and challenges. InternationalJournal of Group Decision and Negotiation,10(2):199–215, 2001.

[9] H. Kitano, S. Tadokor, I. Noda, H. Matsubara,T. Takhasi, A. Shinjou, and S. Shimada.Robocup-rescue: Search and rescue for large scaledisasters as a domain for multi-agent research, 1999.

[10] M. Klein, P. Faratin, H. Sayama, and Y. Bar-Yam.Negotiating complex contracts. Group Decision andNegotiation, 12:111–125, 2003.

[11] P. Klemperer. Auction theory: A guide to theliterature. Journal of Economic Surveys,13(3):227–286, 1999.

[12] V. Krishna. Auction Theory. Academic Press, 2002.

[13] S. Lander and V. Lesser. Understanding the role ofnegotiation in distributed search among hetergeneousagents. In Proceedings of the Thirteenth InternationalJoint Conference on Artificial Intelligence (IJCAI-93),pages 438–444, Chambery, France, 1993.

[14] V. R. Lesser and D. D. Corkill. The DistributedVehicle Monitoring Testbed: A tool for investigatingdistributed problem solving networks. AI Magazine,4(3):15–33, Fall 1983. (Also published in BlackboardSystems, Robert S. Engelmore and Anthony Morgan,editors, pages 353–386, Addison-Wesley, 1988 and inReadings from AI Magazine: Volumes 1–5, RobertEngelmore, editor, pages 69–85, AAAI, Menlo Park,California, 1988).

[15] D. Mendona and W. A. Wallace. Studyingorganizationally-situated improvisation in response toextreme events. International Journal of MassEmergencies and Disasters, 22(2), 2004.

[16] H. Raiffa. The Art and Science of Negotiation.Harvard University Press, Cambridge, MA, USA,1982.

17

[17] J. S. Rosenschein and G. Zlotkin. Rules of Encounter.MIT Press, Cambridge, MA, 1994.

[18] S. Saha and S. Sen. Opportunistic scheduling andpricing in supply chains. KI- Zeitschrift fur KunstlicheIntelligenz (AI - Journal of Artificial Intelligence),18(2):17–22, 2004.

[19] N. Schurr, Janusz Marecki, J. Lewis, M. Tambe, andP. Scerri. The defacto system: Coordinatinghuman-agent teams for the future of disaster response,2005.

18

Toward Automatic Reconfiguration of Robot-SensorNetworks for Urban Search and Rescue

Joshua ReichDepartment of Computer Science

Columbia University1214 Amsterdam Ave, New York NY 10027 USA

reich@cs.columbia.edu

Elizabeth SklarDept of Computer and Information Science

Brooklyn College, City University of New York2900 Bedford Ave, Brooklyn NY, 11210 USA

sklar@sci.brooklyn.cuny.edu

ABSTRACTAn urban search and rescue environment is generally ex-plored with two high-level goals: first, to map the space inthree dimensions using a local, relative coordinate frame ofreference; and second, to identify targets within that space,such as human victims, data recorders, suspected terror-ist devices or other valuable or possibly hazardous objects.The work presented here considers a team of heterogeneousagents and examines strategies in which a potentially verylarge number of small, simple, sensor agents with limitedmobility are deployed by a smaller number of larger roboticagents with limited sensing capabilities but enhanced mo-bility. The key challenge is to reconfigure the network auto-matically, as robots move around and sensors are deployedwithin a dynamic, potentially hazardous environment, whilefocusing on the two high-level goals. Maintaining informa-tion flow throughout the robot-sensor network is vital. Wedescribe our early work on this problem, detailing a simu-lation environment we have built for testing and evaluatingvarious algorithms for automatic network reconfiguration.Preliminary results are presented.

1. INTRODUCTIONThis work explores the use of “robot-sensor networks” for

urban search and rescue (USAR), where the topography andphysical stability of the environment is uncertain and timeis of the essence. The goals of such a system are two-fold:first, to map the space in three dimensions using a local,relative coordinate frame of reference; and second, to iden-tify targets within that space, such as human victims, datarecorders, suspected terrorist devices or other valuable orpossibly hazardous objects. Our approach considers a teamof heterogeneous agents and examines strategies in which apotentially very large number of small, simple, sensor agentswith limited mobility are deployed by a smaller number oflarger robotic agents with limited sensing capabilities butenhanced mobility. While every node in the network neednot be directly connected to every other node, it is vital

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.Copyright 200X ACM X-XXXXX-XX-X/XX/XX ... $5.00.

that information be able, eventually, to make its way todesignated “contact” nodes which can transmit signals backto a “home base”. It is advantageous for the network to pos-sess reliable and complete end-to-end network connectivity;however, even when the network is not fully connected, mo-bile robots may act as conduits of information — either bypositioning themselves tactically to fill connectivity gaps, orby distributing information as they physically travel aroundthe network space. This strategy also enables replacementof failed nodes and dynamic modification of network topol-ogy to provide not only greater network connectivity butalso improved area coverage. The robotic component of ouragent team can leverage its mobility capabilities by allowingdynamic spatial reconfiguration of the robot-sensor networktopology, while the sensor components help to improve local-ization estimates and provide greater situational awareness.

The past several years have shown great advances in boththe capabilities and miniaturization of wireless sensors [16].These advances herald the development of systems that cangather and harness information in ways previously unex-plored. Sensor networks may provide broader and moredynamic perspectives if placed strategically around an envi-ronment, delivering numerous small snapshots over time. Byfusing these snapshots, a coherent picture of an environmentmay be produced — rivaling output currently provided bylarge, complex and expensive remote sensing arrays. Like-wise, sensor networks can facilitate propagation of commu-nication in areas unreachable by centralized broadcast dueto obstacles and/or irregularities in the connectivity land-scape. While traditional non-mobile sensor networks possesstremendous potential, they also face significant challenges.Such networks cannot take an active role in manipulatingand interacting with their environment, nor can they physi-cally reconfigure themselves for more efficient area coverage,in-depth examination of targets, reliable wireless connectiv-ity, or dynamic protection against inclement environmentaldevelopments.

By incorporating intelligent, mobile robots directly intosensor networks, all of these shortcomings may be addressed.Simple, inexpensive, easily programmed, commercial off-the-shelf robotics kits like Garcia [7], or even the new LEGONXT [15], could provide inexpensive test platforms and wire-less networking capabilities. Mobile robots provide the abil-ity to explore and interact with the environment in a dy-namic and decentralized way. In addition to enabling mis-sion capabilities well beyond those provided by sensor net-works, these new systems of networked sensors and robotsallow for the development of new solutions to classical prob-

19

lems such as localization and navigation [3]. Arguably, thedevelopment of mixed sensor-robot networks will allow forexploration of and interaction with environments in wayspreviously infeasible.

One of the biggest challenges in an urban search andrescue environment is the need to maintain consistent andreliable network communication amongst remote rescuers,whether they are human or robot or both. As rescuers movearound an uncertain environment, not only do their relativepositions change, but also it is not unlikely that their envi-ronment will change; collapsed buildings may settle, floodwaters may recede or swell, earthquake sites may shift due toaftershock. The capability for a team of agents to map theirspace collaboratively, identify victims and other targets ofinterest, while maintaining information flow is crucial; andgiven the dynamic nature of the environments they are ex-ploring, it is also important that such ad-hoc networks beable to reconfigure automatically, not only due to changesin position of the agents but also caused by failure of one ormore nodes.

The work presented here, in very early stages of develop-ment, examines the issue of automatic reconfiguration of anetwork of agents under such conditions as described above.The longterm goal of this work is to deploy a physical sys-tem in an urban search and rescue test arena [11], but thepresent stage of work involves development of a simulatorin which crucial features are emulated and where designof algorithms for automatic network reconfiguration can betested and evaluated. This paper begins with backgroundin sensor and robot networks, highlighting current areas ofchallenge within the field. Starting with section 3, our ap-proach to the problem is described, including detailed dis-cussion of our testbed, the algorithm we are evaluating andpreliminary experimental results from testing the algorithmin a simulated USAR environment. We close with discussionof future work.

2. BACKGROUNDThe challenges to realizing the potential of sensor-robot

networks exist at both hardware and software levels. Openproblems include power management, communication, in-formation fusion, message routing, decision-making, role as-signment, system robustness, and system security. Currentresearch has begun to address many of these issues. Sev-eral methodologies have been tested for target detection andtracking, both with fixed sensors [5] and using large-scalemobile robotic teams [12]. Researchers are actively investi-gating novel message routing protocols, some of which en-able self-organization of networks nodes [17]. As many ofthese approaches rely on some type of geographic routingscheme, sensor localization has become an area of inquiry[2]. Fundamental issues such as dealing with power supplylimitations [6] and ensuring coverage of the area to be sensed[10] are also being explored.

Recently a small group of researchers has begun explor-ing the synergy between autonomous robots and sensor net-works. Kotay et al. [2005] have explored several issues us-ing the synergy between GPS-enabled robots and networkedsensors to provide network-wide localization services, pathplanning, and improved robot navigation. Gupta et al.[2004] have suggested a method for the transportation ofresources by combining robots with sensor network services.The Centibots project [12] examines how large numbers of

more sophisticated robots may collaborate to create mapsand subsequently surveil the area by leveraging ad-hoc wire-less networking capabilities. These results, produced at theboundary where robotic teams and sensor networks inter-sect, suggest a large and fascinating problem space openfor exploration. Following is a sampling of the interrelatedissues for which techniques, algorithms, and hardware solu-tions need to be devised:

1. high-level team formation and mission fulfillment,

2. communications and routing,

3. localization and mapping,

4. path planning,

5. target tracking,

6. standardization of hardware services/interfaces, and

7. asymmetric wireless broadcast and network interfer-ence.

While our work touches somewhat on all of these issues, itfocuses mostly on the fifth, third and second, in that order,exploring how such systems can provide useful and robustbase-level behaviors — and do so with minimal hardware re-quirements or dependence on favorable environmental con-ditions.

One commonality amongst much of the works cited aboveis the reliance on sophisticated hardware and/or friendlyor over-simplified environmental conditions. Most work ei-ther assumes the existence of basic services such as local-ization and orientation, or considers only the cases whereat least a fraction of the agents possess essential hardwareused for global localization (e.g., global positioning systemor GPS). While these assumptions allow for investigationof important problems, they fail to provide techniques thatwill be effective when such hardware services (e.g., GPS,magnetic compass) fail or are unavailable (e.g., indoor orUSAR environments). Currently, wireless sensor sizes rangefrom centimeters to millimeters. The smallest robots aregenerally one to two orders of magnitude larger, in the cen-timeter to meter range. Such equipment, while small andinexpensive enough for ubiquitous deployment, may also beseverely constrained in offering sophisticated hardware ser-vices. To allow for the widest range of deployable systems,this work examines systems that make minimal assumptionsconcerning hardware capabilities. Limiting the use of so-phisticated, expensive hardware for network nodes may bemore than compensated for in both cost and performanceby the advantages of density and redundancy that smaller,simpler, less costly sensors and robots can provide. Thisapproach would be particularly advantageous in harsh op-erational environments where loss, destruction, or failure ofnetwork components becomes likely.

3. OUR METHODOLOGYOur immediate goal is to guide robot searchers effectively

to targets by leveraging communications and sensing ser-vices provided by a dense network of non-mobile agent-basedsensors. Additionally, we desire that the system be able tofulfill its mission requirements without any component thathas localization capabilities (in a global sense) — and to do

20

so in a distributed manner. The only knowledge primitivesassumed by the simulation are: for all agents, awareness ofneighbors and nearby targets, and (for robots) approximatedistance from neighbors and approximate direction towardstargets.

We employ a network routing scheme to route not justour system’s communications, but also the movement of itsmobile components. We note that there exist a family ofalgorithms currently used to do route planning within net-works, so as to produce routes with minimal hop distance [8].In most networks, hop distances are not highly related to thephysical distance over which a piece of information is passed.An email to one’s next door neighbor might pass over al-most as many hops as one sent to a correspondent overseas.However, in high density, short-range sensor networks thistends not to be the case; the correspondence between min-imal hop path and physical distance between nodes beingfairly strong in many environments. Consequently, knowl-edge of the minimal hop paths could not only enable efficientmessage routing in the network but also provide a good ap-proximation of the shortest physical paths from one sensorto another several hops away.

As an example, consider the simple robot-sensor networkillustrated in Figure 1. The robot arrives at node A, whichhas been informed that node D, three hops away, has de-tected a target in its vicinity (the target is the star in thefigure, to the northeast of node D). Node A can then informrobot that D is detecting a target and that node B is thenext hop along the shortest routing path to D. By followingsome detectable gradient towards B (e.g., signal strength),robot will be able to come close enough to B to receive infor-mation about and a signal from the next-hop on path to D,namely node C. In this fashion robot is able to quickly findits way towards D without any a priori localization knowl-edge. Once robot has reached D, it will be close enough todirectly detect the target itself.

A C D

B

robot

*

Figure 1: Sample robot-sensor network.Node D detects a target to its northeast. The network can routethe robot along the nodes from its present location, within range

of A, to the node which has detected the target, D.

In order to make the above scheme work several algorith-mic questions need to be addressed:

• Where should the network routing information be cal-culated and stored?

• How should information regarding which sensors aredetecting targets be distributed?

• How should robots go about choosing a course of action(e.g. follow path or search for nearby target)?

• What information should be exchanged between net-work components (both robots and sensors)?

In the remainder of this section, we address these questionsand explain the choices we have made in our implementa-tion.

3.1 Network routing and distribution of targetinformation

Our hard requirements for network routing are that anysensor in the system should provide both hop-distance andnext-hop to a given destination, if a path exists. Addition-ally, in the interest of system scalability and responsiveness,we desire path computation and storage to be local to eachsensor. A number of options are available, the most straight-forward of which is simply to employ a slightly modified ver-sion of the popular Distributed Vector (DV) routing algo-rithm [14], one of the two main Internet routing algorithms.The DV algorithm itself operates in a very straightforwardfashion. Each node in the network keeps a routing table con-taining identifiers of every node to which it knows a path,along with the current hop-distance estimate and next hopalong that path. Each asynchronously sends its routing ta-ble to all neighboring nodes which, in turn, check their ta-bles to learn of new destinations. Additionally, when nodeA sends its routing table to node B, B will check its list ofknown nodes and hop-distances against the table sent by A

and choose A as the next hop for any nodes that would bemore quickly reached through A. If B does make any ad-ditions or adjustments to its table, it will send the revisedtable to all of its own neighbors to alert them to these newor shorter paths. In this manner, routing information willbe diffused throughout the network.

The theoretical performance of DV is quite good and itswide adoption attests to its reliability, simplicity, and scal-ability. However, in our simulation we found a significanttime lag once network density increased past an average of10 neighbors — reflecting the high number of messages be-ing sent before the nodes converged. Additionally, the sizeof the routing table held at each node scales linearly withthe network size — possibly making this approach infeasiblefor very dense networks, at least not without modification.Lastly, while DV provides a sophisticated means for passingunicast messages, it may not provide competitive advantagejustifying its cost in applications where much informationmay be expressed in the form of a network-wide gradient. Inour current work, we are comparing the performance of DVto a network gradient, where nodes learn only hop-distancefrom the nearest sensor detecting a target, supplemented bya more expensive direct message-passing service.

3.2 Robot behaviorOur goal for robot behavior is for each robot to make an

independent decision (as opposed to receiving orders from acentralized node in the network), but at the same time toavoid the computational costs associated with sophisticateddecision-making. Consequently, each robot is given a simplehierarchy of behaviors, using a simple subsumption architec-ture [1], along with state transitions, as illustrated in Figure2. The hierarchy contains three states, numbered in increas-ing order of precedence. The most dominant state is state 2in which a target has been detected. The robot’s behaviorin state 2 is to search for the target until (a) the robot findsthe target, (b) the robot discovers another robot has gotten

21

there first, or (c) the robot loses the target signal. In thefirst case the robot settles near the target and broadcasts asignal of ownership. In the two latter cases, the robot re-turns to behavior state 0 (from which it may immediatelyjump to state 1). State 1 is reached from state 0; whenno target signal is present but some sensor is in range, therobot’s behavior is to traverse the network towards a targetsome hops away. Finally, in state 0 the robot conducts ablind search, looking first for target signals (transition tostate 2) and second for sensor signals (transition to state 1).

Lost

State 0 State 1

State 2

Blind Search Follow Sensors

Approaching Target

Lost Target Signal

Target Signal Target SignalAcquiredAcquired

Sesnor Signal Acquired

Sensor Signal

Figure 2: Robot behavior hierarchy.

3.3 Information exchangeIn our initial implementation, agents only provide each

other with path information to sensors’ nearby targets. Ourcurrent work involves expanding the information exchangecapabilities of the system so that additional data may bepassed between nodes in an efficient manner. We are look-ing for this to improve system performance in several ways.First, once a target has been found and its surroundings ex-plored (for any additional targets), the sensors close enoughto receive the target signal should be marked by the networkaccordingly. This information should then be propagatedthroughout the network, preventing these sensors from be-ing continually revisited by curious robots. Second, sensorsmay mark the passage of robots with a time-stamp and/orvisit counter. By doing so, robots may decide to avoid sen-sors visited very often or very recently, choosing to explorepaths less traveled or even areas entirely out of the networkcoverage. Third, robots may leave “trails” [4], in order tofacilitate quick transference of information back to the homebase.

4. IMPLEMENTATIONWe have used the NetLogo (version 3.0.2) multiagent pro-

gramming environment [18] for constructing our initial sim-ulation. All results presented here are based on experimentsdesigned and executed in this simulator. Figures 3, 4 and 5illustrate the environment. The gray regions represent ob-stacles, both for physical travel by the robot and wirelessconnectivity of the network. We note that in the real world,some physical obstructions may not interfere with wirelessconnectivity and vice versa; for ease in constructing our ini-tial implementation, we chose to make this assumption, butcurrent work is exploring situations in which the two typesof obstructions are handled separately. In the white areas onthe figures, the robots (and the signal) are free to travel. The

dark circles represent agent sensors which are immobile, andthe lines between them show the connectivity of the network.The bug-like symbols represent the mobile, robotic agents.Section 3.2 describes the hierarchical control algorithm wehave implemented for the robots. The sensor agent behav-ior is even more simplistic. In our current implementation,these agents do not possess any decision-making capabili-ties; as described below, they merely broadcast any targetinformation as well as beacon signals for mobile agents.

For the present, we have adopted a simplified non-prob-abilistic model of wireless broadcast. We assume a spher-ical broadcast model, and, for the moment, consider nei-ther broadcast collisions nor other types of signal propaga-tion effects. Current work is exploring this aspect in de-tail, incorporating models of trust in the existing systemand endowing the sensor agents with decision-making abil-ities such that broadcast becomes non-deterministic. Thesensing model (similarly non-probabilistic) is also spherical,while the robots are assumed to possess directional sensingarrays. The simulation allows for the investigation of ar-eas with obstacles to robot movement and can adjust bothpercentage of area covered by obstacles as well as their clus-tering tendency.

Robot movement is modeled probabilistically. When arobot moves forward, it turns randomly a bit to one sideor the other. The degree to which the movement of robotsis skewed is controlled by a global variable and can be ad-justed to consider different robot platforms or surfaces. Therobots have the ability to move around the environment anddisperse a potentially large number of non-mobile sensoragents. Currently two types of sensor dispersal algorithmshave been compared: random distribution radially from thecenter of the robot start location, and uniform random dis-tribution throughout the environment.

5. PRELIMINARY EXPERIMENTSThe primary issue we aimed to assess with our initial im-

plementation was whether at system’s current level of de-velopment, a performance difference could be ascertainedbetween our sensor-robot network and a system employingrobots alone. In order to evaluate the problem space, weconducted 1152 runs, sampling over the following six ad-ditional variables: obstacle density, number of robots, num-ber of non-mobile sensors, dispersal method, broadcast radiusand spread of communication. The metric used for all ex-periments was the number of time steps taken until 90% ofthe targets had been discovered.

The variable with the clearest effect was obstacle density.Spaces with few obstacles, like Figure 5, were easily solvedby both sensor-robot teams and robot-only teams. Spaceswith many obstacles (like Figures 3 and 4) proved signifi-cantly more difficult, often taking upwards of 5 times longerto find 90% of the targets. Consequently, we chose to focusour set of experiments on environments with 25-30% of thearea occupied by obstacles. Sensors were distributed accord-ing to a uniform random distribution, as were targets. Weused 30 robots and 90 sensors for the trials and a broadcastradius varying between 1/8th and 1/12th of the area’s width.

The results of our experiments so far are statistically in-conclusive; as yet, we are unable to show a comparative ad-vantage between the sensor-robot and robot-only teams un-der the parameterization chosen. However, by viewing sev-eral simulations and examining system performance, we are

22

Figure 3: Many Obstacles: Open.

key (applies to figures 3, 4 and 5):

robot obstacle

target sensors

able to generate some qualitative observations that encour-age us to continue with this line of inquiry. On individualtrials, the sensor-robot teams often significantly outperformthe robot-only teams, but these are offset by occasions inwhich the sensor-robot teams becomes bogged down in partsof the network already explored. The sensor-robot teams dovery well in situations where the environment is highly seg-mented and both sensor and targets are fairly well spreadout (e.g., Figure 4). The robots are able to follow the net-work paths successfully through small crevices to reach newcompartments and thereby find targets effectively; in con-trast, with only random guessing about where to move next,the robot-only teams tend to do rather poorly in such spaces.In the space shown in Figure 4, for example, the robot-onlyteam took 1405 time steps to complete the search, while thesensor-robot team managed it in only 728.

In relatively open spaces, like (Figure 3), the robot-onlyteams have much less trouble (in this case the two approachesboth took around 450 time steps). The sensor-robot systemsperform badly when some of the targets have several sensorsnearby, while others have few or no nearby sensors. In thesecases, the robots continually revisit the sensors near targetsalready discovered, keeping too many robots from exploringother areas. The robot-only teams ignore the network inthese situations and perform considerably better.

The main problem the sensor-robot teams experience isthat each robot keeps its own list of target-detecting sen-sors that it has visited. Since robots choose the sensorsthey will visit randomly from the list of unvisited target-detecting sensors, every robot can end up visiting a multiply-detected target several times for each time it looks for asingly-detected target. Moreover robots try to visit everydetectable target before looking for targets un-sensed by the

Figure 4: Many Obstacles: Segmented.

network! Consequently, in certain trials, the network effec-tively traps the robots in one portion of the environment fora significant time-span. We believe that once additional in-formation sharing facilities outlined in section 3.3 have beenimplemented, the sensor-robot system will statistically out-perform robot-only systems when repeating the experimentsoutlined above.

6. SUMMARY AND FUTURE WORKWe have presented early work in the development of strate-

gies for controlling teams of heterogeneous agents, possess-ing a mixture of sensing and mobility characteristics. Takingadvantage of recent advances in sensor networks and routingschemes, we are interested in exploring situations in which apotentially very large number of small, simple, sensor agentswith limited mobility are deployed by a smaller number oflarger robotic agents with limited sensing capabilities butenhanced mobility. Our longterm goal is to apply techniquesdeveloped to urban search and rescue problems.

In the short term, our work is focusing primarily on con-tinued development of simulation platform. The immediatesteps involve: (a) introduction of gradient-based routing,(b) incorporation of enhanced information sharing facilities,and (c) improvement of robot behavior to incorporate newinformation. The next steps entail producing comprehensiveempirical results, evaluating hardware platforms and build-ing prototype hardware systems for testing our strategies.Our plan is to contrast simulated results with those fromour physical prototype, using data collected in the physicalworld to seed learning algorithms for building error modelsin the simulator, which can then be used to improve perfor-mance in the physical setting.

7. REFERENCES[1] R. Brooks. A robust layered control system for a

mobile robot. IEEE Transactions on Robotics andAutomation, 2:14–23, 1986.

23

Figure 5: Screen-shot of simulation with few obsta-

cles.

[2] A. Caruso, S. Chessa, S. De, and A. Urp. Gps freecoordinate assignment and routing in wireless sensornetworks. In IEEE INFOCOM, 2005.

[3] P. Corke, R. Peterson, and D. Rus. Localization andnavigation assisted by cooperating networked sensorsand robots. International Journal of RoboticsResearch, 24(9), 2005.

[4] M. Dorigo, V. Maniezzo, and A. Colorni. The AntSystem: Optimization by a colony of cooperatingagents. IEEE Transactions on Systems, Man andCybernetics-Part B, 26(1):1–13, 1996.

[5] P. Dutta, M. Grimmer, A. Arora, S. Bibyk, andD. Culler. Design of a wireless sensor networkplatform for detecting rare, random, and ephemeralevents. In he Fourth International Conference onInformation Processing in Sensor Networks (IPSN’05), pages 497–502. IEEE, 2005.

[6] P. K. Dutta and D. E. Culler. System softwaretechniques for low-power operation in wireless sensornetworks. In Proceedings of the 2005 InternationalConference on Computer-Aided Design, 2005.

[7] Garcia.http://www.acroname.com/garcia/garcia.html.

[8] J. Gross and J. Yellen. Graph Theory and ItsApplications, Second Edition. Chapman & Hall/CRCPress, 2005.

[9] A. K. Gupta, S. Sekhar, and D. P. Agrawal. Efficientevent detection by collaborative sensors and mobilerobots. In First Annual Ohio Graduate StudentSymposium on Computer and Information Scienceand Engineering, 2004.

[10] N. Heo and P. K. Varshney. A distributed selfspreading algorithm for mobile wireless sensornetworks. In Wireless Communications andNetworking. IEEE, 2003.

[11] A. Jacoff, E. Messina, B. A. Weiss, S. Tadokoro, andY. Nakagawa. Test Arenas and Performance Metrics

for Urban Search and Rescue Robots. In Proceedingsof the 2003 IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS), 2003.

[12] K. Konolige, C. Ortiz, and R. Vincent. Centibots largescale robot teams. In AAMAS, 2003.

[13] K. Kotay, R. Peterson, and D. Rus. Experiments withrobots and sensor networks for mapping andnavigation. In International Conference on Field andService Robotics, 2005.

[14] J. Kurose and K. Ross. Computer Networking: ATop-Down Approach Featuring the Internet. PearsonEducation, 2005.

[15] LEGO. http://mindstorms.lego.com/.

[16] K. Pister.http://robotics.eecs.berkeley.edu/ pister/SmartDust/.

[17] A. Rogers, E. David, and N. R. Jennings.Self-organized routing for wireless microsensornetworks. IEEE Transactions on Systems, Man, andCybernetics—Part A: Systems and Humans, 35(3),2005.

[18] U. Wilensky. NetLogo.http://ccl.northwestern.edu/netlogo, 1999.

24

Agent Technologies for Post-Disaster Urban Planning

Jean OhLanguage Technologies

InstituteCarnegie Mellon University

Pittsburgh, PA

jeanoh@cs.cmu.edu

Jie-Eun HwangGraduate School of Design

Harvard UniversityCambridge, MA

jhwang@gsd.harvard.edu

Stephen F. SmithRobotics Institute

Carnegie Mellon UniversityPittsburgh, PA

sfs@cs.cmu.edu

ABSTRACTUrban planning is a complex decision making process whichmust compensate for the various interests of multiple stake-holders with respect to physical, social, and economic con-straints. Despite growing interest in using A.I. in urbandesign and planning this community remains a field domi-nated by human experts. Recent catastrophic disasters suchas hurricane Katrina, however, have underscored the needfor increased automation and more efficient urban designprocesses. One particularly urgent decision making in post-disaster urban planning is that of finding good locations fortemporary housing. As an illustrative example of the poten-tial of agent technologies in post-disaster planning contextswe propose an agent-based decision support system that canidentify good candidate locations for a specific purpose. Weshowcase an application of our decision support system inpre-disaster mode that identifies a set of ideal locations forpotential revitalization. We then discuss how this systemcan be extended to solve a problem of finding good locationsfor temporary housing in post-disaster mode. Our prelimi-nary experimental results show promising potential of usingagent technologies towards solving real life problems in theurban planning domain.

Categories and Subject Descriptors[Decentralized agent-based architecture]; [Multiagentlearning]

KeywordsUrban planning, decision support systems, machine learn-ing, intelligent survey

1. INTRODUCTIONRecent catastrophic disasters have brought urgent needs fordiverse technologies for disaster relief. In this paper weexplore opportunities of A.I. research for solving real-lifeproblems in aid of post-disaster recovery and reconstruction.Among various complex problems in post-disaster situations

we mainly focus on reconstruction of the community, specif-ically from the urban planning perspectives.

Urban planning is a complex decision making process whichmust compensate for the various interests of multiple stake-holders with respect to physical, social, and economic con-straints. Planners need to collect and thoroughly analyzelarge amounts of data in order to produce robust plans to-wards both short-term and long-term goals. This is normallya careful and time-consuming task, due in part to limitedfinancial resources but also because design decisions oftengenerate cascading effects contingent on both pre-existingphysical urban structures and future design decisions. Re-solving the conflicting interests of multiple entities has beenan important issue in urban design decision making. Par-ticularly in the post-disaster planning case, understandingpersisting local constraints as well as the issues newly in-troduced by the crisis is a key to a successful recovery andreconstruction plan, i.e., a good coordination among vari-ous stakeholders is a necessity. In reality, however, a lotof necessary coordination is conducted only at a superficialdepth. Due to limited time and resources, many importantdecisions are made by high level officials and various stake-holders’ responses are collected subsequently, often throughhasty paperwork.

Although agent-based modeling is gaining popularity in ur-ban planning research community [12, 1] little has been donefor domain experts to recognize benefits of utilizing agenttechnologies in this domain, and this domain still remainsa field strictly dominated by human experts. Recent catas-trophic disasters such as hurricane Katrina, however, haveunderscored the need for increased automation and moreefficient urban design processes.

In pre-disaster mode planning tasks are ordered by prior-ity and resource availability and only a small number oftasks are handled at a time. In the post-disaster situation,however, an overwhelming number of high priority tasks areproduced overnight and planners must make thousands ofcomplex decisions in a very short time. Various types ofnew and updated information, such as damage assessmentand resource availability, arrive in an arbitrary order anddecisions must be made dynamically. It is unlikely that allof the necessary information is available at the time of deci-sion making, thus decision support systems that can providetimely data estimation and inference capability are desper-ately desired.

25

One good example of the kind of decision making that couldbenefit from the timely assistance of autonomous agents isthe problem of finding good locations for temporary housingafter crisis. Location hunting is a complex constraint opti-mization problem that must compensate for various case-specific local constraints as well as a set of well-defined legalconstraints, such as NEPA (National Environmental PolicyAct) guidelines. Due to the urgency of the task and lim-ited resources, candidate selection is hurriedly made, payinglittle attention to many crucial local constraints.

In this paper we focus on the specific task of location find-ing in urban planning as our initial target problem. In par-ticular, our system model is based on urban typology prac-tice, which is a typical methodology in the urban planningdecision making process that classifies urban componentsaccording to their various structural and socioeconomic as-pects. We present an agent-based framework that utilizesmachine learning for intelligent decision support in this do-main, and consider applications for both pre-disaster andpost-disaster urban planning problems. First, we presentan example application of finding good locations for poten-tial revitalization in urban planning in pre-disaster mode.Our preliminary experiments show promising results thatagent-based approach can boost the performance of urbanplanning. We then propose how to apply the same frame-work to the problem of finding good locations for temporaryhousing in post-disaster mode, and discuss further issues sit-uated in a distributed environment of a larger scale disastermanagement.

2. DISTRIBUTED DECISION SUPPORT SYS-TEMS

An agent is an autonomous entity that can make decisionsthrough its own reasoning process. The reasoning criteriacan be as simple as a set of precoded rules, or a complexutility function to be used to trade off various options. Inthe problems of interest in our research the purpose of anagent system is to assist human users in such a way that theagent acts as if it is a shadow play of its human master bylearning the user’s decision criteria.

An assistant agent that is customized to a specific humanuser can perform certain tasks on behalf of the user. For ex-ample, calendar management agents can free up busy usersso that the users can spend time more efficiently on serioustasks. CMRadar [10] is a distributed calendar schedulingsystem wherein individual CMRadar agents assume respon-sibility for managing different user’s calendars and negotiatewith other CMRadar agents to schedule meetings on theirusers’ behalf. A CMRadar agent learns its master user’sscheduling preferences using passive machine learning algo-rithms only through observing several meeting schedulingepisodes.

Unlike the meeting scheduling problem, where each partici-pant is treated more or less equally important, many impor-tant decisions are made exclusively by a group of author-ities in post-disaster mode due to the urgency of pressingissues. Many case studies emphasize the importance of in-volving local community residents in decision making[13],thus efficient methods of incorporating local objectives andconstraints have been sought. We propose a distributed de-

cision support system that can provide better insights todecision makers by learning representative decision modelsfor a specific issue by means of an intelligent survey system.Whereas personal assistant agents have convenient access tothe user’s daily activities that provide training data for pas-sive learning methods, a representative agent system mustactively participate in learning process in order to collect ge-ographically distributed training data. In the next sectionwe illustrate a high level architecture of a representativeagent system.

3. REPRESENTATIVE AGENTSDiverse interest groups are involved in the urban planningdecision making process. In pre-disaster mode, we considerfour major groups of people: urban planners (designers),government officials or other related authority groups, in-vestors, and community residents. It is often true that thevoice of actual community residents is weak due to two mainreasons: 1) lack of a representative organization, and 2) diffi-culty of collecting their broad needs and constraints. Com-mon ways of collecting such opinions are passive methodssuch as voting and surveying. In pursuit of a better balanceamong various stakeholder groups, e.g., by raising the voiceof community residents, it would be ideal to have represen-tative agents that can quickly learn the decision model of agroup of people given a specific issue, e.g. whether a givenlocation is a good site for temporary group housing.

A survey is a traditional method of estimating the opinionsof a large group of people by asking predefined question-naires to a group of randomly selected people. A surveyprovides a snapshot of collective opinions of a group for aspecific issue, but often limited to high-level questionnaires.We attempt to induce more general decision criteria for lo-cation specific issues by linking a survey with physical andsocioeconomic information that is associated with the regionunder consideration.

We have designed RAISE (Representative Agents in Intel-ligent Survey Environment), an agent-based survey systemthat learns a representative model of a large group of peoplefor a location specific issue. We aim to take advantage ofvast amounts of local information available from various GISinformation sources and high-performing machine learningalgorithms to efficiently utilize such data in conjunction withan intelligent survey system. As opposed to using staticquestionnaires we also use an active learning algorithm thatinteractively chooses more informative examples as the nextquestions to ask to guide the learning process.

Figure 1 illustrates a high level architecture of RAISE. Thetarget problem of RAISE is supervised learning in a dis-tributed environment which contains two distributed sub-problems: 1) data is distributed in multiple sources, and2) labeling is conducted by multiple people through varioustypes of user interface.

RAISE provides two types of agents, information agents andsurvey agents, in order to address each subproblem, respec-tively. Information agents collect data from various sourcesto produce a data set that can be used by the learning com-ponent. A large amount of urban planning data is availablein GIS (Geographic Information System) data format from

26

DB

DB

WWW

Active Learner

RAISE

WWW

Surveyagents

Informationagents

Web

Mobile devices

GIS

Inference engine

Figure 1: RAISE (Representative Agents in Intelli-gent Survey Environment) architecture

various information sources. GIS is a powerful tool that in-tegrates a geographic map with semantic information usinga multi-layered structure. Internally, these layers of infor-mation is stored in a relational database.

The most crucial task of RAISE information agents is dataintegration from multiple information sources. For instance,if some subsets of information sources need to be alignedmultiple information agents must coordinate with one an-other in order to produce a seamlessly integrated data set. Inaddition, agents must be able to learn to recognize more re-liable information sources because some information sourcesmay contain conflicting data.

Another important class of agents are survey agents. Fromthe learning component’s perspective survey agents are theentities that provide correct labels for a given unlabeled dataexample. The level of expertise varies depending on subjectgroups participating in a survey. The way of presenting adata example as a question in a survey to human subjectsis an important user interface research issue. For instance,just a set of numeric values in raw form is obviously not agood representation of an architectural component, such asa building, even to domain experts.

Community residents might be able to identify a given en-try just by the name of a building or visual informationsuch as a picture of the building. They make decisions us-ing their local knowledge as opposed to what the systempresents as features. In other words, the features used bynon-expert users are unknown to the system. Hypotheti-cally, we assume that the feature space modeled based ondomain knowledge can represent a decision model that isequivalent to the user’s decision model containing hiddenfeatures. We illustrate this issue again in section 4.1 usinganother example.

Domain experts, such as urban planners, would want to seemore detailed information in addition to what is needed formere identification, e.g., land use code, number of tax en-tries, whether the building is used for multiple commercialpurposes, etc.

• Location of temporary housing

• Sitting of temporary business location

• Sites for dumping disaster debris

• Road closure and reopening

• Bridge closure and reopening

• Restoration of critical infrastructure

• Permitting the reoccupation of damaged homes

Table 1: Short-term decision making issues

The necessity of decision support systems in this domainis far greater in post-disaster mode than normal mode dueto the importance of safety issues and urgency of emergenttasks. The target problems we try to solve using RAISEafter a crisis are short-term planning solutions with care-ful consideration of long-term reconstruction goals. Someexamples of short-term decision making problems are listedin Table 1. In this paper, we target a specific example ofshort-term decision making problems: location hunting. Forinstance, one of the most urgent problems in post-disastersituation is identifying a set of good sites for temporary man-ufactured housing such as trailers. Since temporary housingsites tend to remain longer than the initially intended pe-riod, the location must be carefully chosen and must notinterfere with long-term reconstruction.

The short-term issues in Table 1 are directly related to com-munity’s daily activities thus it is crucial to incorporate com-munity residents’ opinions. Ironically, those people who ac-tually live in the community are often ignored when a deci-sion is being made. In hope of raising the voice of communityresidents we propose an agent-based system, RAISE, thatcollects data from multiple information sources and learns arepresentative decision model of community residents in theform of an interactive survey.

4. URBAN DESIGN PLANNING PROBLEMSThe integrated perspective of form and function in urbanstudies is not an innovative notion. In fact, it has beenthe core subject of urban matters for a long time [4], Pre-vious work, however, has primarily focused on one domi-nant aspect of either form or function from a particular viewpoint, e.g. architecture, psychology, sociology or economics.Furthermore, the range and definition of form and functionvaries according to diverse disciplines. For instance, whilearchitects regard form as three dimensional shape of spaceand building components in the intimate detail, economistsrather view it as two dimensional shape of cartographicplane at the regional or national scale. Architects considerfunction as activities in individual building spaces and thein-betweens, whereas policy makers consider function as per-formance of parcel or zone in the whole system of the city.

Resolving multiple views has been an important issue inurban design decision making. The urban design profes-sion contributes to shape the city through designing physi-cal structures; however, it has generally been an execution of

27

form-based policy in this respect [8]. Recognizing the impor-tance of considering interdisciplinary aspects of a problem,urban designers have developed methodological frameworksto investigate urban morphology in a manner that combinesinterdisciplinary aspects [11]. Our research contributes tothis effort, by applying AI techniques to develop improvedrepresentations and methods for reasoning about urban de-sign issues in an integrated fashion. We focus on an impor-tant methodological framework, typology, which representsthe understanding of urban settings by classification basedon present architectural and socioeconimic elements [4].

In general, urban typology analysis is a long term projectthat requires careful data analysis and field studies. Forinstance, the ARTISTS (Arterial Streets Towards Sustain-ability) project in Europe was developed to identify typesof streets in order to provide better insights to urban plan-ners and economists. This 2.2 billion euros budget projectinvolved 17 European countries and took three years to clas-sify five categories of streets [15]. Their major contributionincludes statistical analysis of street functions and summa-rization of results in a two-dimensional classification tablethat can be used as a general decision criteria. Althoughtheir classification rules were drawn from statistical analysishuman experts were the main forces of this project. Theexperimental results show how they classified 48 streets into5 categories based on their decision rules. Our attempt is tocarry out similar classification task but in an automated wayusing machine learning techniques in the hope of assistingdecision makers heavily loaded with urgent tasks.

We project a typical typology analysis into a simplified three-step process: data analysis, field study, and decision mak-ing. Among these three steps, the field study is the mostexpensive procedure in terms of both labor cost and time.Our experiment shows potential usage of machine learningtechniques in urban typology problems. We also stress thatactive learning algorithms are especially beneficial by reduc-ing the number of labeled examples in training phase. Inpractice, this means labor cost is reduced by avoiding lessinformative field studies.

Supervised machine learning techniques have been success-fully applied in various domains such as text categorization[17]. Most of machine learning algorithms expect data to bea well defined set of tuples, but in reality this is rarely thecase. For example, if data is stored in relational databasewith multiple tables the data must be preprocessed into agiant single table. Building an inference network from a re-lational database is an interesting area of research [6] andwe also anticipate that our future work may be in this area.For the sake of simplicity we assume in what follows that wealready have the data formatted into a set of tuples in ourexperiments.

4.1 ModelingModeling an urban typology as a machine learning problemis based on two important assumptions: 1) a set of relevantfeatures that define an input to a learning algorithm areknown in advance, and 2) data that describe the features area well-structured set of vectors. Applying machine learningalgorithms to a well defined set of data is a straightforwardtask. However, a major difficulty of formulating urban ty-

Public-ness

BuiltForm Function

Use Patterns

Streetscape Building

Massing

Lot(Parcels)

Frontage Sign

Yard Entrance Front Transparency

Type of Signage

AbstractClass

Semantic Class

Feature From DB

Feature By User

Feature Intangible

Popularity

Business Type

Height, Area,

Periphery,Distance,

.

.

ArchitecturalStyle

Types of User

Groups

Types of Activities

Population of People

Quality of Maintenance /

Service

Awareness of Content

Distance,Area,

Vegetation,..

Num of Door,Stair Size,

.

.

Num of Window,

Dimension of Windows,Material of Windows

Location,Size,

Material...

Visibility

Bold Line : User Annotatable

Legends

Figure 2: Features determining public-ness of urbancomponent

pology into a machine learning problem resides in featurespace modeling and compiling a set of relevant data.

The human experts’ elicitation of relevant features is oftenvague and incomplete. We exemplify a modeling of fea-ture space in Figure 2. This example depicts the featuredependency graph that represents a perception of public-ness. Public-ness is a meaningful concept in urban designand relates to how people perceive whether a given urbancomponent is public or private. We modeled this examplebased on a survey that was given to both domain expertsand non-experts. Although this example does not directlyaddress our specific target problem of location finding thefeatures in the graph, such as Massing, are commonly usedas urban decision criteria, and thus they are relevant to ourdiscussion.

Among these features the entries that are drawn in bold-face in Figure 2 are the set of features that users consid-ered important in decision making. Because the system canonly recognize well-structured data, e.g., features stored indatabases, only the features shown in grey are included inour model. This example illustrates our modeling assump-tion that domain experts’ model of relevant features are of-ten abstract semantic concepts that depend on descriptivefeatures that are available in low level databases.

Massing, for instance, is a feature that differentiates build-ings by their structural size information. In our informationsources Massing is represented as multiple features, height,area, periphery, distance to nearest neighbor, etc. Our sur-vey result also reveals the existence of hidden features thatare completely isolated from what is available in low leveldatabase. These hidden features were denoted by intangi-ble features in the picture, e.g., features related to ”UsePatterns”. We learn from this example that a majorityof features in a human user’s model are abstract concepts,whereas the system only has access to low level databases.We make a specific assumption that abstract concepts thathuman experts consider relevant in fact depend on low level

28

Commonfeatures

number of buildings, land use, buildingheight, perimeter, lot size, stories, shapelength, shape area, gross area, living area

MainStreets

parcel business type, built year, renova-tion year

TemporaryHousing Site

cost, past land use history

Table 2: Available features for site selection

features in databases. We also assume that the system hasaccess to such domain specific information sources. Thechallenge then is to infer the mapping from low level fea-tures to abstract concepts.

5. FINDING MAIN STREETSThis section describes our preliminary experiment on a pro-totypical example of location finding process to demonstratethe efficiency of using A.I. techniques in this problem do-main. We chose the specific problem of identifying a certaintype of urban setting, Main Streets, based on architecturaland socioeconomic features of its vicinity. Although thismay appear a semantically different problem we note thatpost-disaster location hunting is conducted through a similarprocedure when selecting potential candidate sites suitablefor different purposes. Some examples of common featuresthat are used for site selection for different purposes arelisted in Table 21 .

The concept of Main Street Approach is introduced from thecity revitalization projects dated back in 1970s, which wasan attempt to identify commercial districts that have po-tentials for revitalization. The idea behind this wave was tocombine historic preservation with economic development torestore prosperity and vitality to downtowns and neighbor-hood business districts. Suffering from declined prosperityagainst the regional mall and rapid urban development [7],Main Street became the major issue of community plan-ning. The criteria of choosing a right commercial districtvaries from city to city, thus it is hard to find a generalizedset of rules to distinguish Main Streets from rest of districts.Since one cannot apply one standard that works on a cityto another a local organization is entitled to perform theirown data analysis for each city.

The Main Street approach is, as many urban design subjectsare, usually pursued in partnership between public and pri-vate sectors. For instance, the city of Boston has the MainStreet program in the department of neighborhood develop-ment in the city hall. Such a team of public sectors collab-orates with private sectors, e.g., local Main Street directorswho are usually elected or hired by the community. In acity or regional level, the Main Street is a vital strip withinthe whole vessel network of the city. At the same time, theMain Street is the center of the local area in a local neigh-borhood level. Since each Main Street has unique character-istics and problems identified by the neighborhood in whichit belongs, it is important to understand and identify thelocal context of the community. Additionally, along withthe consideration of historical preservation, the Main Street

1This list contains only the features that are availablethrough local GIS information sources.

Figure 3: Main Streets in Boston, Massachusetts

approach conveys reallocation of existing architectural andsocioeconomic resources, as opposed to urban renewal, inthe neighborhood.

Accordingly, Main Streets raise an important issue that stemsfrom the complexity of communications among multiple ac-tors. The set of actors involved in Main Street design processincludes city officials, local directors, design professionals,communities, developers, investors, etc. The key to a suc-cessful Main Street design lies in resolving diverse interestsand constraints of multiple actors from architectural, social,economic, and historical perspectives. We propose a system-atic way to work out the ”multiple views” problem of urbantypology by providing an intelligent decision support systemthat can learn various actors’ typology decision criteria.

We showcase a framework for domain experts to interac-tively classify Main Streets in the city of Boston (Figure 3).Boston provides an ideal testbed for evaluation because acomplete set of ideal districts were already identified as MainStreets by field experts. We used relational database tablesexported from GIS information sources that are availablefrom the city of Boston. The data was then preprocessed tobe suitable for general classifiers. Initially we started withtwo database tables: buildings and parcels. Note that adata entry in these tables represents a building and a par-cel, respectively, whereas our target concept, Main Streets,is defined as a district which is usually composed of severalhundreds of buildings and parcels.

First, we applied unsupervised learning methods to groupbuildings and parcels into a set of candidate districts. Weused a single-linkage clustering algorithm in which everydata point starts with a separate cluster and merges with theclosest neighboring cluster until a given proximity thresholdis satisfied. The proximity threshold was chosen empiricallyto generate reasonable size clusters.

Our algorithm for identifying district candidates consists oftwo clustering steps. Since the backbone of Main Streets isa strip of commercial buildings we first clustered buildingsthat are associated with commercial land use code in or-der to retrieve strips of commercial buildings. At this step,small clusters that contained less than 5 commercial build-

29

ings were filtered out. In the second step, the commercialstrips identified in the first step were treated as a single clus-ter when the second round of clustering started, i.e., the setof initial clusters in the second round was the union of com-mercial strips, non-commercial buildings, and all of parcels.The number of buildings and parcels in the resulting districtcandidates were in the range of hundreds.

For simplicity, we used Euclidean distance between the twocenters of buildings as the distance measure. In order to re-fine cluster boundaries we need to incorporate more accurateseparator data, e.g., geographic obstacles such as mountainsor rivers, and man-made obstacles such as bridges and high-ways. This will be an interesting topic for a future work.Using a raw data set containing 90,649 buildings and 99,897parcels (total around 190,000 data points) our algorithmidentified 76 candidate districts. Each candidate cluster cor-responded to one data row for a classifier, and aggregatedcharacteristics of a candidate cluster, such as average heightof the buildings, were used as features.

In our initial experiment, we tried a set of classifiers to de-termine the best-fitting classifier in our particular problemsolving. Among a set of Decision Trees, a Nave Bayes clas-sifier, a kNN (k-Nearest Neighbors) classifier, and an SVM(Support Vector Macine) classifier, an SVM classifier bestperformed [17]2. In general, SVM is considered one of thebest performing classifiers in many practical domains. De-spite SVM’s high quality performance users outside A.I.,such as designers, tend to prefer Decision Trees or genera-tive models due to the fact that their results are more com-prehensible. As a proposed resolution for explaining SVMresults to human users we learn a decision tree that is equiv-alent to the learned SVM classifier in terms of classificationresults on the test set. That is, after training an SVM clas-sifier using a set of training data the system labels the re-maining set of data with SVM’s prediction. Finally we traina decision tree using the original set of training data plusthe remainder of data labeled by the learned SVM.

Interfacing a classifier with human users introduces manyinteresting research issues in both ways, i.e., from humanusers to classifiers and from classifiers to human users. Forinstance, difficulty of explaining the rationale of classifier tohuman users is described in the SVM example above. Itis also an interesting issue how to tell the system domainexpert’s “tips”. One simple way is to generate simulatedtraining examples based on the rules given by human expertsand retrain the system using augmented training data.

Labeling is an expensive process in this domain because la-beling one district requires thoughtful analysis of huge dataand it further involves field study. This cost-bounded do-main constraint leads us to favor learning algorithms thatwork well with relatively small number of training examples.One such idea is active learning in which learning system ac-tively chooses the next training example to be labeled. Wetook Tong and Koller’s approach over SVM [16]. The basicidea is to suggest data points that are near the separationboundary, which is quite intuitive and is also proven to be

2Due to limited space we omit formal definitions of vari-ous classifiers and refer to Yang’s work [17] that extensivelyevaluates various types of classifiers.

Figure 4: Active learning algorithm vs. Randomizedalgorithm

very effective in other practical domains such as text classi-fication.

Semi-supervised learning is another approach that is usefulwhen the number of labeled data is small. This approachutilizes distribution of a large amount of inexpensive unla-beled data to guide supervised learning. For example, co-training method [2] learns two classifiers using disjoint setsof features, i.e., two different views over the same data, andadmits only those predictions upon which both classifiersagree. A more recent approach includes incorporating clus-tering into active learning [9]. Using prior data distributiontheir system first clusters data and suggests cluster repre-sentatives to active learner. Their algorithm selects not onlythe data points close to classification boundary but also rep-resentatives of unlabeled data. We adopted their idea to findthe initial samples to be labeled. This technique, however,didn’t make much difference in our experiment mainly be-cause the size of unlabeled data was not large enough (Afterpreprocessing we had only 76 district candidates). We wouldexpect higher impact on performance if we had a larger setof data.

We used precision, recall, and their harmonic mean as eval-uation metrics. In our example, precision p is the ratio ofthe number of correctly identified Main Streets to the totalnumber of trials. On the other hand, recall r is the ratioof the number of correctly identified Main Streets to thetotal number of Main Streets in Boston. Because the twomeasures are in inverse relation their harmonic mean is of-ten used as a compromising measure. F1 measure, whichis a harmonic mean of precision p and recall r is defined inequation (1).

F1 =2pr

p + r(1)

Since we had a relatively small sized data set after pre-processing we used Leave-One-Out-Cross-Validation (LOOCV)to evaluate the general performance of Main Streets classi-

30

Precision Recall F1 measureLOOCV 0.842 0.762 0.800

Table 3: Leave-One-Out-Cross-Validation Result

fier. LOOCV is a cross validation technique where one datapoint is left for testing while a classifier is trained using therest of data points. The LOOCV results in Table 3 showspromisingly good performance by achieving high F1 mea-sure of 0.8. The results read that the system made 6 correctpredictions out of every 7 trials, identifying 76% of MainStreets.

We also compared the performance of the active learningstrategy to the performance of the random learning strat-egy. Under the random learning strategy the system alsolearns an SVM classifier by incrementally taking more train-ing examples. Whereas the active learning strategy takesadvantage of the distribution of unlabeled data in selectinga next data point, the random learning strategy chooses anarbitrary data point. We evaluated the performance of thetwo approaches in terms of their learning speed.

Figure 4 shows the performance of active learning strategyand random learning strategy. The experimental results inFigure 4 are average performance over a set of 20 indepen-dent trials. The experimental results first indicate that find-ing Main Streets is a class of urban design decision makingproblems that can be developed by using a machine learn-ing approach. The results also show that the active learningalgorithm significantly3 outperforms the random learning al-gorithm, achieving high classification accuracy after given arelatively small number of examples.

6. LOCATION HUNTING FOR TEMPORARYHOUSING

At an abstract level, the decision making process in post-disaster mode is not different from the pre-disaster mode.Planners seek good solutions that optimize the interests andconstraints of multiple entities. The scale of the problem,however, is far greater. There are several important fac-tors that increase the difficulty in post-disaster mode. Firstand foremost, time is precious. Fast temporary recoveryis desired, but short-term solutions must be in harmonywith long-term reconstruction plans. Second, the load oftasks is overwhelming, for instance, over 150,000 propertieswere damaged or destroyed as a result of hurricane Kat-rina in 20054. Third, a much larger group of entities areinvolved due to crisis, including external aid groups suchas emergency management team, telecommunication ser-vices, transportation services, utility services, education sys-tems, economic development agencies, environmental agen-cies, etc. Fourth, it is unlikely that planners have all re-quired information at hand. Damage assessment is part ofon-going process while planning for reconstruction is beingdone. The planning team should expect dynamic updateof information thus robustness and flexibility should be in-cluded in planning objectives.

3This is statistically significant with a strong evidence ofp-value 0.01.4This is based on the estimate made by RMS (Risk Man-agement Solutions) on September 2, 2005.

Demand for temporary housing in that areaSite topographyProperty owner willingnessCostPast land useExistence of conflicting redevelopment plansAccess to existing utilitiesEngineering feasibilityEnvironmental/cultural resource sensitivities

Table 4: Temporary Housing Site Selection Criteria

Providing temporary housing for those who have been dis-placed in the aftermath of disasters is one of the most urgentissues in disaster management. When the demand for emer-gency housing exceeds what existing housing facilities canaccommodate, new temporary housing sites are constructedfor a group of manufactured homes and mobile trailers, e.g.FEMAville – FEMA (Federal Emergency Management As-sociation) trailer park.

Six months after hurricane Katrina only half of around 130,000requests for temporary manufactured housing and mobiletrailers were fulfilled, leaving tens of thousands of residentswithout a place to live [14, 5]. The major problem was notin the shortage of trailer supply, but in the failure to findproper locations to install the trailers. In addition, the poorquality of lot specification on paperwork hindered the instal-lation process, dropping the daily installation rate down to65%. A more fundamental problem that has been seriouslycriticized is rooted in the lack of public involvement, i.e., theopinions of local community residents were not reflected indecision making [3].

As shown in the failure of the Katrina temporary housingproject, finding good locations for emergency group housingis a complicated problem. First, designated officials suchas FEMA’s contractors choose a set of candidate sites byreviewing local information: aerial photos, maps, site recon-naissance field surveys, and local officials’ comments. Fac-tors considered in selecting a site are listed in Table 4 [5].For a selected site that satisfies the site selection criteria anin-depth analysis of Environmental Assessment (EA) is con-ducted before a final decision is made. Usually a completeEA is limited to one or two sites at a time due to limitedresources and the searching for alternative sites continues inparallel. The result of EA is either a positive confirmationthat the construction of temporary housing in the selectedlocation does not have significant impact on surrounding en-vironment, or a rejection due to potentially significant im-pact. The resulting EA reports are posted for public re-sponse, but only for a brief period of time, e.g., typically 2days, due to emergency nature of this action. It has alsobeen criticized that expertise of local community membershas been poorly incorporated in site selection process.

We design another application of RAISE to assist the siteselection process. As we have shown in the Main Streets ex-ample, we can model this temporary housing site selection asa distributed classification problem. The major difficulty inmodeling urban planning problem as a machine learning tasklies in feature space modeling and availability of relevant

31

data. In order to address the multiple views problem fur-ther we model RAISE agents for three stakeholder groups:government officials who make final decisions, disaster vic-tims who needs emergency housing, and property owners.The government officials are working on behalf of disastervictims to maximize social welfare, thus they need to coor-dinate to understand supply and demand of each other. Theproperty owners in this model have priority to act selfishlyto maximize their own benefits. In fact, the failure of theKatrina temporary housing project is attributable to suchselfish actions, the so called NIMBY (not in my backyard)problem. We aim to help resolving this problem with a mul-tiagent system approach by assisting policy makers to designa better mechanism.

7. CONCLUSION AND DISCUSSIONRecent disasters have brought increased concerns for post-disaster recovery and reconstruction. The baseline mottoduring planning for post-disaster recovery is that post-disasterplanning is an extension of a long-term community develop-ment plan, thus, incorporating local information and thecity’s comprehensive plan is the key to successful planning.

Although it is easy to consider post-disaster planning as anindependent task case study shows that post-disaster recov-ery plans that are well integrated with community’s com-prehensive plan are more effective in finding creative solu-tions [13]. In addition, it provides opportunity to utilizeresources more efficiently in order to contribute to prob-lem solving in a larger picture. For example, sometimesscare resources suddenly become available after the disasterand good plans maximize resource utility by identifying longwaiting tasks that have been in the queue for these scare re-sources. Post-disaster planning also provides opportunitiesto fix existing problems due to previous suboptimal planningdecisions. The decision making policy of designated emer-gency managers, such as FEMA officials, is primarily basedon safety and urgency of tasks. They develop their own ur-gent operations that are focused on immediate response andrecovery functions following a disaster. However, local com-munity’s coordination with emergency managers is crucialfor successful plans, because community members are theones who actually monitor and implement the plans.

In this paper we discussed agent-based modeling of urbanplanning problems both in pre-disaster mode and post-disastermode. We presented a framework, RAISE, to build a rep-resentative agent in the form of an intelligent survey sys-tem. Our preliminary experiment on a location predictionproject, Finding Main Streets, provides a good showcaseexample of the opportunities that agent technologies pro-vide towards solving real life problems, in particular in post-disaster management problems.

8. ACKNOWLEDGEMENTSThe authors thank Yiming Yang for fruitful discussions onthe Main Streets project. This research was sponsored inpart by the Department of Defense Advanced Research ProjectsAgency (DARPA) under contract #NBCHD030010.

9. REFERENCES[1] I. Benenson and P. Torrens. Geosimulation:

Automata-Based Modeling of Urban Phenomena. John

Wiley & Sons, 2004.

[2] A. Blum and T. Mitchell. Combining labeled andunlabeled data with co-training. In COLT: Proceedingsof the Workshop on Computational Learning Theory,Morgan Kaufmann Publishers, pages 92–100, 1998.

[3] J. S. Brooks, C. Foreman, B. Lurcott, G. Mouton, andR. Roths. Charting the course for rebuilding a greatamerican city: an assessment of the planning functionin post-katrina new orleans. American PlanningAssociation, 2005.

[4] G. Caniggia and G. Maffei. Architectural Compositionand Building Typology: Integrating Basic Building.Alinea Editrice, Firenze, Italy, 1979.

[5] FEMA. Environmental assessment, emergencytemporary housing, hurrican katrina and rita.Technical report, Edgard, Saint John the BaptistParish, Louisiana, 2005.

[6] L. Getoor. Learning Statistical Models from RelationalData. PhD thesis, Stanford University, 2001.

[7] J. Jacobs. The death and life of great American cities.Modern Library, New York, 1993.

[8] A. Krieger. Territories of Urban Design. HarvardDesign School, 2004.

[9] H. T. Nguyen and A. Smeulders. Active learning usingpre-clustering. In Proceedings of InternationalConference on Machine Learning, 2004.

[10] J. Oh and S. F. Smith. Learning User Preferences inDistributed Calendar Scheduling. Lecture Notes inComputer Science, 3616:3–16, 2005.

[11] B. P. Representation of places: reality and realism incity design. University of California Press, Berkeley,California, 1998.

[12] D. C. Parker, S. M. Manson, M. A. Janssen, M. J.Hoffmann, and P. Deadman. Multi-agent systems forthe simulation of land-use and land-cover change: Areview. In Annals of the Association of AmericanGeographers, 2002.

[13] J. Schwab, K. C. Topping, C. D. Eadie, and R. E.deyle amd Richard A. Smith. Planning forPost-Disaster Recovery and Reconstruction. AmericanPlanning Association, 1998.

[14] J. Steinhauer and E. Lipton. Storm victims face bigdelay to get trailers. The New York Times, February9, 2006.

[15] A. Svensson. Arterial Streets For People. Technicalreport, Lund University, Department of Technologyand Society, Sweden, 2004.

[16] S. Tong and D. Koller. Support vector machine activelearning with applications to text classification. InP. Langley, editor, Proceedings of 17th InternationalConference on Machine Learning, pages 999–1006,Stanford, 2000. Morgan Kaufmann.

[17] Y. Yang. An evaluation of statistical approaches totext categorization. Information Retrieval,1(1/2):69–90, 1999.

32

Point to Point Vs Broadcast Communication for ConflictResolution

Alessandro FarinelliDipartimento di Informatica e

SistemisticaUniversity of Rome “La

Sapienza”Via Salaria 113

00198 Rome, Italy

farinelli@dis.uniroma1.it

Luca IocchiDipartimento di Informatica e

SistemisticaUniversity of Rome “La

Sapienza”Via Salaria 113

00198 Rome, Italy

iocchi@dis.uniroma1.it

Daniele NardiDipartimento di Informatica e

SistemisticaUniversity of Rome “La

Sapienza”Via Salaria 113

00198 Rome, Italy

nardi@dis.uniroma1.it

ABSTRACTTask Assignment for Multi-Robot Systems is a main issue to attaingood performance in complex real world environments. In severalapplication domains tasks to be executed are not inserted into thesystem by an external entity but are perceived by robots during mis-sion execution. In this paper we explicitly focus on detecting andsolving conflicts that may arise during the task assignment process.We propose a conflict detection method based only on point to pointmessage. The approach is able to guarantee a conflict free alloca-tion using a very limited communication bandwidth. Moreover, wepresent an approach to make the system robust to possible networkfailures.

1. INTRODUCTIONCooperation among robots is nowadays regarded as one of the

most challenging and critical issues towards fieldable robotic sys-tems. A central problem for achieving cooperation in Multi RobotSystems is Task Assignment, i.e. the problem of decomposing thetask faced by the system into smaller sub-tasks, and ensure thatthey can be accomplished by individual robots without interferenceand, more generally, with better performance.

Task Assignment has been deeply investigated in both Multi AgentSystems (MAS) and Multi Robot Systems (MRS) [2–4, 6, 9] andseveral successful approaches have been proposed. However, thegrowing complexity of applications makes it desirable to improvecurrent approaches to Task Assignment, in order to suitably dealwith more and more challenging requirements: dynamic task evo-lution, strict bounds on communication and constraints among tasksto be executed. But, most notably, in real world applications involv-ing MRS, tasks to be assigned cannot be inserted into the systemin a centralized fashion: they are perceived by each robot duringmission execution.

For example, let us consider an heterogeneous MRS involved ina search and rescue task. Robots are equipped with different sen-sors such as color cameras, laser range finders, infra red sensors.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.AAMAS’06 May 8–12 2006, Hakodate, Hokkaido, Japan.Copyright 2006 ACM 1-59593-303-4/06/0005 ...$5.00.

The tasks that robots should perform can be of various kind, let usconsider that the task of the whole system is to explore the envi-ronment and analyze detected objects. Robots need to cooperateto correctly spread over the environment and share significant in-formation (e.g. known map, detected objects). Sub-tasks in thiscase are characterized by interesting points that should be reachedby robots. An interest point could be either a new part of the en-vironment to explore or an objects that need to be analyzed withdifferent sensors, for example a color camera to analyze the shapeor the color and an infrared sensor to detect heat. Such a scenariopresents several interesting and challenging issues to address fromthe coordination perspective. Tasks (i.e., objects to be analyzed) arediscovered during mission execution, and dynamic task assignmentmust be used to improve performance of the system. Moreover, ob-jects might need to be analyzed with different sensors (mounted ondifferent robots) at the same time, thus tasks might be tied by ex-ecution constraints. Robots should spread over the environmentavoiding conflicts on task execution such as several robots trying toexplore the same part of the environments or too many redundantrobots trying to analyze the same object. Moreover, communica-tion among robots is subject to strict constraints. The bandwidthrobots can use is limited and messages can be lost due to tempo-rary network breakdown or permanent robot failures and it cannotbe assumed that each pair of robots can directly communicate witheach other.

While the general ideas presented in this paper could be appliedto other coordination mechanism, in this contribution we focus ontechniques based on Token Passing [6]. In such approaches tokensare used to represent tasks that must be executed by the agents.Each team member creates, executes and propagates these tokensbased on its knowledge of the environment. The basic approach re-lies on the assumption that one token is associated to every task tobe executed and that the token is maintained only by the agent thatis performing such a task. If the agent is not in the condition of per-forming the task it can decide to pass the token on to another teammember. Token Passing assigns tasks using only a broad knowl-edge of team mates, sharing a minimal set of information amongteam members. The approach ensures that task allocation is highlyreactive requiring very low communication.

Such techniques are very well suited for allocating roles in largeteams of robots acting in very dynamic environments where tasksappears and disappears very quickly. However, a main issue for To-ken Passing is to ensure the coherence maintenance of the createdtokens. If more tokens are created for the same task conflicts mayarise in task execution, leading to severe inefficiencies of the whole

33

system. In our reference scenario since tasks are created by agentsin distributed fashion, avoiding conflicts during task execution is afundamental issue to be addressed.

Previous works addressed this problem for task assignment. In[9] the authors propose a market based approach to task assign-ment addressing the issue of conflict resolution. The conflict de-tection mechanism uses broadcast communication thus having thesame limitations previously highlighted. A token based methodwhich is able to solve conflicts is presented in [7]. The methodis specifically targeted towards large scale MAS. Conflicts in thissetting, can be revealed and solved if a overlaps among sub-teamsexist. Authors show that for such large scale teams (i.e. hundredsof agents) chances of having overlaps among sub-teams is high,and thus conflicts can be solved most of the time. Our target ap-plication scenario is targeted toward medium size robotic system(tens of robots) where the overlaps among sub-team are not likelyenough to guarantee good performance of the system.

In our previous work [1] we proposed a distributed conflict de-tection algorithm for the Token Passing approach based on broad-cast communication. Assuming each robot can directly commu-nicate with every other team member and that no message is lostthe algorithm guarantees a conflict free allocation with a limitedcommunication overhead in terms of number of messages. Thealgorithm has been tested on a team of AIBO robots involved ina cooperative foraging tasks. However, the two specified assump-tions consistently limit the applicability of our approach; moreover,while the algorithm uses a limited number of messages, the band-width requirement can be not acceptable for several domains.

In this contribution we present an extension to our previous con-flict detection approach, based only on point to point communi-cations. The main idea of the approach is to extend the conceptof tokens introducing Negative Tokens to detect conflicts. Nega-tive Tokens are tokens that do not represent tasks to be executedbut announce to team members that a specific task is being exe-cuted by someone else in the team. Propagating Negative Tokensamong robots we are able to detect and solve conflicts on task exe-cution. Moreover, we propose a solution to address the problem ofnetwork failures during mission execution. Network failures repre-sent a problematic issue for Token Passing based method becausesince tokens are required to be unique, if a message is lost a taskmaybe not executed. Our approach requires the communicationsystem to be able to detect when a message cannot be relayed to arobot. This can be obtained using a communication protocol basedon acknowledgment. In particular, we re-send messages until anacknowledgment message is received or a time-out occurs. When asender robot detects that a message was not received (e.g., the time-out exceeds) it re-sends the message to some other team member.

To evaluate and test the characteristics of the Negative Tokensapproach we set up an abstract environment that simulates the taskassignment process. We performed several experiments in differ-ent operative conditions. The measures we extract from the exper-iments are the required bandwidth and the system performance interms of time needed to perform the tasks. Experiments show thatthe Negative Tokens approach is able to attain similar performancewith respect to the previous conflict detection method, while requir-ing an extremely lower bandwidth (almost one order of magnitude).Moreover, the experiments performed with message loss show thatthe system is able to give good performance also in presence ofnetwork failures.

In the following section we present a formal definition of the taskassignment problem. Section 3 present the basic Token Passingtechniques. Section 4 presents in detail our approach to conflictdetection and network failures. In section 5 we show the obtained

experimental results and section 6 concludes the paper.

2. THE TASK ASSIGNMENT PROBLEMThe problem of assigning a set of tasks to a set of robots can

be easily framed as a Generalized Assignment Problem (GAP) [8].However, while GAP is well defined for a static environment, whereagents and tasks are fixed and capabilities and resources do not de-pend on time, in multi-robot applications a problem with the de-fined parameters changing with time must be solved. Indeed sev-eral methods for Dynamic Task Assignment implicitly take intoconsideration such an aspect: solutions that consider the dynam-ics of the world are proposed and Task Allocation methods thatapproximate solutions of the GAP problem at each time step arederived [2, 5, 9].

The GAP formulation fails to model all the relevant aspects forour interest domains. In particular, it does not consider two mainissues: i) tasks to be accomplished can be tied by constraints, ii)the set of tasks is not known a priori when the mission starts, but itis discovered and dynamically updated during task execution.

We will use the following notation: E = {e1 . . . en} denotes theset of robots. While in general robots involved in the Task Assign-ment process can also vary over time, in this contribution we focuson a predefined static set of robots.

We will denote tasks by τ [ts,te], where [ts, te] is the time intervalin which the task is present in the system. We denote with Γt theset of tasks which are present at time t, i.e., Γt = {τ [ts,te] | ts ≤t ≤ te}, and with m(t) = |Γt|.

Since values characterizing a task τ may vary over time we useτ t

k to denote the status of task τk at time t. However, in the fol-lowing, time specifications for the tasks will be dropped, whennot relevant. Each task is composed by a set of roles or oper-ations τi = {r1, . . . , rk}, satisfying the following properties: i)∀i, j i 6= j ⇒ τi ∩ τj = ∅; ii) |τ t

i | = c ∀t ∈ [ts, te]. We finallydefine the set of all possible roles at time t as Rt =

Sm(t)i=1 τ t

i .Notice that each role can comprise a set of sub-roles and so on;

for the sake of simplicity, we consider only two levels of the pos-sible hierarchy, i.e. tasks which are divided in roles; hence, for thecoordination process, roles can be considered as atomic actions.Each robot has different capabilities for performing each role anddifferent resources available.

Moreover, we define, for each r ∈ Rt, the set of all roles con-strained to r as Cr ⊆ Rt. While in general constraints can possiblybe of several types (AND, OR, XOR), in this article we focus onlyon AND constraints. Thus Cr represents the set of roles that mustbe executed concurrently by different agents. The properties ofeach constrained set Cr are: i) r ∈ Cr; ii) r′ ∈ Cr → r ∈ Cr′ .Non-constrained roles are determined by |Cr| = 1.

A set of roles Cr subject to AND constraints must be performedsimultaneously by |Cr| teammates. Notice that if a role r is uncon-strained, Cr = {r}.

We express the capabilities and the resources depending on timewith Cap(ei, rj , t), Res(ei, rj , t); where Cap(ei, rj , t) representsthe reward for the team when robot ei performs role rj at time t,Res(ei, rj , t) represents the resources needed by ei to perform rj

at time t. Finally, ei.res(t) represents the available resources forei at time t.

A dynamic allocation matrix, denoted by At, is used to establishTask Assignment; in At, aei,rj ,t = 1 if the robot ei is assigned tothe task rj at time t, and 0 otherwise. Consequently, the problemis to find a dynamic allocation matrix that maximizes the followingfunction

34

f(At) =X

t

Xi

Xrj∈Rt

Cap(ei, rj , t)× aei,rj ,t

subject to:

∀t∀rj ∈ Rt

Xi

Xrk∈Crj

aei,rk,t = |Crj | ∨

Xi

Xrk∈Crj

aei,rk,t = 0

∀t∀iX

rj∈Rt

Res(ei, rj , t)× aei,rj ,t ≤ ei.res(t)

∀t∀rj ∈ Rt

Xi

aei,rj ,t ≤ 1

It is important to notice that this problem definition allows for so-lutions that can oscillate between different allocations that have thesame value of f(At). Such oscillations can also happen when noisyperception affects computation of the capabilities. This can beavoided by taking into account in the implementation of Cap(ei, rj , t)the cost of interrupting a task for switching to another.

3. TOKEN PASSING APPROACH TO TASKASSIGNMENT

The problem of Task Assignment presented in section 2 has beensuccessfully addressed by a Token Passing approach [6].

Tokens represent tasks to be executed and are exchanged throughthe system in order to collect information and to allocate the tasksto the agents.

When an agent receives a token, it decides whether to performthe task associated to it or to pass the token on to another agent.This decision is taken based only on local information: each agentfollows a greedy policy, i.e., it tries to maximize its utility, given thetokens it can currently access, its resource constraints and a broadknowledge on team composition. The ability of the team to assigntasks is related to the computation of the capabilities Cap(ei, rj , t).Tasks are executed by the agent that has the corresponding tokenonly if this capability is higher than a given threshold. This thresh-old can be computed in different ways depending on the scenario.For example, when tasks are a priori known, this threshold can befixed before inserting the token, or it can be established by the firstagent receiving the token based on its local information.

If the capability of the agent is higher than the required thresh-old, the agent considers the possibility to allocate that task to itself.Otherwise, the agent adds some information about the task in thetoken and then sends the token to another agent. The token storesthe list of agents that already refuted the task, in this way when anagent passes a token away can choose an agent that has not previ-ously discarded it.

Thresholds guide the search towards good solutions for the al-location problem. While such mechanism cannot give guaranteesconcerning the optimality of the solutions found, it has been ex-perimentally shown that it can consistently increase the algorithmperformance [6].

When tasks are constrained, these are composed by roles to besimultaneously executed. In this case, tokens are associated to theroles in the tasks. When considering constrained tasks, assignmentsbased on thresholds on the agent capabilities will lead to potentialdeadlocks or inefficiencies. For example, consider two roles, rj

and rk, that need to be simultaneously performed. When a teammember a accepts role rj , it may reject other roles that it couldpotentially perform. If there is no team member currently availableto perform role rk, a will wait and will not be assigned to anotherrole. Thus, an explicit enforcement of the AND constraints amongroles is needed.

The general idea is to use potential tokens to represent roles thatare tied by AND constraints. Potential tokens retain agents: whenan agent receives a potential token it can perform other roles (i.e.,the potential token does not impact on the current resource load ofthe agent). A manager agent exists for each group of Anded roles.When enough agents have been retained for the task execution, themanager agent sends a lock message to each of the retained agents.When the lock message arrives, the retained agent should start theexecution of the role, possibly releasing the current role and send-ing away the related token. The choice on which role(s) should bestopped is performed based on a greedy local policy. If the role tobe stopped is a constrained role, the agent will inform the task man-ager and the allocation process for that role will be restarted. Thismechanism for allocating AND constrained roles has been testedand validated in several domains and operative conditions (see [6]).

To further clarify the token based assignment, let us consider thefollowing situation, two tasks τ1, τ2 and three agents e1, e2, e3.The task τ1 comprises one role r1 while τ2 comprises two roles r2

and r3 tied by an AND constraint. Suppose agent e2 is handlingroles r2 and r3 and it is not capable of performing them. Supposeagent e1 is retained for role r2, while no one else is retained forrole r3. Finally, suppose agent e1 receives a token for role r1, andit is capable of performing the role. The agent e1 will thus keepthe token and start performing role r1. If at this point agent e3

considers itself retained for role r3, it will notify that to agent e2

(which is the task manager). Agent e2 will send a lock messageto both agent e1 and agent e3. Agent e3 will start performing roler3 and agent e1 will refute the role r1 sending the token to anotheragent and start executing role r2. In this way the execution of theroles will correctly meet the AND constraint between roles r2 andr3.

4. CONFLICT RESOLUTION AVOIDINGBROADCAST COMMUNICATION

The Token Passing approach presented in section 3 is based onthe assumption that one token is associated to every task to be exe-cuted and that the token is maintained by the agent that is perform-ing such a task, or passed to another agent. This assumption holdswhen tokens are inserted into the system in a coherent fashion, andunder this assumption the algorithm ensures that no conflict arisesamong agents, (i.e. two agents trying to execute the same role).However, when tasks are perceived and tokens generated by agentsduring mission execution, conflicts on task execution may arise.In fact, several agents may perceive the same task, and an uncon-trolled number of tokens can be created, leading too many agentsto execute the same role.

In [1] we presented an approach that ensures that exactly n agentswill participate in the same task simultaneously. Such an approachis based on a distributed conflict detection and resolution mecha-nism and makes use of broadcast communication among agents.

The extension we present here avoid broadcast communicationamong agents making use only of point to point messages. Agentssend token not only to offer tasks that they cannot execute, but alsoto inform other agents about tasks they are executing or managing.We call this extension Negative Token approach since we use tokensthat prevent other agent to execute tasks.

35

In the Negative Token approach whenever an agent discovers anew task to be accomplished it creates a token for it and send anannounce token to one of its neighboring agent. The announce to-ken store a list of visited agents which is used to propagate thetoken though all the agent network. When the token reached allteam members it is discarded.

Algorithm 1: Coherence Maintenance with p2p messagesONPERCRECEIVED(task)(1) if (task 6∈ KTS)(2) KTS = KTS ∪ task(3) annMsgS = annMsgS ∪ {task, MyId}(4) TkS = TkS ∪ Tk(task)1 ∪ · · · ∪ Tk(task)s

(5) PROPAGATE(msg(Announce,task))

ONTASKACCOMPLISHMENT(task)(1) ATS = ATS ∪ task(2) PROPAGATE(msg(AccomplishedTask,task))

MSGANALYSIS(msg)(1) PROPAGATE(msg)(2) if msg is AccomplishedTask(3) ATS = ATS ∪msg.task(4) if msg is Announce(5) if (msg.task 6∈ KTS)(6) KTS = KTS ∪ {msg.task}(7) annMsgS = annMsgS∪{msg.task, msg.creator}(8) else(9) AnnIt = GETANNOUNCE(Msg.Task)(10) if AnnIt.creator ≤ msg.creator(11) ITS = ITS ∪ {AnnIt.task, AnnIt.creator}(12) UPDATE(AnnMsgS, msg)(13) else(14) ITS = ITS ∪ {msg.task, msg.creator}

Algorithm 1 shows the pseudo-code for the Negative Token ap-proach. The algorithm requires a total ordering among teammates.In particular, we consider a static fixed priority based on agent id.The algorithm uses local data structures that each agent maintains:i) Known Task Set (KTS) which contains at each time step all thetask known to the agent (by direct perception or through networkcommunication); ii) Accomplished Task Set (ATS) which containsat each time step all the tasks the agent considers accomplished;iii) Invalid Task Set (ITS) which contains at each time step thetasks that the agent considers invalid along with the informationon the agent that created the task; iv) Announced Message Set(annMsgS) which is a set of announced messages received. ann-MsgS is updated in order to store, at each time step and for eachannounced task, the announce message received by the highest pri-ority teammate, and is used to decide whether an announced taskshould be considered invalid; v) Token Set (TkS) which is the setof tokens that the agent currently holds. Messages sent betweenagents have four fields: (1) type, which denotes the type of themessage; (2) task, which contains information about the perceivedtask (e.g. object position), which is valid when type is announceor accomplishedTask; (3) token (valid only when the messageis a token), which contains information about the token (e.g. tasktype, task information, etc.), (4) senderId, which is an identifierfor the robot that sent the message. (5) creator, which is an identi-fier for the robot that created the token. (6) visitedAgentQueue,which is a queue containing the identifiers of visited agent for thismessage.

Whenever a new perception is received, an announce message

for the task discovery is stored in the annMsgS (procedure On-PercReceived, line 3) and then sent to one of the neighboring agent(line 5). Whenever a task is accomplished, an accomplished taskmessage is sent to one of the neighboring agent (procedure On-TaskAccomplishment, line 2). The MsgAnalysis procedure propa-gates and process the coordination messages. Each received mes-sage will be propagated using the Propagate function. Agents prop-agate messages according to the visited agent queue. The algorithmto propagate messages must guarantee that all the agents receive themessage using only information on their neighboring agents. Tothis end a depth first visit of the connection graph using the visitedagent queue is a suitable approach under the assumption that theagent network is connected. When all agents have been visited themessage can be discarded.

If a received message is an AccomplishedTask message, theagent adds the task to its ATS. if the message is an Announcemessage, the agent checks whether the task has been already an-nounced by checking its KTS (line 5). If the task is not present inits KTS, it adds the task in the KTS and inserts the correspond-ing announce message in its annMsgS; if the task was alreadypresent in the KTS, the agent detects a conflict; using annMsgS(procedure MsgAnalysis, line 9) it checks whether the invalid taskis the new announced task or the one previously received and, con-sequently, updates the annMsgS and the ITS. Each robot peri-odically removes all tasks which are present in the ATS and in theITS from the tokens it currently holds.

Assuming no message loss the algorithm ensures that all con-flicts will be eventually detected and solved. The maximum timeneeded to detect a conflict depends on network topology and num-ber of team members. The algorithm requires a very low networkbandwidth, because it trades-off time to detect a conflict with re-spect to number of messages to be sent in parallel to team members.We can leverage this trade-off depending on the particular applica-tion scenario deciding to send more than one announce messagein parallel. In this perspective, the broadcast conflict detection ap-proach described in [1] can be seen as an extreme case were allagents can reach directly the other team members and the maxi-mum number of parallel message is alway sent. With respect tosuch an approach this method not only greatly reduce the requiredbandwidth but remove the important assumption that each agentshould be able to directly reach all its team mates.

In this contribution, detected conflicts are solved using a staticfixed priority defined among agents. Notice that any policy thatgives a global ordering of team members and that does not requirefurther communication can be used in place of fixed priority asa global preference criterion. Another option could be to invali-date tasks, which have been created more recently. This, however,would require to have a synchronized clock among agents. In anycase, setting a static fixed priority among agents can obviously re-sult in non optimal behavior of the team; for example, assumingthat Cap(e1, rk, t) > Cap(e2, rk, t) following a static prioritybased on id, we yield to the less capable agent the access to thetask rk. While in principle the difference among capabilities canbe unbounded, generally, when tasks are discovered using percep-tion capabilities, agents perceive tasks when they are close to theobject location, (e.g. if two robots perceive the same object theirdistance from the object is comparable); therefore, the loss of per-formance due to the use of a fixed priority is limited.

The presented algorithm, and the token passing approach in gen-eral, do not have any specific mechanism to address possible mes-sage loss. Message loss can be particularly problematic for tokenbased approach to task assignment because tokens are required tobe unique, therefore if a token is lost the corresponding task could

36

remain unaccomplished.Since message loss is a very important issue for multi-robot sys-

tem coordination approach. in this contribution we present a simpleextension to the token passing approach that makes it more robustto possible network failure.

We model network failures as temporary or permanent discon-nection of agents from the rest of team. In this black-out peri-ods disconnected agents cannot send or receive messages from anyteam members, but they can still act correctly. Such model of net-work failure captures an important class of problems which are re-lated to robotic system embedded in real world environment. Infact, it is often the case that members of a robotic team are discon-nected due for example to interferences with the communicationmeans or due to particular configuration of the environment.

We assume that the agent sending a message knows whetherthe recipient agent correctly received the message. This can bedone using a communication protocol based on acknowledgment.Whenever, a sender agent ei reveals that a message cannot be re-layed to an agent ej it inserts agent ej inside the visited agents forthat message and propagate the message on. However, the fact thatagent ej could not be reached by that message is registered insert-ing the agent inside a specific list of unreachable agents for thatmessage. The message will keep on being processed according tothe task assignment process. In particular, if the message is an an-nounce message several policies could be used to determine whenthe message propagation should stop. If we want to be sure all con-flicts will be detected and solved we should keep on propagatingthe message up to when the unreachable agent list is empty payingthe price of a higher communication overhead. On the other handwe could decide to stop the token as soon as the message reachesits creator and all its neighbors are inside the visited agent queue.Such a policy cannot guarantee that all conflicts will be solved butis able to make the system more robust to network failure withoutany impact on communication overhead. Depending on the appli-cation scenario we can decide to employ the policy that maximizethe trade-off between communication overhead and required algo-rithm performance.

5. EXPERIMENTS AND RESULTSThe basic token passing approach has been extensively tested

in different operative domains both on simulated software agents[6] and on real robots [1]. The conducted experiments show thatthe method is able to provide very good performance compared tostandard task assignment techniques while maintaining a very lowcommunication overhead.

In this contribution our main aim is to study the advantages anddrawbacks of the Negative Token method opposed to our previousapproach for conflict detection based on broadcast communication.

We set up an abstract simulation environment where agents per-ceive interesting objects and exchange messages to coordinate. Ob-jects can initiate two kinds of tasks: constrained and unconstrainedtasks. Constrained tasks have to be executed simultaneously by acertain number of agents to be correctly accomplished. Each taskhas a time to complete, and when a task is correctly allocated to anagent its complete time decreases. A correct allocation is an allo-cation that fulfill all the constraints specified in Section 2, i.e. thereare no conflict on role allocation and execution constraints are ful-filled. Notice that, this is a very strict model of the world, in factusually constraints on task execution degrade the performance ofthe system but do not totally invalidate the task execution. More-over, if tasks are not being accomplished for a certain period of timethey reset their time to complete. Finally, since our main interest isto reduce the number of conflicts we performed the experiments in

the worst case scenario, where all agents perceive all tasks insertedinto the system, giving rise to the maximum number of conflicts.

To measure the performance of our system, we use the allocationvalue f(At) defined in 2. Since in our experiments we focus on theconflict resolution method, the agent capabilities are the same forall tasks. Therefore, the allocation value becomes the sum over alltasks, of the time steps for which each task is correctly allocated tosome agent. Moreover, we measure the bandwidth required by thesystem during the mission execution.

To evaluate the communication overhead, we measure the band-width usage as the number of messages exchanged for each timestep. Notice that, we are not interested in an average value of thebandwidth usage, but we want to evaluate the actual behavior ofthe bandwidth over time. In fact, since we require a Negative Tokento visit all team members before being discarded, the total num-ber of messages used by the Negative Token and by the broadcastapproach will be almost the same. However, the broadcast ap-proach has a much higher bandwidth requirement sending severalmessages at the same time. Therefore, in the following we reportthe bandwidth over time (Figure 2) or the maximum bandwidth re-quirement (e.g., maximum number of messages sent at the sametime step during the mission execution). To evaluate the numberof exchanged message, we assume that the overhead of a broadcastmessage is higher than the overhead of a point to point message. Inparticular, we count a broadcast message as point to point messagetimes the number of robots. While for a more precise analysis ofthe overhead one should consider the specific network used1, webelieve that for this level of analysis this is a reasonable assump-tion.

Figure 1: Allocation value over time for 20 agents and 10 un-constrained tasks

Figure 1 and Figure 2 show respectively the allocation value andthe bandwidth over time, for a single experiment. The experimentcomprises twenty agents with eight unconstrained tasks. Tasks areinserted in the system at different points in time. As it is possible tosee the Negative Token method gives a great advantage in terms ofbandwidth and a very small drawback in terms of allocation value.In particular, from figure 1 it is possible to see that both methodscomplete all tasks, but the broadcast method is quicker than theNegative Tokens one. This is due to the conflicts present when tasksenters the system. The broadcast method solve the conflicts al-1For example for an IEEE 802.11 wireless network the cost ofsending a broadcast message might be different with respect to awired Ethernet LAN

37

Figure 2: Bandwidth requirement over time

most instantaneously while the Negative Token method needs moretime to detect and solve conflicts. On the other side, the broadcastmethod pays the price of a large bandwidth requirement which isalmost one order of magnitude higher than the one required by theNegative Token method. Notice that, the spikes present in the band-width behavior, are referred to time steps where objects are insertedinto the system and conflicts are detected and solved.

Figure 3: Allocation value varying agent number with uncon-strained tasks

Figures 3 and 4 show the performance and the maximum band-width for the two methods varying the number of agents. The ex-periments have a constant ratio Agent/tasks where the number ofagents is twice the number of tasks. In the two figures the tasksare all unconstrained task. Since all conflicts are always solved forboth methods we use as performance measure the complete timeof all tasks. The shown results are averaged over ten simulation ofthe same configuration. The figures confirm that the Negative To-ken method shows a limited decrease in performance with respectto the gain obtained in terms of required bandwidth.

Figures 5 and 6 show the same measures for experiments withconstrained tasks. Number of tasks are half the number of agentsand each task has two AND-constrained roles. The curve behaviorsmainly confirm the previous results.

To study the behavior of our method in presence of network fail-

Figure 4: Max bandwidth requirement varying agent numberwith unconstrained tasks

Figure 5: Allocation value varying agent number with con-strained tasks

Figure 6: Max bandwidth requirement varying agent numberwith constrained tasks

38

Figure 7: Number of uncompleted roles for unconstrainedtasks, varying number of disconnected agents

Figure 8: Number of uncompleted roles for constrained tasks,varying number of disconnected agents

ure, we performed experiments varying the number of agents thatare disconnected during the mission execution from one to ten overtwenty agents. Both the agent’s id and the time at which the agentwill be disconnected are drawn from a random distribution.

Figures 7 and 8 show respectively the performance for the twomethods with unconstrained and AND-constrained tasks. Since inthe presence of message loss both the methods may not be able tosolve all conflicts, some of the tasks might not be accomplished.Therefore in this case the measure we use for the performance isthe number of uncompleted tasks. In the experiments we decidedto use the policy that minimize the bandwidth usage, therefore wedo not resend lost message, consequently conflicts could remainunsolved. In fact, performance degrades when the number of dis-connected agents is higher. However, even with such a policy, boththe methods are able to accomplish most of the tasks. In particular,when tasks are unconstrained the Negative Token method seems tobe better than the broadcast one, while when tasks are constrainedit is the opposite. To explain why in the unconstrained task sce-nario the Negative Tokens method has better performance we haveto consider in detail what happen when an agent is disconnected.Suppose agent ei is disconnected at time step t and at time t + 1a new object is inserted into the system. All agents will start send-ing the announce tokens for the new object, agent ei will try tosend the announce messages as well. Consider that agent ei triesto send an announce message to agent ej the message will not berelayed, therefore ei will include ej into the visited agents for theannounce message and will try to send to agent ek. This processwill go on up to when agent ei is reconnected, at this point its mes-sage will be relayed to all agents it did not try to reach during thedisconnection time. In a similar situation the broadcast announcemessage of agent ei will be immediately lost, therefore the con-flicts related to the new object will never be detected. In the caseof constrained tasks the broadcast has better performance. In thiscase the allocation of constrained tasks entails a back and forth ofmessages among agents, and agents might be disconnected in anyof this communication phase. The broadcast method quickly re-solve conflicts converging towards stable solutions of the alloca-tion; therefore agent disconnection are less problematic for the sys-tem performance. On the other side the Negative Tokens approachkeeps alive invalid tokens for a longer time, and possible disconnec-tion during this phase are more penalizing in terms of performancedecrease.

6. CONCLUSIONS AND FUTURE WORKIn this article we have presented a distributed algorithm for Task

Assignment in dynamic environments that uses only point to pointmessages. The presented approach is based on Token Passing torole allocation and extend our previous work on distributed con-flict detection based on broadcast communication. Moreover, weaddressed the problem of network failures, further extending theapproach to operate with a limited performance decrease in caseof agent disconnection. The experiments performed show that ourNegative Token approach is able to maintain good performancewhile dramatically reducing the bandwidth usage to attain coor-dination.

As future work several interesting extensions could be consid-ered in order to realize a more efficient task assignment approach.In particular, each agent could maintain a model of its neighbor-ing team members, to make better decision on which agent shouldbe the recipient of the message. This could enhance the conflictdetection mechanism and the overall system performance.

39

7. ACKNOWLEDGMENTThis effort was supported by the European Office of Aerospace

Research and Development under grant number 053015. The viewsand conclusions contained herein are those of the authors and shouldnot be interpreted as necessarily representing the official policies orendorsements, either expressed or implied, of the European Officeof Aerospace Research and Development.

8. REFERENCES[1] A. Farinelli, L. Iocchi, D. Nardi, and V. A. Ziparo. Task

assignment with dynamic perception and constrained tasks ina multi-robot system. In Proc. of the IEEE Int. Conf. onRobotics and Automation (ICRA), pages 1535–1540, 2005.

[2] B. Gerkey and J. M. Mataric. Multi-robot task allocation:Analyzing the complexity and optimality of key architectures.In Proc. of the Int. Conf. on Robotics and Automation(ICRA’03), Taipei, Taiwan, Sep 14 - 19 2003.

[3] R. Mailler, V. Lesser, and B. Horling. Cooperative negotiationfor soft real-time distributed resource allocation. InProceedings of AAMAS’03, 2003.

[4] P. J. Modi, H. Jung, M. Tambe, W. M. Shen, and S. Kulkarni.A dynamic distributed constraint satisfaction approach toresource allocation. Lecture Notes in Computer Science,2239:685–700, 2001.

[5] L. E. Parker. ALLIANCE: An architecture for fault tolerantmultirobot cooperation. IEEE Transactions on Robotics andAutomation, 14(2):220–240, April 1998.

[6] P. Scerri, A. Farinelli, S. Okamoto, and M. Tambe. Tokenapproach for role allocation in extreme teams. In In Proc. ofAAMAS 05, pages 727–734, 2005.

[7] P. Scerri, Y. Xu, E. Liao, G. Lai, and K. Sycara. Scalingteamwork to very large teams. In In Proceedings of AAMAS,July 2004.

[8] D. Shmoys and E. Tardos. An approximation algorithm for thegeneralized assignment problem. Mathematical Programming,62:461–474, 1993.

[9] R. Zlot, A. Stenz, M. B. Dias, and S. Thayer. Multi robotexploration controlled by a market economy. In Proc. of theInt. Conf. on Robotics and Automation (ICRA’02), pages3016–3023, Washington DC, May 2002.

40

Lessons Learned from Disaster Management ∗

Nathan Schurr, Pratik Patil, Fred Pighin, Milind Tambe,University of Southern California, Los Angeles, CA 90089, {schurr, pratiksp, pighin, tambe}@usc.edu

ABSTRACTThe DEFACTO system is a multiagent based tool for training inci-dent commanders of large scale disasters. In this paper, we high-light some of the lessons that we have learned from our interactionwith the Los Angeles Fire Department (LAFD) and how they haveaffected the way that we continued the design of our disaster man-agement training system. These lessons were gleaned from LAFDfeedback and initial training exercises and they include: system de-sign, visualization, improving trainee situational awareness, adjust-ing training level of difficulty and situation scale. We have takenthese lessons and used them to improve the DEFACTO system’straining capabilities. We have conducted initial training exercisesto illustrate the utility of the system in terms of providing usefulfeedback to the trainee.

1. INTRODUCTIONThe recent hurricanes that have hit the gulf coast of the US have

served to reaffirm the need for emergency response agencies to bebetter prepared for large scale disasters. Both natural and man-made (terrorism) disasters are growing in scale, however the re-sponse to these incidents continues to be managed by a single per-son, namely the incident commander. The incident commandermust monitor and direct the entire event while maintaining com-plete responsibility. Because of this, incident commanders muststart to be trained to handle these large scale events and assist inthe coordination of the team.

In order to fulfill this need and leverage the advantages of multia-gents, we have continued to develop the DEFACTO system (Demon-strating Effective Flexible Agent Coordination of Teams via Om-nipresence). DEFACTO is a multiagent based tool for training in-cident commanders for large scale disasters (man-made or natural).

Our system combines a high fidelity simulator, a redesigned hu-

∗This research was supported by the United States Departmentof Homeland Security through the Center for Risk and EconomicAnalysis of Terrorism Events (CREATE). However, any opinions,findings, and conclusions or recommendations in this document arethose of the author and do not necessarily reflect views of the U.S.Department of Homeland Security.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.Copyright 200X ACM X-XXXXX-XX-X/XX/XX ... $5.00.

man interface, and a multiagent team driving all of the behaviors.Training incident commanders provides a dynamic scenario in whichdecisions must be made correctly and quickly because human safetyis at risk. When using DEFACTO, incident commanders have theopportunity to see the disaster in simulation and the coordinationand resource constraints unfold so that they can be better preparedwhen commanding over an actual disaster. Applying DEFACTO todisaster response aims to benefit the training of incident comman-ders in the fire department.

With DEFACTO, our objective is to both enable the human tohave a clear idea of the team’s state and improve agent-human teamperformance. We want DEFACTO agent-human teams to betterprepare firefighters for current human-only teams. We believe thatby leveraging multiagents, DEFACTO will result in better disasterresponse methods and better incident commanders.

Previously, we have discussed building our initial prototype sys-tem, DEFACTO [8]. Recently, the Los Angeles Fire Department(LAFD) have begun to evaluate the DEFACTO system. In this pa-per, we highlight some of the lessons that we have learned from ourinteraction with the LAFD and how they have affected the way thatwe continued to design of our training system. These lessons weregleaned from LAFD feedback and initial training exercises.

The lessons learned from the feedack from the LAFD include:system design, visualization, improving trainee situational aware-ness, adjusting training level of difficulty and situation scale. Wehave taken these lessons and used them to improve the DEFACTOsystem’s training capabilities.

We have also perfromed initial training exercise experiments toillustrate the utility of the system in terms of providing useful feed-back to the trainee. We ended up finding that allowing more fireengines to be at the disposal of the incident commander sometimesnot only didn’t improve, but rather worsened team performance.There were even some instances in which the agent team wouldhave performed better had the team never listened to human adviceat all. We also provide analysis of such behaviors, thereby illustrat-ing the utility of DEFACTO resulting from the feedback given totrainees.

2. MOTIVATIONIn this section, we will first start with an explanation of the cur-

rent methods for training that the LAFD currently use. Then weexplain some of the advantages that our multiagent approach hasover these methods.

The incident commander’s main duties during a fire shoulder allresponsibility for the safety of the firefighters. In order to do this,the incident commander must have constant contact with the fire-fighters and have a complete picture of the entire situation. Theincident commander must make certain that dangerous choices are

41

LAFD Exercise: Simulations by People Playing Roles

� LAFD officials simulate fire progression and the resource availability

� Battalion Chief allocates available resources to tasks

(a) Current Incident Commander Training Exercise(b) Fire Captain Roemer using the DEFACTO trainingsystem

Figure 1: Old vs. New training methods

avoided and the firefighters are informed and directed as needed.We were allowed to observe a Command Post Exercise that sim-

ulated the place where the incident commander is stationed duringa fire (see Figure 1(a)). The Incident commander has an assistantby his side who keeps track on a large sheet of paper where all ofthe resources (personnel and equipment) are located. A sketch ofthe fire is also made on this sheet, and the fire and fire engines’location is also managed.

The Command Post is currently simulated by projecting a sin-gle static image of a fire in an apartment. In the back of the room,several firefighters are taken off duty in order to play the role of fire-fighters on the scene. They each communicate on separate channelsover walkie talkies in order to coordinate by sharing informationand accepting orders. The fire spreading is simulated solely byhaving one of the off-duty firefighters in the back speaking over thewalkie talkie and describing the fire spreading.

The LAFD’s curent approach, however, has several limitations.First, it requires a number of officers to be taken off duty, whichdecreases the number of resources available to the city for a disas-ter during training. Second, the disaster conditions created are notaccurate in the way that they appear or progress. Since the imagethat the incident commander is seeing is static, there is no infor-mation about state or conditions of the fire that can be ascertainedfrom watching it, which is contrary to the actual scene of a disas-ter response. Furthermore, the fire’s behavior is determined by thereports of the acting fire fighters over the walkie talkie, which attimes might not be a plausible progression of fire in reality. Third,this method of training restricts it to a smaller scale of fire becauseof the limited personnel and rigid fire representation.

Our system aims to enhance the training of the incident com-manders (see Figure 1(b)). Our approach allows for training to notbe so personnel heavy, because fire fighter actors will be replacedby agents. By doing this we can start to train incident comman-ders with a larger team. Through our simualtion, we can also startto simulate larger events in order to push the greater number ofavailable resources to their limit. Also, by simulating the fire pro-gression, we can place the Incident commander in a more realisticsituation and force them to react to realistic challenges that arise.

3. SYSTEM ARCHITECTUREIn this section, we will describe the technologies used in three

major components of DEFACTO: the Omni-Viewer, proxy-based

Disaster Scenario

Proxy

Proxy

Proxy

Proxy

Team

Omni-Viewer

DEFACTO

Incident Commander

Figure 2: System Architecture

team coordination, and proxy-based adjustable autonomy. The Omni-Viewer is an advanced human interface for interacting with an agent-assisted response effort. The Omni-Viewer has been introducedbefore [8], however has since been redesigned after incorporatinglessons learned from the LAFD. The Omni-Viewer now providesfor both global and local views of an unfolding situation, allow-ing a human decision-maker to obtain precisely the information re-quired for a particular decision. A team of completely distributedproxies, where each proxy encapsulates advanced coordination rea-soning based on the theory of teamwork, controls and coordinatesagents in a simulated environment. The use of the proxy-basedteam brings realistic coordination complexity to the training systemand allows a more realistic assessment of the interactions betweenhumans and agent-assisted response. These same proxies also en-able us to implement the adjustable autonomy necessary to balancethe decisions of the agents and human.

DEFACTO operates in a disaster response simulation environ-ment. The simulation environment itself is provided by the RoboCupRescue Simulator [3]. To interface with DEFACTO, each fire en-gine is controlled by a proxy in order to handle the coordinationand execution of adjustable autonomy strategies. Consequently, theproxies can try to allocate fire engines to fires in a distributed man-ner, but can also transfer control to the more expert user (incidentcommander). The user can then use the Omni-Viewer to allocateengines to the fires that he has control over. In our scenario, sev-eral buildings are initially on fire, and these fires spread to adjacentbuildings if they are not quickly contained. The goal is to have a

42

Figure 3: Proxy Architecture

human interact with the team of fire engines in order to save thegreatest number of buildings. Our overall system architecture ap-plied to disaster response can be seen in Figure 2.

3.1 Omni-ViewerOur goal of allowing fluid human interaction with agents re-

quires a visualization system that provides the human with a globalview of agent activity as well as shows the local view of a particu-lar agent when needed. Hence, we have developed an omnipresentviewer, or Omni-Viewer, which will allow the human user diverseinteraction with remote agent teams. While a global view is ob-tainable from a two-dimensional map, a local perspective is bestobtained from a 3D viewer, since the 3D view incorporates the per-spective and occlusion effects generated by a particular viewpoint.

To address our discrepant goals, the Omni-Viewer allows forboth a conventional map-like top down 2D view and a detailed 3Dviewer. The viewer shows the global overview as events are pro-gressing and provides a list of tasks that the agents have transferredto the human, but also provides the freedom to move to desired lo-cations and views. In particular, the user can drop to the virtualground level, thereby obtaining the perspective (local view) of aparticular agent. At this level, the user can fly freely around thescene, observing the local logistics involved as various entities areperforming their duties. This can be helpful in evaluating the phys-ical ground circumstances and altering the team’s behavior accord-ingly. It also allows the user to feel immersed in the scene wherevarious factors (psychological, etc.) may come into effect.

3.2 Proxy: Team CoordinationA key hypothesis in this work is that intelligent distributed agents

will be a key element of a disaster response. Taking advantageof emerging robust, high bandwidth communication infrastructure,we believe that a critical role of these intelligent agents will be tomanage coordination between all members of the response team.Specifically, we are using coordination algorithms inspired by the-ories of teamwork to manage the distributed response [6]. The gen-eral coordination algorithms are encapsulated inproxies, with eachteam member having its own proxy which represents it in the team.The current version of the proxies is calledMachinetta[7] and ex-tends the earlier Teamcore proxies [5]. Machinetta is implementedin Java and is freely available on the web. Notice that the conceptof a reusable proxy differs from many other “multiagent toolkits”in that it provides the coordinationalgorithms, e.g., algorithms forallocating tasks, as opposed to theinfrastructure, e.g., APIs for re-liable communication.

Communication: communication with other proxiesCoordination: reasoning about team plans and communicationState: the working memory of the proxyAdjustable Autonomy: reasoning about whether to act autonomously

or pass control to the team member

RAP Interface: communication with the team member

The Machinetta software consists of five main modules, threeof which are domain independent and two of which are tailoredfor specific domains. The three domain independent modules arefor coordination reasoning, maintaining local beliefs (state) and ad-justable autonomy. The domain specific modules are for commu-nication between proxies and communication between a proxy anda team member. The modules interact with each other only via theproxy’s local belief state with a blackboard design and are designedto be “plug and play.” Thus new adjustable autonomy algorithmscan be used with existing coordination algorithms. The coordina-tion reasoning is responsible for reasoning about interactions withother proxies, thereby implementing the coordination algorithms.

Teams of proxies implementteam oriented plans(TOPs) whichdescribe joint activities to be performed in terms of the individualroles and any constraints between those roles. Generally, TOPsare instantiated dynamically from TOP templates at runtime whenpreconditions associated with the templates are filled. Typically, alarge team will be simultaneously executing many TOPs. For ex-ample, a disaster response team might be executing multiple fightfire TOPs. Such fight fire TOPs might specify a breakdown offighting a fire into activities such as checking for civilians, ensur-ing power and gas is turned off, and spraying water. Constraintsbetween these roles will specify interactions such as required exe-cution ordering and whether one role can be performed if anotheris not currently being performed. Notice that TOPs do not specifythe coordination or communication required to execute a plan; theproxy determines the coordination that should be performed.

Current versions of Machinetta include a token-based role allo-cation algorithm. The decision for the agent becomes whether toassign values from the tokens it currently has to its variable or topass the tokens on. First, the team member can choose the mini-mum capability the agent should have in order to assign the value.This minimum capability is referred to as thethreshold. The thresh-old is calculated once (Algorithm 1, line 6), and attached to thetoken as it moves around the team.

Second, the agent must check whether the value can be assignedwhile respecting its local resource constraints (Algorithm 1, line9). If the value cannot be assigned within the resource constraintsof the team member, it must choose a value(s) to reject and pass onto other teammates in the form of a token(s) (Algorithm 1, line 12).The agent keeps values that maximize the use of its capabilities(performed in the MAX CAP function, Algorithm 1, line 10).

Algorithm 1 TOKENMONITOR(Cap, Resources)

1: V ← ∅2: while truedo3: msg ← getMsg()4: token← msg5: if token.threshold = NULL then6: token.threshold← COMPUTETHRESHOLD(token)7: if token.threshold ≤ Cap(token.value) then8: V ← V ∪ token.value9: if

Pv∈V Resources(v) ≥ agent.resources then

10: out← V− MAX CAP(V alues)11: for all v ∈ out do12: PASSON(newtoken(v))13: V alues← V alues− out14: else15: PASSON(token) /* threshold > Cap(token.value) */

3.3 Proxy: Adjustable Autonomy

43

One key aspect of the proxy-based coordination is AdjustableAutonomy. Adjustable autonomy refers to an agent’s ability to dy-namically change its own autonomy, possibly to transfer controlover a decision to a human. Previous work on adjustable autonomycould be categorized as either involving a single person interactingwith a single agent (the agent itself may interact with others) or asingle person directly interacting with a team. In the single-agentsingle-human category, the concept of flexible transfer-of-controlstrategy has shown promise [6]. A transfer-of-control strategy is apreplanned sequence of actions to transfer control over a decisionamong multiple entities. For example, anAH1H2 strategy impliesthat an agent (A) attempts a decision and if the agent fails in the de-cision then the control over the decision is passed to a humanH1,and then ifH1 cannot reach a decision, then the control is passedto H2. Since previous work focused on single-agent single-humaninteraction, strategies were individual agent strategies where only asingle agent acted at a time.

An optimal transfer-of-control strategy optimally balances therisks of not getting a high quality decision against the risk of costsincurred due to a delay in getting that decision. Flexibility in suchstrategies implies that an agent dynamically chooses the one thatis optimal, based on the situation, among multiple such strategies(H1A, AH1, AH1A, etc.) rather than always rigidly choosing onestrategy. The notion of flexible strategies, however, has not been ap-plied in the context of humans interacting with agent-teams. Thus,a key question is whether such flexible transfer of control strategiesare relevant in agent-teams, particularly in a large-scale applicationsuch as ours.

DEFACTO aims to answer this question by implementing transfer-of-control strategies in the context of agent teams. One key ad-vance in DEFACTO is that the strategies are not limited to indi-vidual agent strategies, but also enables team-level strategies. Forexample, rather than transferring control from a human to a singleagent, a team-level strategy could transfer control from a human toan agent-team. Concretely, each proxy is provided with all strategyoptions; the key is to select the right strategy given the situation. Anexample of a team level strategy would combineAT Strategy andH Strategy in order to makeAT H Strategy. The default team strat-egy,AT , keeps control over a decision with the agent team for theentire duration of the decision. TheH strategy always immediatelytransfers control to the human.AT H strategy is the conjunction ofteam levelAT strategy withH strategy. This strategy aims to sig-nificantly reduce the burden on the user by allowing the decision tofirst pass through all agents before finally going to the user, if theagent team fails to reach a decision.

4. LESSONS LEARNED FROM INITIAL DE-PLOYMENT FEEDBACK

Through our communication with strategic training division ofthe LAFD (see Figure 1(b)), we have learned a lot of lessons thathave influenced the continuing development of our system.

4.1 PerspectiveJust as in multiagent systems, the Incident commander must over-

come the challenge of managing a team that each possess only apartial local view. This is highlighted in fighting a fire by incidentcommanders keeping in mind that there are five views to every fire(4 sides and the top). Only by taking into account what is hap-pening on all five sides of the fire, can the fire company make aneffective decision on how many people to send where. Because ofthis, a local view (see Figure 4(a)) can augment the global view (see

Figure 4(b)) becomes helpful in determining the local perspectivesof team members. For example, by taking the perspective of a firecompany in the back of the building, the incident commander canbe aware that they might not see the smoke from the second floor,which is only visible from the front of the building. The incidentcommander can then make a decision to communicate that to thefire company or make an allocation accordingly.

The 3D perspective of the Omni-Viewer was initially thought tobe an example of a futuristic vision of the actual view given to theincident commander. But after allowing the fire fighters to lookat the display, they remarked, that they have such views availableto them already, especially in large scale fires (the very fires we aretrying to simulate). At the scene of these fires are often a news heli-copter is at the scene and the incident commander can patch into thefeed and display it at his command post. Consequently our trainingsimulation can already start to prepare the Incident commander toincorporate a diverse arrary of information sources.

4.2 Fire BehaviorWe also learned how important smoke and fire behavior is to the

firefighters in order affect their decisions. Upon our first showingof initial prototypes to the incident commanders, they looked at oursimulation, with flames swirling up out of the roof (see Figure 5(a)).We artificially increased fire intensity in order to show off the firebehavior and this hampered their ability to evaluate the situationand allocations. They all agreed that every firefighter should bepulled out because that building is lost and might fall at any minute!In our efforts to put a challenging fire in front of them to fight, wehad caused them to walk away from the training. Once we startto add training abilities, such as to watch the fire spread in 3D, wehave to also start to be more aware of how to accurately show afire that incident commander would face. We have consequentlyaltered the smoke and fire behavior (see Figure 5(b)). The smokeappears less “dramatic” to the a lay person than a towering inferno,but provides a more effective training environment.

4.3 Gradual TrainingInitially, we were primarily concerned with changes to the sys-

tem that allowed for a more accurate simulation of what the inci-dent commander would actually see. Alternatively, we have alsoadded features, not because of their accuracy, but also to aid intraining by isolating certain tasks. Very often in reality and in oursimulations, dense urban areas obscure the ability to see where allof the resources (i.e. fire engines) are and prevent a quick view ofthe situation (see Figure 6(a)). To this aim, we have added a newmode using the 3D, but having the buildings each have no height,which we refer to as Flat World (see Figure 6(b)). By using thisflat view, the trainee is allowed to concentrate on the allocating re-sources correctly, without the extra task of developing an accurateworld view with obscuring high rise buildings.

4.4 User IntentA very important lesson that we learned from the LAFD, was

that the incident commander cannot be given all information for theteam and thus the human does not know all about the status of theteam members and vice versa. Consequently, this lack of completeawareness of the agent team’s intentions can lead to some harmfulallocations by the human (incident commander). In order for infor-mation to be selectively available to the incident commander, wehave allowed the incident commander to query for the status of aparticular agent. Figure 7 shows an arrow above the Fire Engineat the center of the screen that has been selected. On the left, thestatistics are displayed. The incident comander is able to select a

44

(a) Local Perspective (b) Global Perspective

Figure 4: Local vs. Global Perspectives in the Omni-Viewer

(a) Old Fire (b) New Smoke

Figure 5: Improvement in fire visualization

(a) Normal (b) Flat World

Figure 6: Improvement in locating resources (fire engines and ambulances)

45

Figure 7: Selecting for closer look at a Fire Engine.

particular fire engine and find out the equipment status, personnelstatus, and the current tasks are being performed by the fire fightersaboard that engine. This detailed information can be accessed ifdesired by the incident commander, but is not thrown to the screenby all agents, in order to not overwhelm the incident commander.

4.5 ScaleIn addition, we have also learned of new challenges that we are

currently attempting to tackle by enhancing the system. One of thebiggest challenges in order to start simulating a large urban fire isthe sheer scale of the resources that must be managed. Accordingto the fire captains, in order to respond to a single high rise build-ing with a few floors on fire, roughly 200 resources (fire engines,paramedics etc.) would need to be managed at the scene. Coor-dinating such a large number of agents on a team is a challenge.Also, as the incident scales to hundreds of resources, the incidentcommander ends up giving more autonomy to the team or else facebeing overwhelmed. We argue, that adjustable autonomy will startto play a bigger and more essential roll in allowing for the incidentcommander to monitor the situation.

5. LESSONS LEARNED FROM TRAININGEXERCISES

5.1 Training ExercisesIn order to study the potential of DEFACTO, we performed some

training exercises with volunteers. These initial experiments showedus that humans can both help and hurt the team performance. Thekey point is that DEFACTO allows such experiments with trainingexercises and more importantly allows for analysis and feedbackregarding the exercises. Thus trainees can gain useful insight as towhy their decisions led to problematic/beneficial situations.

In particular, some of our initial experimental results were pub-lished earlier in [8], but now we are able to provide analysis andfeedback. The results of our training exercise experiments are shownin Figure 8, which shows the results of subjects 1, 2, and 3. Eachsubject was confronted with the task of aiding fire engines in sav-ing a city hit by a disaster. For each subject, we tested three strate-gies, specifically,H, AH (individual agent, then human) andAT H(agent team, then human); their performance was compared withthe completely autonomousAT strategy.AH is an individual agentstrategy, tested for comparison withAT H, where agents act indi-vidually, and pass those tasks to a human user that they cannot im-mediately perform. Each experiment was conducted with the same

AH ALL

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

NUMBER OF AGENTS SENT TO BUILDING

PR

OB

AB

ILIT

Y B

UIL

DIN

G S

AV

ED

AH ALL

Figure 10: AH for all subjects.

initial locations of fires and building damage. For each strategy wetested, varied the number of fire engines between 4, 6 and 10. Eachchart in Figure 8 shows the varying number of fire engines on thex-axis, and the team performance in terms of numbers of buildingsaved on the y-axis. For instance, strategyAT saves 50 buildingwith 4 agents. Each data point on the graph is an average of threeruns. Each run itself took 15 minutes, and each user was requiredto participate in 27 experiments, which together with 2 hours ofgetting oriented with the system, equates to about 9 hours of exper-iments per volunteer.

Figure 8 enables us to conclude the following:

• Human involvement with agent teams does not necessarilylead to improvement in team performance.Contrary to ex-pectations and prior results, human involvement does notuniformly improve team performance, as seen by human-involving strategies performing worse than theAT strategyin some cases. For instance, for subject 3AH strategy pro-vides higher team performance thanAT for 4 agents, yet at10 agents human influence is clearly not beneficial.

• Providing more agents at a human’s command does not nec-essarily improve the agent team performance.As seen forsubject 2 and subject 3, increasing agents from 4 to 6 givenAH andAT H strategies is seen to degrade performance. Incontrast, for theAT strategy, the performance of the fully au-tonomous agent team continues to improve with additions ofagents, thus indicating that the reduction inAH andAT Hperformance is due to human involvement. As the number ofagents increase to 10, the agent team does recover.

• Complex team-level strategies are helpful in practice: AT Hleads to improvement overH with 4 agents for all subjects,although surprising domination ofAH over AT H in somecases indicates thatAH may also need a useful strategy tohave available in a team setting.

Note that the phenomena described range over multiple users,multiple runs, and multiple strategies. Unfortunately, the strate-gies including the humans and agents (AH andAT H) for 6 agentsshow a noticeable decrease in performance for subjects 2 and 3(see Figure 8). It would be useful to understand which factors con-tributed to this phenomena from a trainee’s perspective.

5.2 Analysis

46

0

50

100

150

200

250

300

3 4 5 6 7 8 9 10 11Number of Agents

Bui

ldin

gs S

aved

A H AH ATH

(a) Subject 1

0

50

100

150

200

250

300

3 4 5 6 7 8 9 10 11Number of Agents

Bui

ldin

gs S

aved

A H AH ATH

(b) Subject 2

0

50

100

150

200

250

300

3 4 5 6 7 8 9 10 11Number of Agents

Bui

ldin

gs S

aved

A H AH ATH

(c) Subject 3

Figure 8: Performance.

2

2.5

3

3.5

4

3 4 5 6 7 8 9 10 11Number of Agents

Age

nts/

Fire

AH ATH

(a) Subject 1

2

2.5

3

3.5

4

3 4 5 6 7 8 9 10 11Number of Agents

Age

nts/

Fire

AH ATH

(b) Subject 2

2

2.5

3

3.5

4

3 4 5 6 7 8 9 10 11Number of Agents

Age

nts/

Fire

AH ATH

(c) Subject 3

Figure 9: Amount of agents assigned per fire.

ATH ALL

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

NUMBER OF AGENTS SENT TO BUILDING

PR

OB

AB

ILIT

Y B

UIL

DIN

G S

AV

ED

ATH ALL

Figure 11: ATH for all subjects.

We decided to a more in depth analysis of what exactly was caus-ing the degrading performance when 6 agents were at the disposalof the incident commander. Figure 9 shows the number agents onthe x-axis and the average amount of fire engines allocated to eachfire on the y-axis.AH andAT H for 6 agents result in significantlyless average fire engines per task (fire) and therefore lower average.Another interesting thing that we found was that this lower aver-age was not due to the fact that the incident commander was over-whelmed and making less decisions (allocations). Figures 12(a),12(b), and 12(c) all show how the number of buildings attacked donot go down in the case of 6 agents, where poor performance isseen.

Figures 10 and 11 show the number of agents assigned to abuilding on the x-axis and the probability that the given buildingwould be saved on the y-axis. The correlation between these val-ues demonstrate the correlation between number of agents assignedto the quality of the decision.

We can conclude from this analysis that the degradation in per-formance occurred at 6 agents because fire engine teams were splitup, leading to fewer fire-engines being allocated per building onaverage. Indeed, leaving fewer than 3 fire engines per fire leadsto a significant reduction in fire extinguishing capability. We canprovide such feedback of overall performance, showing the perfor-mance reduction at six fire engines, and our analysis to a trainee.The key point here is that DEFACTO is capable of allowing forsuch exercises, and their analyses, and providing feedback to po-tential trainees, so they improve their decision making, Thus, inthis current set of exercises, trainees can understand that with sixfire engines, they had managed to split up existing resources inap-propriately.

47

0

50

100

150

200

250

300

2 4 6 8 10 12

Number of Agents

Bu

ildin

gs

Att

acke

d

AH ATH

(a) Subject 1

0

50

100

150

200

250

2 4 6 8 10 12

Number of Agents

Bu

ildin

gs

Att

acke

d

AH ATH

(b) Subject 2

0

50

100

150

200

250

300

350

2 4 6 8 10 12

Number of Agents

Bu

ildin

gs

Att

acke

d

AH ATH

(c) Subject 3

Figure 12: Number of buildings attacked.

6. RELATED WORK AND SUMMARYIn terms of related work, it is important to mention products

like JCATS [9] and EPICS [4]. JCATS represents a self-contained,high-resolution joint simulation in use for entity-level training inopen, urban and subterranean environments. Developed by LawrenceLivermore National Laboratory, JCATS gives users the capabilityto detail the replication of small group and individual activities dur-ing a simulated operation. At this point however, JCATS cannotsimulate agents. Finally, EPICS is a computer-based, scenario-driven, high-resolution simulation. It is used by emergency re-sponse agencies to train for emergency situations that require multi-echelon and/or inter-agency communication and coordination. De-veloped by the U.S. Army Training and Doctrine Command Analy-sis Center, EPICS is also used for exercising communications andcommand and control procedures at multiple levels. Similar toJCATS however, EPICS does not currently allow agents to par-ticipate in the simulation. More recently multiagents have beensuccesfully applied to training navy tactics [10] and teams of Un-inhabited Air Vehicles [1, 2]. Our work is similar to these in spirit,however our focus and lessons learned are based on the train ofincident commanders in disaster rescue environments.

In summary, in order to train incident commanders for large scaledisasters, we have been working on the DEFACTO training system.This multiagent system tool has begun to be used by fire captainsfrom the Los Angeles Fire Department. We have learned somevaluable lessons from their feedback and the analysis of some ini-tial training exercise experiments. These lessons were gleaned fromLAFD feedback and initial training exercises. The lessons learnedfrom the feedack from the LAFD include: system design, visual-ization, improving trainee situational awareness, adjusting traininglevel of difficulty and situation scale. We have taken these lessonsand used them to improve the DEFACTO system’s training abili-ties. We have conducted initial training exercises to illustrate theutility of the system in terms of providing useful feedback to thetrainee. Through DEFACTO, we hope to improve training tools forand consequently improve the preparedness of incident comman-ders.

7. ACKNOWLEDGMENTSThanks to CREATE center for their support. Also, thanks to Fire

Captains of the LAFD: Ronald Roemer, David Perez, and RolandSprewell for their time and invaluable input to this project.

8. REFERENCES[1] J. W. Baxter and G. S. Horn. Controlling teams of

uninhabited air vehicles. InProceedings of the fourth

international joint conference on Autonomous agents andmultiagent systems (AAMAS), 2005.

[2] S. Karim and C. Heinze. Experiences with the design andimplementation of an agent-based autonomous uavcontroller. InProceedings of the fourth international jointconference on Autonomous agents and multiagent systems(AAMAS), 2005.

[3] H. Kitano, S. Tadokoro, I. Noda, H. Matsubara, T. Takahashi,A. Shinjoh, and S. Shimada. Robocup rescue: Search andrescue in large-scale disasters as a domain for autonomousagents research. InIEEE SMC, volume VI, pages 739–743,Tokyo, October 1999.

[4] L. L. N. Laboratory. Jcats - joint conflict and tacticalsimulation. Inhttp://www.jfcom.mil/about/factjcats.htm,2005.

[5] D. V. Pynadath and M. Tambe. Automated teamwork amongheterogeneous software agents and humans.Journal ofAutonomous Agents and Multi-Agent Systems (JAAMAS),7:71–100, 2003.

[6] P. Scerri, D. Pynadath, and M. Tambe. Towards adjustableautonomy for the real world.Journal of ArtificialIntelligence Research, 17:171–228, 2002.

[7] P. Scerri, D. V. Pynadath, L. Johnson, P. Rosenbloom,N. Schurr, M. Si, and M. Tambe. A prototype infrastructurefor distributed robot-agent-person teams. InAAMAS, 2003.

[8] N. Schurr, J. Marecki, P. Scerri, J. P. Lewis, and M. Tambe.The defacto system: Training tool for incident commanders.In The Seventeenth Innovative Applications of ArtificialIntelligence Conference (IAAI), 2005.

[9] A. S. Technology. Epics - emergency preparedness incidentcommander simulation. Inhttp://epics.astcorp.com, 2005.

[10] W. A. van Doesburg, A. Heuvelink, and E. L. van den Broek.Tacop: A cognitive agent for a naval training simulationenvironment. InProceedings of the fourth international jointconference on Autonomous agents and multiagent systems(AAMAS), 2005.

48

Section 2

Agent-Based Simulations (Agent Models and Teamwork)

49

Swarm-GAP: A Swarm BasedApproximation Algorithm for E-GAP

Paulo R. Ferreira Jr. and Ana L. C. BazzanInstituto de Informatica

Universidade Federal do Rio Grande do SulCaixa Postal 15064 - CEP 90501-970

Porto Alegre / RS, Brasil� prferreiraj, bazzan � @inf.ufrgs.br

ABSTRACT��������� ��������������������������������� !�"���$#������%������ ���&��'��(���)��������*+,���&����-��*-./�102�*���� !�3������ !�����4����*�56��7��.�������4�/������984��56#:�4����"��;�/��=<���=���>�( 6���&���4�- !������?A@3+��B�/������=���>���(��*-���&������4�(+���0C�1D:���-�; !�2�2<�-'�'����E���F���/��=�����D�.������E���&�G' �����4�H��*���' �JIAK2���-���/�-�ELM�-�/�����'��N-���O '�'���*���������GPA����D�'��- RQSI$<TL O PVU>? WX��#/���4#:�����M�B�/�Y0C�-'&��' ������7��+� ���1��#�#�����K/� 6�Y������+/�3���4'�./����4������I$<TL O PZD��������9�4�9��+/����+��-��������<�*���'[�/�02 ������\���3' ��D:���H !�2�/��' �M�]���4 ^*-��'��4����-�H�������2*� ��'V�������*����Q_�=`3���� !�U>a�*���'�' �-�Eb�`"�Y�� �<=L O P�?2@3+���"��'��4����7��+� c./���-�"�J#����4D���<D��'���=���*H�/��*-��� ���; !�����-'_a�D��������d�4�6��+��M����*- ��'e �/���-*����3���-���/�-�/*�5����#:�>���%���� !��/�,*����������X�����82�-?EI[��*+f���4�-���9+����9���X��&�� 02 �/.���'��+/������+��4' �6./�����!���4������+��>�3`�7��+;�H�=��� ).�'�./�3��������*- �������6`����+6��+�������82�J���,*-�4 !#/./������+����J���-�&���-��*>5�?g@3+/�;b2`3���� �<TL O Ph !�-�����'���`i*-�4 ! 9.��/�*�������4�,���&�E./���-�F�� !#/'��J !��*>+����/ �� !�-?3W\�H��+���`��+&�Y�j��+��3b�`"���� �<TL O P,��*+���-0C�������>`"�Y��/�$04����5B*-'��4���V���(��+/�V�4�������*+/ ��0C���GD�5G�)����������5G*��-�������'��N-���E��#/#/���C��*>+e?

Keywordsk �Y���4�;b�*���'��El;./'7��]���4�-���!b25��=���- !�-aA@j����8m�����Xn(�-���4.���*-� O '�'���<*��Y�� ���o�� O �4�-���Gb25��=���- !�-a����4'�'��-*����0C�p���&�qI[ !�>���4�-��� O �4�����r �-+���02 ���

General TermsO '�������7��+� !�����&�;I$K�#:����� !�-���������4�

1. INTRODUCTIONO �4�-�����$���-*+��/�4'��4��5���[#&�Y���[���:��+��"��'��4D&��'&�>�������V���� !#����Y02 �/��/������=�����$ 6�������4�� !�-����?�l;.�'7�� �����-���$�=5��=���- !�s�������$�"`� ���V�����������������4'��9���&�g���-*+/�� t�.��-�M��+&�Y�H+��Y04��D:���-�p.������p���H�G����`u`3�-5���Z���-����4�f����%���� 6�Y����4�f�=5��=���- !�H���,#/���Y02 �/�d�/��*-�����4�v��./#�#:�������;�- !�����4����*�5d��7��.&�Y����4���-?Ww+��-�qx����,x��4+��������-a(#&�Y��� !�����*-�-aM���&�q����+��>�;#/�����%�-��������&��'��`�����8!��!������ !�[���J��������*+G���&�����-��*�.��F0��*���� !�V���j�/������=���>�"��+/�-��������� !�; 9.��=�G*-���������&�����Z��+��-7�,��*�����4�/�E*-�4�/��]����������f��+��Z����+�<�����-yj��*����027�� ���9���&�g��+������-���>���'V#:�����%���� 6���/*-�!���3��+��-7�H�����������-?

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.AAMAS’06 May 8–12 2006, Hakodate, Hokkaido, Japan.Copyright 2006 ACM 1-59593-303-4/06/0005 ... z 5.00.

@3+���$82��&�B���������.�������4�9��A����� 6�����*���'�'759�25��&�� !�*4a�`�7��+9����`Xx������#:�����1��#�#:���Y�� �/�6`�+��'��9����+/�����1�����J��K2�� �/�4./ ��+/���ea:�=�����-�>���M*-'��4�=<����ZD�5\']���&����' ���;���){&���2�/�����aV����*4? r ����]���-�-a3�/������=�����6��'7`3�-5/���20C�4'�0C�����1']�Y���4�F�2./ )D:�>�[���e#����4#�'��4a�D:����+!���V02 *>��� 6�[���[`3��' ':������-��*-./�����-?@3+/�; !���� 0��Y�� ���f�|���!�=��.&�25/��/�Z*��2����/���������4�q�� !�4�/�p��*����������d�1���-��*�.��(��*-�-��������9*-���d��' ���JD:���|�4./�&�� ����+��� )./'7��]���4�-���V�=5��=<���- !�-y�*-�� ! )./��7�}5�?Z@3+/�d������������*+f���-�4����/��/�p 9.�'7�� �����-���)�=5��=<���- !�)*-���������&��������q��+����4./�4+~�/��=�����D�.������m�����8f��'�'���*�������4�w+������+���`��d����4��7x�*��������4��0�����*��-�3��6��+/�F' ���=�3�%��`w5����Y���1�7�-�/a��4��ae���Y�S?� 5��&�� ! *H�����,' �Y���4�)��*-��'��)�-�2027���4�/ !�-�����(��������2�/./*-�J����`�*+&��'7<'��-�����-�[��9��+/ �����-�����Y��*>+e?[@e��4��7�� ���&��'&��#/#/���4��*+��-�[�����30C����5H �/���|<x&*� �����3��!��+/���82��&�����s��*-�-��������)���&�)��+������(�����(�|��`o�/��`w�=��.��/��-����-�4����/��/����+������./D��=�-*>�J�����Y�S?@3+/�d�����8m��'�'���*�������4�~#/����D�'��- � �f�/������=�����)���-��*-./�G*-��.�' �mD�����-�-�o���;�XI$K2���-�&�����wLM�-�/�����'��N-��� O ��������� !�-���dPA����D�'��- �QSI$<L O PVU��7�Y����?VI$<=L O PX��"���d��'�'���*��Y�� �4�G#/����D�'��- �`�+/ *+d�>K��� !����-���+�����������4�/ !�-���$���/�����82�j���M���4���2���� 6��K�� ! N� �/�F���������'�����`3����e?@3+���F�������'j���>`"����,��F*-�4 !#/./�����;��5��&�� ! *-��'�'75�?(@j����82�1���- 6��������-����./��*-�-�d�����v*-' �������-�d���1*���#&��D/�' 7����-�6���\D:�E��'�'��2*-��������?�@3+�������-�����[+&��0C���M*-�>�������6�� !��.����A���&���-���4.���*-�(���&�)�/7���������2�['��-04�-'����������*- �������X`�7��+X����*+X*-' �����9����*-��#&��D��'�7�T5�?E@3+/�!�����82�J*����XD��������������-' �����-�Z�����;��+��-7�F�������'���./ )D:�>�(���F����t�.�7�����E*���#&��D/�' 7����-�*����E*+&���/�4�B�Y0C�>����� !�4?W\�!#����4#:�����d�G���Y0C�-'V��#�#�����K/� 6�Y�� ���X��'�������7��+� ����,I$<TL O PD&�������!���!��+��"��+/�-���������*���': !�2�/��' �[���j���0������4�6���s' ��D:���3��6����*- ��'������-*>���1*-�4'��4�/��-�-aj*���' '��-�Xb�`"�Y�� �<TL O P�?:@3+���B��'��4�������+/ ^������-������BD��"�� !#/'��(���&�)�����-*����0C�"���B���4'�0C�(' �Y���4�F��*���'��F����=����/*-�-�V���sI$<L O Pf`�+��>���B��+/�H��'�' ��*-���� �4�,��F�Y��*>+/�0C���G��,���� �=�����D/./�����d�%����+�<��4�e?X�����4#:��������04�E���4�-�����9��./���/ �/�Zb2`3���� �<TL O P������G��'�'���`3�������*-������/��&�Y���H7���F��*�����4�/��`����+;' ��`h*��4 ! )./�/ *-������4�e?W\�"�|��*-.��V���6�B��#�#����C��*+!D��������)�4��*-�4'�������-����������*- ��'&�������*����QS��'����9*-��'�'����6�=`"�Y�� !�U>a�`�+������F#/'��-���T5)���s�-02 �/�-�/*-�-�V���s�-*-��' ���4�*���'��.�*�*-�-������K���=��a3���-��#/����d��+/�;��#�#������-����' ��*>8X���F��K�#�'��*-7�!*-������/7<�&�Y����4�s?;@3+/�-���6+2./�&�����-�/�B���"��+/�4.������&���H �/���-*����9������#��H���;��+��*+&���/�4�-�M �,��+��J�-�2027���4�/ !�-���M�����,���6��+/�J���-�����M���[��+��J*-�4'�����5.�����/�6��+��J#�' ���=�� *���}5g��Z�/�02 ������Z���V']��D:����?H@3+/�����)�����J`��-'�'A��K2<#:����� !�-�������d��+������������*���'� !�2�/�-'�����+����F*-��#���./���-����+���(#�' ���=���*���}5�?W\�H������#��F�����G��K2���-���d��+/ �" !�����-'��"���).����B��,b2`3���� �<TL O Pj?b�`"�Y�� �<=L O P�`3���Z��K�#:����� !�-����������u�q��� !#�'��X*-�-�������'��N-�-���� )./']�Y����4������027���4�/ !�-����?���7���=��a4`3�"�- !#/����*���'�'�5)�/�>x&�/����+��3#&�Y<��� !�����>���J���"��+��6�4���4#������\�=`3���� � !�2���-'A��+&���J 6�YK/� !�N-�-�1��+���=5��=���- �����`3�����? O �|������as`��!*��4 !#&�Y���)��+/��#:�����|���� 6���/*-��������+��b2`3���� �<TL O P�`�7��+w�Z*-���������'��N-���~����������5v��'�������7��+� E? O ����K2<#:�-*����-�ea���+/�H*-���������'��N-���g��#�#����C��*+,�4.���#:�����%���� !�Mb2`3���� �<TL O Pj?� ��`��-0C�>��a"�4./�d��'�������7��+� �#��>���%���� !��`��-'�'F��~��+��EIA<TL O P���*-��<

50

�&�Y����f��*+���-02����\����`3����/�G���C��'���`3���pQ_�4�w��0C�>����4��U���+&���~��+������������5d�����-�-?[�� ����'�'75�a�`3�1 !������./���F��+��M� !#&��*��"��d��+��Bb2`3���� �<L O P~#:�>���%���� 6����*-�1`�+��-�;�����8��F�Y���H���-0C�����'j������������-' ��������?��M./���K2���-�/�� ���g��,��+��H�=`"���� R !�2�/�-'s���d������'j`�7��+,��+��H�����82�M���������<���-' ��������,� !#/���Y0C�-����+��B��0C������4�H#:�����|���� 6����*-�1 �,�4���d?@3+/��[#&��#:���[��[�����C���/�N-���!���A�|�4'�'��Y`��-�Vb��-*>�� ���6�1���-��*����D:�-�A��+��I$<=L O P���H��������'��-�Ab��-*>�� ���\�G�/���*-./���1��+��9����' ��0������H����#:�-*����B���' ��D:���M�� 02�����4�E �;���2*�]��'��������*��F*-�4'�������-�-�eb���*�����4�E�!��������2�/./*-�-���+���b�`"�Y�� �<=L O P��4b���*�����4�6�(��+���`��A��+��3�- !#/����*���'�'�59�-0���' .�������4����Ab�`"�Y�� �<=L O P������&�;b2�-*�����4�E�J#����-���-�����"��./�"*-�4�/*-'�.�����4�/�F���&��%.���./���B�/7���-*>�� �����(���$��+/��"`3����8�?

2. GAP AND E-GAP@3+/��LM�-���>���'� N���� O ������4�/ !�-���jPA����D�'��- �Q�L O P�UA�7�-���2����"�4���/<�����'���'�'���*�������4�~#/���4D/'��- �`�+/ *+f��K/�� !����-�J��+/�G��������4�/ !�-������������82�1���;�����-�����-as���-��#:�-*>�� �/�G��+������4�-�����B*���#&��*������-�-a� 6�YK/� !�N�<����6���������'j����`3�����?F@3+/�)L O P~`3���M�>K������&�/�-�;D�5\�7�Y���$���!*���#/<��./���J��5����� !�*H�/�� 6������F���&�E�����������-#:�-�&���-�/*-��-�M�� !�����������82�-?@3+���9�>K�����������4�X��9*���' '����mIAK2���-���/���2<TL O P�QSI$<TL O PVU>? O �J����]�D:���%�����4a&��+/�B�����8E��' '���*��Y�� �4�g��,' �������9��*���'��)���&�E��5����� !�*1�-�20�7<���4�/ !�-�����"*����ED:�B !�2�/��' '����;���F���EIA<TL O Pj?��M�>K���a2`3�H�/����*����D�������;�|���� 6��' �N-�)L O Pw�����E7���(��K2���-������4�ea�`�+��*+, ����+��1�%��*-./�M�����+����#���#:����?@3+/�B�C��#E*����ED:�M�%���� 6��'��N-���E�����|�4'�'��Y`��-? k ���(./�(�/��x����B ¡�����+��3�����[���:�����8��$���MD:����' '���*��Y�����6���&�M¢g��+��3�����[���������-�����-?[I[��*+�����-���M£3¤d¢q+&���F�!'�� !����-�,�� !��.����(���A���-����./��*-�J¥Y¦[���!#:�����%���� ��'�'[�����82�6QS�;����/�4'��!�T5�#:�����"�������4./��*��6§�bg.�������U>?!Wo+��-�\�G�����8¨ ¤m ^ �1��K��-*-.������gD�5Z���4�-���B£�ae�����8 ¨ *-������./ !�-�1©�¦ ª!.���7���1���£�y �1���-���4.���*-�4?)IV��*+X���4�-���B£(��'����E+����B�d*���#���D/ '�7�}5g���G#:�����%���� ����*>+G�����8 ¨ ���0C�-�;D�5G«�¦ ªdQS�9¬~«�¦ ªJ­���U>?@3+/�F��'�'���*�������4�d 6������7K�®Ja4`�+��>���F¯�¦ ªM��[��+���0���'�.���������+���7<S��+����`����&�J�T<S��+;*��4'�.� !�ea/����4�0C�-�;D�5GI[t2.�������4�p�4?

¯�¦ ªM° �±£}²��"£}³B¯�´%´|µ�©>¯2¶=·�¸!¹>º6�»µY¶�¼&·�¥�½(£}³�· Q=�YU@3+/�BL O Pm��K/�� !������V��+��F 6�YK/� 9.� u#/����x/�3 6������7K6®Ha�`�+��*+ 6��K�� !�N-�-�d��+/�p�=5��=���- ¾���>`"�Y����4�04�-��D�5��2aM��.�D��T�-*��;���f��+�������-���������-���4.���*-�3'� !7�����������A�����H��+��V*-�4�/�=���������A���&+&��02����F����'75�4�/�H�����-���F��'�'���*��������;�����-��*+;�����8:?V�

®o°w¯2¥�¿2ÀG¯2Á:Â�à X¦SÄ�ÅX

ª�ÄCÆ «�¦ ª�ÇV¯Ã¦ ª Q��4U

��.�*+G��+����È £[¤�¢�É XªYÄCÆ ©-¦ ª�Ç�¯�¦ ªJ­v¥Y¦

�����È ¨ ¤E EÉ X¦SÄ�Å ¯/¦ ªJ­��

I$<TL O Pv� !#/���Y0C���(L O Pv��;�}`��6��7�������-���"`"�-5��-�ÊdË%Ë|ÌsÍ4Î/Ï�Ð|Ì:Ñ�Í4Ì&Ñ�Ò-Ï-ÓYÎ&Ð%Ñ�Ï�Ò!Î:ÔpÌ:ÑsÕXÏ-Î�Ò�ÖsÒ4× �����8��Ø��Ù��<��C��#*����~D:�G �����>�����-' �������~D�5f���~���&�f*-�4�/�=����������? O '�'����������<���-' �����-�X�����8��JD�5Z��+���J*-�����=����� ���9 )./�=�JD:�6��'�'���*��������f�����+��M���� !�F�� !�(���)D:�M*-������ �/�>�����GD�5!��+/�F����`3����G*-�� !#�./<�������4�e?Z����'�' ��`�����v�7�Y�Y�Sa[' �>�).��9����x&�/�6Ú_Û�° ��Ü�Ý É-Þ�Þ Þ�É Üsß � a`�+��>��� ÜAà ° � ¨ à-á É-Þ�Þ�Þ�É à�â � ���-�������-�G��+/�Z8�<���+w�����;���)���O � � *-�����=�������/���E�����8��-?�@3+2.��-a���+/�H#&�Y����]��'j����`3����E½ ¦ ª�%���(��'�'���*���������)�����8 ¨ ���9���4������£$��"���0C�-�dD�56IVt�.�������4�G��?

½�¦ ª1°8

>

>

<

>

>

:

«�¦ ª�ÇV¯�¦ ªã£}² È ÜAà ¤eÚ_Û/É ¨�ä¤ Ü$à«�¦ ª�ÇV¯�¦ ªã£}²Zå Ü à ¤eÚ_Û�½(£S¶=¼ ¨ ¤ Ü à�æÈ ¨ à>ç ¤ ÜAà É�¯�¦ ª�è ç~é°��� µ�¶=¼&·�¥�½(£}³Y·QS�CU

ê!ë�ì Î�Ó�íwísîjÑjÎ&Ô\Ð%Í4Î�Ë%Ë|îqÍ4Ì�Ôpï�ðjÏ ë íqÌ�ñ ë Ó6Ï�Ð%Ô ë × @3+/�o�������'����`3����vòó �9*-�� !#�.������\��fI$<=L O Pi���J��+��6��./ ô������+������`3����m��X��+/�6' ���=�)¶1��� !�!�=���-#��-?,§}�\��+���J*�������aA��+��!����<t2./�-�/*-�d���M��'�'���*�������4�����Y0C���9��� !�d��)*-������ �/�����-�v���4������=���+��1������4'��J��' '���*��Y�� �4�,.������G������)��+/�JL O P�? O ����7�� ���&��' '75Ca�,���-' �-5\*-�4�=�9¸�ªd*-��.�' �\D��!./�����\��X��������J���E#�./�����+p��+�����4�������B`�+/�-�Z��+/�-�g�����8 ¨ `3���H�/���H��'�'���*��������\���B��� !�J¶�?@3+��)��D��=��*����0C�)���V��+��)I$<=L O Po �M���G 6�YK� !�N-�)òã�4�0C�-�D�5GIVt�.�������4�;�/?

òõ° XöX

¦]÷�Ä�Å�÷X

ª>÷�Ä�Æ$÷ ½ö¦ ª ÇV¯ ö¦ ª(ø X

öX

ª÷ÄCÆ$÷ Q=� ø ¯ö¦ ª UsÇV¸ öª

Q_��U��.�*+G��+&�Y�È ¶ È £ ö ¤�¢ ö É X

ª÷ÄCÆ$÷ ©ö¦ ª ÇV¯ ö¦ ª ­~¥ ö¦

�����È ¶ È ¨ ö ¤,  ö É X

¦]÷�Ä�Å�÷ ¯ö¦ ª ­��

3. DIVISION OF LABOR IN SWARMSb2�2*�]��'e�������*���*-��' ������-����+/�Y`���0� ���-��*��-�"�����-*��4'��4�4�*���'s��.�*�*-�-����/./�G���p��+/�-7�6�����C���/�N�������4�q`�+��*+q��!�4D/������0C���v��q�/�02�����4�q���' ��D:����a1��#:�-*- ��' �N�������4�saB*-��'�' ��*����0C�p���-�4./']�Y����4�sa1����*4?^� �Y�S?�@3+���������/�����M��+/�G*-�4'��4��5f*+&���/�4�;�Y0C�>����� !�4?v@3+������E*+&�����4���6�������������*- �������~`�7��+~��+/�E#/+&�����E���1*-��' ����5q���-0C��' ��#� !�-����a[��� !�;���5����Y��aC�%���2�6��0����' ��D��'�7�T5Ca�#��������Y����4�!#/���-����.����4a2���&�)*-'� 6�Y���*(*-�4��<�/7����4�/�-? � �-��#/����E��+/ �d������=�� *,04�Y�� ������4�/�G��w*-�4'�����5ey �d*-�4���/7<����4�/�-a����2*�]��'j �/���-*����(�/�)+&��0C�1�-*��4'��4�4�*���'���./*-*-�����-?O ����*- ��'&�������*���*-��'��4��5)`�7��+!+2./�&�2�����/�$������+/�4./�����&�)���e !�- �<D:�����)��#:��������-�9`�7��+���./�!����5X��K�#�'��*-7��*-������� ��������4�e? O �m��&��7<02]��.&��'e`�����8C�>�M*-�����/���F��*-*-�����(��+��B�������/�����$��+/�B*-�4'��4��5��&7�[�T.��=�+&���V�M�_��7��'�5���� !#/' �"'���*���'&��/�|���� 6������4�ea2���&�)�/�B��������[���*>+������������*��2����/���������4�s?E�����4 ô����/�02]��.&��'[`�����8C�>���)���4�����-�C������4�sa$��+��*-��' ����5gD:��+&��02����1�- !�>���4�-�F`"���+/�4.��B����5;�T5�#:�H���V��K�#/' �*-7�1*��2����<�/���������4�g���M#�' ���/�������?M@3+��J8C��5G�%���Y��./���9���[��+���F�- !�������-���MD:��<+&��02�������"��+/�M#�' ���=���*-7�}5G��E���0������4�;���$' ��D:���(����� ���M��+��M*-�4'�����5�7��ù��S?J����'��4����-�1���-��#:���&�E���d*+�����������d*-���&�����������1D�5,�����T.��=��������+��V���������[���:��&���0� ��.&��'C`3����84�����A�-�/�C���4���9��9��+/�30��Y�� ��.��������82�-?@3+/�����./']��N����H��'S?d�7��ú��[#/���-�������H�d !�2�/��'��|���B�����8Z��'�' ��*-���� �4������#/����-�H���B��+��[#/' ���=���*-7�T5B������ 02�����4�H���/' ��D:������H*-�4'�������-�����/����<*- ��'s������-*>���J�7��ù��S?V§T�����>���*�����4�/�F�� !�4�/�� !�� )D:�>���3������+��1*-�4'�����5�����;��+��H ���/�02 �/.���'j#:����*-�-#�����4�g���V'���*���'A�/�-�����(���-��./'��M��Z�6�252<�&�� ! *3����=���� D/./����4�9���&�����82�-?A@3+���A !�2�/��'��/����*����D:�-�$��+���*-�4'�����5�����8d����=���� D/./����4�G.�����/�9��+/�1�=��� 9.�'�.��3#����2�/./*-���6D25������82�3��+&�Y��������~���XD��,#:�����|���� !���q���&�q���w��&�� 02 �/.���'����-��#:�4�����,��+����-��+/<�4' �;���-' �������E���!����*>+E�����8:?�@3+��J������-�/��7�T5;���A��+/ �(�=��� )./' ./�(*����D:�M��������*- �������6`����+G�H#�+/�����4 !�����(*-����*-�������������4�ea��H�2./ )D:���V����-�/*-�4./���������)�� !�����,����/�02]��.&��' �J#:�����%���� !��/�,��+/�!�����8:aA�������25����+/���(t2.�������7������0C�M*�.��1���-�/�����GD�56 ���/�02 �/.���'��-? O �G��&�/�02 �/.���'��+&�Y�M#:����*-�-�04�-�)Q_�4? �/?M���]�����M`"��' 82����G�Y���4.����E����&���4 !'75�U��������8�=��� )./' ./�1+/��4+��>�B��+����Z����B��������*- �������p��+����-��+/�4' �eas+&���H�d+����+����#/����D&��D��'���}5d����#:�����|���� û��+/��"�����8:?

51

O ����.� !����J��+/�1��K���=���-��*��B���A c�����82�"���)D:�1#:�����%���� !���ea�����*+�����8 ¨ +��Y04�g�m³�ªZ�=��� )./'�.��6��������*- ��������?i§}��¢ü��7�������-���d����/7<02 �/.&��'��V*����!#:�����%���� ���+/�- Ea4����*+6����/�02]��.&��'�£j+&��0C�(�M���-��#:�4�/�����+/������+��4' �Hý�¦ ª"��������*- �������H���F�������8 ¨ ?$@3+��V����/�02]��.&��'�£&�-���4�����-���G��+��M�����8 ¨ #:�����%���� 6����*-�M`�7��+E#/����D&��D/�' 7�}5��

þ�ÿ�� � Q�³ ª UA°³��ª

³ �ª�� ý �¦ ª Q��4U@3+/�v#�+�5����*���')��#:�-*�]��'� N-���� �4���������*- ��'! �/���-*��\����*-�������-�X��*���' '����9 !����#/+���' ���4�*���'/#:�4'75��>��+���� i���A�� !#/'75H#:�4'75� !����#/+���� �D�5D����' ���4��=���-?G@3+/��#:�4'75� !����#�+/��� �+&���J�d8C��5,��.�'��)���;�/�����>�� ! �/���+��)���0������4�p���3' ��D:���H��p�����1*-�4'��4�/��-�6� ú��S?)@3+/�)���4' �� �>���H���"��������B*-�4'��4��5p������.���.���'�'75Z��+/��' �Y���4�-�=�9�������1`�7��+\D���;+/���4���J���&��=`������<�'��8C�M�=��`"�-?��M��.���'�'75�as����+/���H�������B������ !�-�/�.� ¡�� N����\���&��%������4�B�%���(�|������?�@3+/��(��(t2./7���B�4����t�.&�����M���)7���������� 6��'s��*����02�<����-�-? r 5H��7�������-���� ������/�( ���/�02 �/.&��'4��+����-��+���']���-a�7�j �j#:������ D/'��V���*���#/��./���H��+/ �M#/+�5��� *-��'�0���������}5,��,��+��H��+������������*���'A !�2�/��'_?1@3+����&�� 02 �/.���'e��+/������+��4' ���(�%���F��+��B�����8��F�/��*������������M#/����#:��������4�&��'�'�5���!��+��J��&���0� ��.&��'j*���#���D/ '�7����-�1���6#:�����%���� R��+/��F�����82�-?M§T�E��+������-�/���4a� ���/�02 �/.&��'��j`�7��+J']�Y���4��*���#���D��'�7�T5B�%���A�(���������/�����8��j+&��0C��)+/ ��+����������&���-��*�5!����#:�����|���� û�����82�(������+����������?@3+/�[����*- ��'��������*����jD:�-+���0�����j���-�- !�����"x/�e��+��$����t�.�7���- !�������e�����5����� !�*F�����6']�Y���4�1��*���' �1�-�2027���4�/ !�-�����-? O �4���2����*-�4./' �G�/��*- �/�`�+��*+!�����89���H#:�����%���� ��T.��=�3���V����*- ��':������-*>���3�/��?[@3+/�Mb2`3���� �<L O P���'��4����7��+/ ���!D&�����-�~���v��+��G��+/�-�����>�� *-��'( !�2�/�-'�����+���`����\��+/��J����*�����4�sa$��/*-'�.&�/��/�;��+���#/���4D���D/�' ��=���*G���-*-�����4�m#�����*-�-����4./ �/���GD�56��+/�1�����&�/����*�5d���&�d��+/�1#:�4'75/ !����#�+/ �� E?

4. THE SWARM-GAP@3+/����� �����b�`"�Y�� �<TL O Pw��M���;��' '���`���+/�)�����-�����1���G�/��*- �/���&�� 02 �/.���'�'75M`�+/�*>+1�����8(������K��-*�./���4a�`�7��+/�4.�������5M82 ���M���������4��<*- ��������s? O �V �g�7�Y�Y�Sa�`��1������.� !����+&�Y����+��(*-�4 ! 9.��/�*���������d���2��������M�%���'_aj���&�E��+����F��+��9�����-�����182�/�Y`i��'�'������82�F��+&�Y�1��+/�4.�' �,D:���K��-*-.������H���j`3�-'�'����j��+��V�2.� 9D:���s���&���4�������$��20C�4'�04���H��H��+�����'�'���<*��Y�� ���g#�����*-�-���-?(@3+/ �M������.� !#��� ���E �������������&��D/'��97�[�4�/�B��+����82���+&�Y�"��+/ �"82�/��`�' �-�/�4�1*����G*-�4 !�F�]���4 c��+/�M�%��*��"��+&���3��+��1�����-�����82����`���+/�6����#��>�����47���G���������82�)���&�\��+/ �9*-���fD:�6' �-�������-�f����'75�4�/*-�4? O �)�|���!��+/�G*-�4 ! 9.��/�*���������saV��f��+��d�|./��./���d`��E������-������d���-' ��K,��+���H������.� !#��� ���p�����Z#:�����|���� ����-�=���M`�7��+p.��/����' ��D�'��*-�� ! ).��/�*��Y�� ���;*+������/�-'��-?O �4�-�����j���b�`"�Y�� �<TL O P,�/��*- �/�V`�+/�*>+H�����81���(��K��-*-.����VD&��������4�v��+��;���� !�E !�-*+�������� �.������vD�5f����*- ��'M��/���-*����-?�@3+��;���-�/<�/����*�59���:��+������4������£s���B�>K/��*-./���3�����8 ¨ ��V�4�04�-�!D�597���V����������&��'��+/������+��4' �GýY¦ ª�a2��+��F�����86�=��� 9.�'�.��"³B�����6��+��M*-���!*-��-���(���$�>K/�><*-.��� ���dÁ2ª1���s�����8 ¨ a2`�+��*+d������-' �������!`����+d����+������3D�5!��� O � �*-�����=����������a&���(��+���`��FIVt�.�������4�,ù2?3@3+/�H*��4���=��������q��(./�����E����E�/���*-��.����H��Y���!���,���-*����-�����!��+��)`3�� ��+��9���3��+���*-��� !*-��-���9�����K��-*-.�����4�;�4�G��+��1*-�� !#�./��Y����4�G������+��M���-���/�-�/*�5�?

þ�ÿ�� � Q�³YUA° ³ �³ �ª � ý �¦ ª � �VÁ2ª QSùCU

I[��*+X�����8p+&���H��+��!���� !�6��������*- �������m�=�� 9.�'�./�-a���+/�����!��9�/�#/���������}5g�4�Z�����8Z��'�' ��*��Y�� �4�e?!@3+/�)�=�� 9.�'�./�H³)��1��+/�)���� !�9�|����-0C�>��5v�����8 ¨ ���&�~����d0���'�./�,`3���G�- !#/7�� *-��'�'75o���������� !����-�v��� 6��K�� !�N-�!��+��d�=5��=���- ô����`"�Y��e?m@3+���)`3���!���4�/�d��+����4./�4+m��+����K�#:����� !�-�����"�/���*-./�������;��;��+/�B�/��K2�����-*>�� ���s?@3+/�,�>K/��*-./����4�q*���� !*-��-���dÁ2ªZ�������2*�]�Y�����q���p��+��;�����8 ¨ ��*-�� !#�./���-�p.�������E��+���������dD:���}`��-�-�X��+��!�2.� )D:�>�J���F��'�'���*������-������82�A���&�H��+/�V�������'��2.� 9D:���j���������82��`�+/�*>+)����������']�Y�����J���(�����8¨ D�5! !�����������$� O � � ���-' ������4�/��+��#s?V@3+��F������ !��� � ª �2���&��� � ª ����-#����-���-������+/�-���B#������ !�����>���"���-��#:�-*>�� 04�-'75�?

� Ð|Õ:ðjÓ ë������ ð$Ô�� ë ÓBÌ��[Í��jÎ&ÑjÕ ë í\Ô ë Ò�Ò-Î�Õ ë Ò�Î&ÍCÍ4Ì�Ó�íjÐ%ÑjÕ,Ï-ÌÏ�� ë Ñeð$Ô�� ë Ó!Ì��(Î�Õ ë Ñ�Ï�Ò

Á2ªM° (Ý! #" $ � "" % � " ³�·&� ��ª'� é°(� �(ª)� æ � ��ª'�'*��� µY¶�¼&·�¥�½(£}³�· Q��4U

b�`"�Y�� �<=L O Po./���-�1��+/�)#:�4'75� !����#/+���� ¡���G������.�#g��+��)�����-�������+/������+��4' ���d��*-*�����/��/�X���p��+��,���4�-�����6*���#���D/ '�7����-�-?�I[t�.&��������� Ò ë Ï�+ ��+/�;���4�-�����)��+����-��+���']�fýY¦ ª,���G�G ! �2./�J��+��d*���#&��D/�'���}5©�¯�,&¯2©�£_¶TºC¦�Q ¨ UF���3���4�-���B£3���G#:�����%���� ������8 ¨ !��2.��J�;Q_D:�-*���.������+��B��+����-��+���' �g�����E*���#&��D/�' 7�}5g�Y���9��20C�>�����-'75E#/����#:��������4����'�0���'7<.����U>a��� 04�-�;D�5GIVt�.&�Y����4�E��?ý�¦ ª1°�� ø ©>¯�,�¯�¹>£�´%£_¶TºC¦=Q ¨ U QS�CU

O �4�-�����A��.��/������Hb2`3���� �<TL O Pp*-�� ! )./���*��Y���"�=5���*+/�������4./��'75.�����/�E�d���48C�-�ZD&�����-�p#/�������2*��4'_?!Ww+��-�\���X���4�-���B���-*-�-�04�-�B��+�����484�-�sae7�B+����M��+��J�����+2�1���G�/�����>�� ! �/�H`�+��*+Z�����8,7�1`��'�'[��K���<*-.����4?V�M��*-�M7����(�/�����F`�7��+G��+��M��K��-*-.�����4�sa�7�����-�&���3��+��F���484�-����v��������+����E���4�-����?�@s�f*-�4 !#/'������,��+��Z��'�'���*��Y�� �4�sa1��'�'B�����-����� )./�=�3���-*-�-�0C�1��+/�1����8C�-�;���&�d���8C�1����(���-*-�����4�s?@3+/�1���484�-�; !�-�������4�H��'����!*��Y����������F�)�����(���$��./#�'��-�BQ_£�É ¨ U�`�7��+#&��7���)���M���4�-���������X�����8:?\@3+/��) !�-�������4�d��/�|���� !�J��+��d�����-�������D:��./�!`�+��*+w�����E��+/�,��0����' ��D/'��,�����82�d���&�f`�+��*+w���4�������!����<*-�-�0C�-�\��+/�!����8C�-�\5C���dQ%���g��0C��]�X�/�����/'���*8��U>? O �B`3�6*����X���-�4a��+����2./ )D:���1���"*>+����/�4���p !�-���������-�J��p��+��!��'�������7��+� ô��J�-t2.���'���G��+/�)�2.� 9D:���1�������4�-�����-?!@3+/�����N-�!���3��+/�9���484�-�p !�-�������4����#/����#:��������4����'����p��+/�;�2.� 9D:�������M�����82�-?w�$���./���\�E��+/��`��!��+��'�������Y�!������`�����Z �m��+/�d�2.� 9D:���J���F !�-�������4�-�)*>+����/�4���f�� !�������+��B���4�-������������*+���-04�H���E��'�'���*�������4�s?O ������*+6��� !�F�J�/7�������-���3���4�����"�=�����������+/�F#�����*-�-���-a�*�����������/���+��M����8C�-�;���&�d���-�&�� �/�)��+/�Mx����=�� !�-���������4?V@3+��H���4�-���������-'��-*����������/�� 6'75d��+��H���>K��F���4���2�F���!���-�&�;��+/�H����8C�-�e?(@3+���M��F� !#�����<����������1��'�' ��`q��'�'/���4���2�������M 6��8C�[��+�����A�/�-*� ��������$`�7��+��/7������������� !��.������"���[��0����' ��D/'��H�����82�-?����4.����!�!��+���`��M��+��J*-�4 ! 9.��/�*�������4�,#/����*-�-���1�=���������/�d`�7��+�����-���.-$a[`�+��*+o���������� !���������+��;�����82�67�6`"�������6���\��K��-*�./���4?O �]������a/-����-���/�6��+/�,����8C�-�q���f���4�-����01aV������/�4 !'75~����' ��*�������?@3+���(#�����*-�-����x������+����"`�+��-�E��'�'j�����-���������-*��-�0C�1��+��M����8C�-�s?@3+/�(��'��4����7��+/ ¡�(��������'��[��+��(b2`3���� �<TL O PZ !#/'��- !�-�����������s?@3+��p���4�-�����;�=��Y���;`�+/�*>+��m./�� t�.��, �������4���E ���-����7x&*��Y����4�o�����'�'���`û��+��G��'7�������&�Y���G���484�-�f*>����������4�e?f@�+��;���4�-�����9��+/���-��+/�4' �/������;�����d��*-*-���������\����*+q�����-���6*���#&��D��'���}5�a3� !#/' �� !�-��������,��+��#:�4'75� !����#/+���� c�4�0C���ED�5GI[t�.&�Y�� ���,�2?

52

� Ð|Õ:ðjÓ ë213�54 Ë%Ë%ð�Ò-Ï�Ó�Î�Ï�Ð|Ì:ÑXÌ��[Î/Õ ë Ñ�Ï�Ò76:Í4Ì�ÔXÔgðjÑ�Ð%Í4Î/Ï�Ð|Ì:ÑXï�ÓYÌ98Í ë Ò�ÒI[��*+f�����-���)*-���������4'��J`�+�����+/���97�)��97���J��� !�����,���7�� �����G�����'�'���*�������4�hQ_'�����G�CU>?p§��(7����9��+/�d*-�����4a[��+/�G�����-����*>���������-�9��+�����484�-�d 6���������4�M �/*-'�.&������)��'�'������82�"���&�6#:�4���������J��+&�Y���/�J�����8�����'�'��2*-�������f5C���;Q_'�����Z���YU>?\§}����+��G���4�-������)�/���) �f*+������4�G������484�-�!*>����������4�ea27�V`3��7�3./�����'�7�V����*-�-�0C�-���M���48C���! !�-�������4�JQ_'������-�CU>?�M�/*-�1����*>+E�����-���(+&���"���-*-�-�0C�-�G��+/�M���484�-�; !�-�������4�4a/��+/��5G��' '�/��*- �/�(`�+/����+��>�3���3�����V���J��K��-*-.����F�����(�����8:?�Q_'� �/�J���4U���*�*-����2<����1���1��+/�-7�[���-���/�-�/*�5J���B�>K/��*-./���3��+���A�����8:aC�4�0C�-��D�59��t�.&��������ù�?V@�+���F���-*-�����4�,��'����!�/�-#:�-���/�"���G`�+�����+/���"��+��B�����-���(+&���"��+�����-����./��*-�;`�+��*+~ �bm����t�./����-�vD�5m��+/�G�����8:?q@3+��;����8C�-�v !�-�=<�������9��F !�2�/7x����G���!*-�����������(��+/�H��/�|���� 6������4�E��+&�Y�(��+��H���4���2���[��K��-*-.�������M��+/�"���-'���*������)�����8��-?A@3+/�(t�.�������7�T5J���:���-����./��*-������������-���(+&������(�/�-*>�����������;D�56��+��B�� !�4.����3����t�.�7�����G�>K/��*-./������E�����+��M�����8ZQ_' ��/�J���CU>?O �B��+/�!���&�p���"��+/��H#/����*-�-���-a���+/������8C�-�\ !���������4�!��J�������H������;���4�-���"������/�� !'�5!���-'��-*����-�E�� !�4�/�J��+������B�����-�����"`�+/�*>+G+&��0C����������-*��-�0C���d��+��M���484�-�;��;��+/��(��'�'���*�������4�fQ_'�������M���9���!�4�4U>?

ÊdË|Õ&Ì&Ó�Ð|Ï��$Ô � b�`"�Y�� �<=L O P�Q_ ���"¯�¿�·��:¶;:�¸�U�4�(��'�' ��*Y���4./�������=< ����2� Ë]ÌeÌ&ï���  >< �����(���A��'�'e�����82��/� �5< �Y0���'�]��D�'��-n(�-���4.���*-�CQ�U>��2� ��Ì&Ó6Î&Ë%Ë ¨ ��,  íjÌù�� ýª�<�� ø ©�¯�,&¯2¹�£�´|£_¶=º�Q ¨ U�2� ë Ñ�í?��Ì&Ó��� Ð@� ��'�' ��*Y���4./�������1�c�����-����§=�E°i� Ï�� ë Ñú�� ���484�-�:l;���6°����>`(@j��8C�-�&l;����Q�U���2� �}Ì�ÓdÎ�Ë%Ë ¨ �E  íjÌ�4��� ���48C���:l;���/? �����sQ �>a�<��YU�Y��� ë Ñ�í?�}Ì&Ó���2� ë Ë%Ò ë�-��� ���484�-�:l;���6°h���-*-�-�0C��l;����Q�U�Y��� ë Ñ�íXÐ@���ù2� AB<����484�-�:l;����? �4��� O 0���'� ��D�'���@�����82��Q�U�Y��� ��Ì&ÓdÎ�Ë%Ë ¶3���A ísÌ���2� Ð@� ���4.�'��������>W\�-�-' ¬ þ�ÿ ÷ Q�³YU ���&� � ¬F°����t�.�7���-�/n(�-���4.���*-�CQ%¶�U Ï�� ë Ñ��ú2� ���48C���:l;���/? �����YQ �>a:���4������§=�&U���2� �5<���<A����t�.�7������nF�����4./��*��CQ%¶�U�2��� ë Ñ�íXÐ@��4��� ë Ñ�í?�}Ì�Ó���2� ¢C<����48C���:l;���/? ����� O 0���' ��D�'�� O ���-������Q�U����� £D< �4�>��nM���&�/�� ,Q�¢�U�4��� ���-����@s�48C���:l;���/Q_|U��ù2� ��'�'���*Y���4.�������� �=��4��� ë Ñ�íXË|ÌeÌ&ï

5. EXPERIMENTAL RESULTS

� Ð|Õ:ðjÓ ë/E3��F Ì�ÔpïjÎ�Ó�Ð%ÑjÕEÒ-Ï-Ð_Ôgð�Ë%ðjÒ1Î�Ñ�íZÎ�Í���Ð ë ñ ë ígÓ ë2ì Î�Ó�í�Ò

@s�d��K��-*-.����H��+/�J��K�#:����� !�-�����-a:�!��#:�-*-7x&*J��� ).�' �Y�����F`"���M� �<#�'��- !���������; �HGC��0���?F§T�,�-��*+g��K�#��>��� 6�����(`��9+���0C�)���4���6�����82������f*+&���/�4�G��+/�G�2.� 9D:���)���1���4�������-?~@3+��G����`3������6�����;*��4 �<#�.������!�Y0C���M�����4�J���4./�&���-a2`�+/�����M��d����*+d����.����6�����1��'�' ��*-���� �4����#:�����|���� !����?X@3+��d����`3���������+���`��v��m��+��d�����#�+/ *��!�����d��+����0C������4�H�Y04���M���9��./���"������+��1��� )./']�Y����4�s?O x/���=�E��K�#:����� !�-���G`3���E#:�����%���� !���w���fx&���w��+/�p�=��� 9.�'�.��0���'�.��~QSI[t2.�������4�hù4Ud��+����E 6��K�� ! N��-�d��+��Z����`3�����`�+/�-����+���2.� 9D:���1���"���4���2���B*+&�����4���H#����4#:��������4����'�'75Z���G��+/�)��./ )D:�>�1��������82�-?"§T�;��+���(*�������a:��+/�����H��F���-7��+/���M�!0��Y��]�Y����4�g��E��+/�H*���#���<D��'�7����-�B����t�./����-�g�|���J���-*+Z�����8g����5,�����8E���-' �������ZD�5g��� O � �*-�����=����������?����4.����,�p��+/��`��!��+��E��*+���-0C�-�v���>`"�Y��v�%���d�/7�����������!0���'�.��-����3�=�� 9.�'�./�H���&�p�/7�������-���B�2.� )D:�>�1�������4�-�����-?�W\��.����!������>��<�-���(t2.�������7����-�����[���4�-�����"������K�#:����� !�-�����/7�����������(#����4#:��������4�/����-' �����-�Z���d��+��9�2.� )D:�>�M���������8���Q=���4�d 6�-���/�H�4�¡������+/�!�����4������82�-a������G !�����/�B�4�4�6aV�����4�G !�����/�H���C�da[�Y���4�G !�����/�H���4�da�����4�� !�����/�1���4�4�da:���4���! !�-�����1�����C�c�1�C���4�� !�����/�(���4�4��U>?O �(`3�9*-���g���-�4a:`�+/�-�,��+��H�2.� 9D:���(�������4�-�����M��M��t�.&��'j���6��������-�������)��+&���X��+��d�2./ )D:�>�J���(�����8��-aA��+/�d�=��� 9.�'�.��H��+&�Y�) 6��K2<� ! N��-�G��+/�g����`3�������E��Þ �C�oQ__? ��?ü�|���g���4����aB�4���4��aB�����o�C���4������-�����U>? � ��`3��0C����a"`�+��-�q��+/�,�2.� 9D:���6���J�����-�����;���-*����-�����-�-a��+��J�=��� 9.�'�.��(����']�Y�����,���6��+/�JD:�-�=�M����`3����g���*>�����������-?H@3+���1��K2<#:����� !�-���[��+���`��[��+&�Y��aC���B 6��K�� !�N-�3��+/�"����`3����/�-a���+/���=��� 9.�'�.�� )./�=��0����� �����J��*�*-����� �/�!��+/�B#/���4#:������ ���,D:���}`3���-�E��+/�B�2.� 9D:������A�����-�����F���&�d��+��1�2./ )D:���3�4�"�����82�-?����4.����1�)��'����)��+���`��3��+��F*-./��04�M���j��+���3��t�.&�Y����4�sa�' ��D:��' '����E³4a��+&�Y�A 6��K�� ! N��-�j��+������>`"������A�|���[����*+)�2.� 9D:�����������4�-�����-? � �>���`3�6���E�/���J./���! !�����)��+����m���4�4�,���4�-�����HD���*���./���6��p��+���H*��������+��1�=��� )./'�.��"��(*��4���=�������?W\�Jx��M��+��)I[t�.&��������\ú6��+����1*���#���./�����1��+/�J���-' ������4�/��+��#\D:��<�T`��-�-�X��+��6D����=�)�=�� 9.�'�./�90���' ./�-�)�����X��+/�d�2./ )D:���H���M���4�-�����-?����4./���G�,��'����Z��+���`��9��+/�6*-.���0C�6���(��+/��9��t�.&�Y�� ���saA' ��D:�-'�'����m�-?O �[ !�-��<e�������������db2�-*�������6��aC`3�(�4�/��#/�[��+���[�- !#/����*���'�'�5)D�.��' ���t�.&�Y����4�G����*-�� !#�./���M�=��� )./'�.��"��,b2`3���� �<TL O P�?

³(° � ø · I'JLK MN ç�OQP9R�S N ÷ SUTáJ� N ÷WVYX ZU[ á

JN ç�O]\_^a`�S;ba^aT;c��� QSúCU

§}�d��+��M���-*-���&�G�>K/#:����� !�-����a�`��B��0���'�.&�Y���Jb�`"�Y�� �<=L O Pf*��4 �<#&�Y������J7���V���-��./'����V`�7��+6��+/�����-��./'����3��*+/ ��0C���!D�5!�B�����-�-��5�a�*-�-��<�����'��N-���X��'��4�������+/ E?G§}�p��+���H��K�#:����� !�-���1`3�!*>+����/�4����+��)�����8

53

� Ð|Õ:ðjÓ ë2de��fJë Ò-ÏBÒ-Ï�Ð%Ôgð�Ë%ð�Ò �hg!êûë Î&Í��\Ñeð$Ô�� ë ÓHÌ��VÎ/Õ ë Ñ�Ï�Ò

� Ð|Õ:ðjÓ ë(i3�jF Ì�Ôpï�Î/Ó�Ð|Ò-Ì:Ñ¡Ì��EÓ ë�ì Î�Ó�í�ÒvÌ���+ ì Î/Ó4Ô�8akGÊmlÎ&ÑjíqÎpÕ&Ó ë2ë íjîvÒ-Ï-ÓYÎ�Ï ë Õ&î

����t�.�7���- !�������"D/./�"��+/�����B������ O � � *-�4�/�=�����������-?@3+/�E�����-����5v��'��4�������+/ ��'�'���*��������d��+/�;D:�-�=�6t2.���'�7x&���v���4���2�Q__? �4?G��0����' ��D/'��d���&�Z+&��02����;�-�/�4.���+p���-���4.���*-�-�UM���E����*+m��0����'7<��D/'��d�����8:? O �9��K�#:�-*�������aA��+/�d�����-����5m��#/#/���C��*+v��./��#:�����|���� !�b2`3���� �<TL O P�? � �Y`��-04����aBb�`"�Y�� �<=L O Pü#:�����|���� !�d`��-'�'B��o��+��I$<=L O Po��*-�-��������,��*+���-02����d����`3����/�J���4�R����'75g'���`3���6Q_�4�p��0�<������4�YU3��+����G��+/�1�����-���25G�4�/�-�-?@3+/�d' ���=���>K/#:�>�� !�����J !������./���YbZ��+/�6� !#&��*��9���(*-�� !#�./����/���+��H����`"�Y��/�-aj@3+/ �M��� !�J*-�4�/��]����������d��+��9���&�,*-�4�/�=�����������BD:�><�T`��-���;�����82�-?[§T�d��+������K�#:����� !�-����a/ù4�4�c���$��+/�1�����82�F�Y��� O � �*-�����=�������/���;��E�����4.�#/�����[�2?����4.�����ùM��+���`��A��+/�3��K�#:�-*������9�/�-*-�-59��6b�`"�Y�� �<=L O Pg#:�����%����< 6���/*-�)`�+����\`��6*-������ �/�>�J��+�� O � � *��4���=�����������9���&�p���-04�����'0���'�.����M�%������QSI[t�.&�Y�� ���púCU>?J@3+/�9����`3������B�����Y`i`�+��-��� é°u��a� !#/���Y02����J��+��B��0C������4�H#:�>���%���� 6����*-�1 �,�4���d?

6. RELATED WORKb�`"�Y�� ¡D&�������,��#�#����C��*+/�-�F�|���M�4#��� !�N��Y����4�g#����4D/'��- !�(+&��0C�D:�-�-�J#/�������-�������HD��>�%�����3��J��+/��'�7���������.����4?A@3+�� O �2�A����'��4��5)�M#/<��� !�N�������4�6��'��4�������+/ !�M� ùY�e�Y���(��.�*-*��-���=�%./':*-�-�������' �N-�-�6����' .�����4����%���B���-04�����'$#����4D�'��- !�F���g��+/ �M�������g� ��aA�Y����? � ��`3��0C����a:��+������J���!' ��*8;���[�=��.��/��-�M��D:��./�M�/��=�����D�./���-�E0C������������F���[�4#��� !�N��Y����4�#/����D�'��- !�-?d�/��`ü�/��=�����D/./�����\��#�#/���4��*+���� O nFI�nFI[P��Mn[@"I � a 6���/'756�%��*-./�����;�4�;��� !#�'��1#/���4D/'��- !�-?O �� �=�����D/./����� ��#/#/���4��*+ �%���� 6���2./�_��*���./����/� �25��&�� !�*��*+�����.�'�����pD&�������v�4�~����*- ��'F�������*����6 !�2���-'3`"���6#/����#:�4�����v��

� Ð|Õ:ðjÓ ëon3�Hê!ë2ì Î/Ó�íjÒ���Ì&ÓGíjÐqp ë Ó ë Ñ�Ï�ñ�Î&Ë%ð ë Ò)Ì�� �

� ���e���&�!�>K������&�/�-��D�5E� ���S?[§}�!��+/���*������4a2`3����#d'��8C�M���4�-�����V���-#�����<���-���� )./'7���<�#/./��#:�����M 6��*+���/�-�"*���#&��D�'��B���$#�����*-�-��������!�/7������������=��D��-?g@3+������6��)�E*-�4�=�J���,������.�#\��+/�6 6��*+/ �/���|���� ��4�/�!�T5�#:����"�=�4Dw���f��������+/����?u�F��`��=�4D/�;�������0C�g����X#/���2��.�*�����4�w'� �/�4?@3+��9 6��*>+/����-�F��+���.�' �,*>+/���4���9`�+/����+��>�1���1�����M���d#����2*��-���M��+���=��Ds?M@3+��-7�M����������1��F���6 !���� !�N-�B��+/�J������./#E�� !�H��g����/���F��� !���� !�N-�F��+��M�������'s#/����*-�-������/���� !��?@s�B��*+���-04�3��+���A !���� !�N��Y�� ���sa4�-��*+����&��+/�3 6��*+�������A��+/�4./']���#:�-*- ��'��N-�,��~#:�����|���� !����Z�4���;���d�g�|��`û�T5�#:�-�)���V�=�4D/�-?q@3+�����#:�-*- ��'��N�������4�~ ����*+���-04���mD�5m�g !�-*+&������� ������#�7�����mD�5\��+���4�/��#��� 04�9�����8g��'�'��2*-������C�\D:�-+&��02����B���3�=`3���� !�-?)@3+��-7�B���-��./'7�����+���`w��+&�Y����+��F����*- ��'e �/���-*��" !�2���-'&��3*-�4 !#:����7���0C�4a����"��d���4 !�*������-����.�#:�>�� ���"���9#/����0����.���'756��.�*-*-�����=�%.�'s���4�-���"D&�����-�d�=5��=���- !�-?@3+/�~b2`3���� �<TL O P¡./���-�ZD������*���'�'75i��+/�m���� !�m#����4D&��D��'���=�� *�=`"�Y�� �����8q��'�' ��*��Y�� �4�� !�2�/��'F./�����q �i� ��a(���S? � ��`��-0C����a� �b2`3���� �<TL O Pu��+/�,���4���2���d�����g��#:�-*-�'� N����wD�5~�/�>�_��./'��dD:�-*-��.������+��H��/�|���� 6��������Z��D:�4.��F��+��J�����-�����M*-��#&��D��'�7�� ���1 �F#&�Y���M���A��+��#/����D�'��- ^�/�>x&�/�������s?9@3+/����.���+Z��+���1*���#���D��'�7����-�B`3�)��#:�-*- ��'��N-���+��������-�����H.�������G��+/��#:��'�5� !����#/+���� � !�2�/��'A��+���`��\��XI[t�.&��<����4�,ù2?�/./����+/���� !�����4a[`��;#/���4#:�����;���p !�2����]5m��+/�G�����&�/����*�5X��t�.&��<����4�X���,�/����'V`�7��+X��+/�6�������������']�Y����4����+/�#��)D:�>�T`��-�-�X��+/�������82�-a���F��+/��`��g��,IVt�.&�Y����4�gù2?�§}�X� ��a:������+/�B�����8��F�����B�������'75; ���/��<#:�-�&���-����?@3+/�1`�����86��+�������������2�/.�*��-�"��+/�BIA<TL O P�Q��7�Y���%U"��'����!#����-���-��������;��#�#�����K/� 6�Y�� ���;��'��4����7��+/ û���)����' 04�B����=�����*-���(������+/ �"#����4D/<'��- �*-��'�'���� k ��`"<�*-�� ! ).��/�*�������4� O #�#�����K/� 6�Y�� ��� � �"�1PwQ k O <� �"�1PVU>? O �oIA<TL O Pü*-���oD:�g !�2���-'�'����o���;� � �"�1P�aV`�+��*+ !�����/�J��+�������'��4����7��+� !�9���g���4'�0C� � �"�1P[�)*-���fD:�G��#/#�'�����X������4'�0C�g��+��ZIA<TL O P�? � �"�1Pû��'�������7��+� !�E�Y���Z��+/�Z�=��Y����<����]<���+/��<��������G�����8G��'�' ��*��Y�� �4�E.����/���3��+/�1 9.�'7�� �����-����*-�� ! )./��7�}5!#:����<��#:�-*����0C��?�b��-04�����'s��'�������7��+� !�"`�7��+,�/7�����������F��#�#/���4��*+�����`3�>������-*-������'�5\#����4#:�4���-�wQ_�4? �/? O � �1P[@1�7�-�/a"���Y�Sa��M#/� O P��)�7�����"���&�� PV�1P(�7���Y�|U>?Mb2��.��/��-�����-�4����/��/�6��+/��M��'�������7��+� !�F#:�����%���� 6���/*-�.����/���J*-�� 6#/'���K\��*-���&�Y�� ������+/��`ü��+����J���,��������*+m�|���9��+��6��#/��7< 6��'[����'�./����4�m��\��+/��J82��&�Z����#����4D/' �� ô��J�>K/#:�������0C����\������ !����A*-�4 ! 9.��/�*���������,���&�;*-�� !#�./��Y����4�&��'e�� !�����-��a���aj��a��Y�S?"§T��<�=����/*-�-�A���&��+��"I$<=L O PZ !�2�/�-'�'����)��� � �"�1Pg���-��./'7�[ �)#����4D�'��- !������ 6�����*���'�'75G !�����1*-�� !#�'���K6��+/�-�G��+/�B�����-��*-7�����;��D:�Y0C�4?@s�J�/����'�`�7��+!']�Y���4�F��*-��'��(������ !�[���j���4�-�����V���4'�02����JI$<=L O P\������*-�-��������59���B !���� !�N-����+��"*-�� ! )./���*��Y�� ���!�� !�4�/�M��+��������-��������1 9.�*+g���1#:�4�����D�'��4? r ����]���-�-as��g��+���M82 ���,���V#/����D�'��- Ea�7�1��D:���������J���g���������f��#/#/����K�� 6�������\����' .�����4�v���J�_���=�!���9#:�4�����D�'����+&���G���)x��&�6��+��1��#/��� 6��'e��E���;.����%������ D/'��B��� 6��?k O < � �"�1Po��H� � �"�1P���'�������7��+� ��/�-0C��' ��#:���g���;������'$`�7��+

54

��+���[IA<TL O PZ��#:�-*- ��'�*+&�Y���*���������=���*-�-?VI$<=L O Pp��[���4'�0C���)D�5 k O <� �"�1PX��d���d��#�#�����K/� 6�Y�����)�_����+���4�e?V@3+��F��'��4����7��+� c�25/���� !7<*���' '75)*-�4 !#/./���-�A�M ! �/� 6��'2*���#���D��'�7�T59��+/������+��4' �9�|���V����*+������8:aD&�������9�4�9��+��3#����4D�'��- i��#���*-7x&*���������'��(QS��'�'����4�-�����A*���#���D/ '�7����-�-a�����8d���-t2./7���- !�-�����-a���0����']��D�'��H���-���4.���*-�-�-a:�>��*YU>a:�����;.����1��+������� 6��K�� !�N-�F��+��1�>K/#:��*������d�������'e���>`"�Y��e? O �4�������(�/��*- �/�M���!��'�'���<*��Y���J�J�����8d7�$�����*���#&��D/�' 7�}5;��(�������������+��-�d��+���"��+/������+���']��?k O < � �"�1Po.����-�H�G����8C�-�pD��������p#���������*-��'[���;� !#/���Y0C�9*-�4 �< )./���*��Y����4��#:�����|���� 6���/*-�4? O �!�����-���[���-*-�-�0C�����M���484�-�sa����-*- �/���`�+��*+)�����82�A���1��K��-*-.����"./�� �/�1��+/�3��+/������+��4' �)*-�� ! )./���*��Y�����J����+��"���48C���6 !�-�������4�4a������!�����&����+/�����484�-�����9��������+����V����&���4 !'75*+������-�6�����-����? O ���������4����'����48C�����-aC*���'�'�����r'sut!vw)tyx{z7|�t}s�~�vw9��a/�����.����-�ED�5d��+��J���4�-�����(���! 6��84�B*-�� ! !��� ��-�����3���-�4����� �/�!��+��J��'7<'���*�������4�p���3��� O � � *��4���=�����������,�����82�-?H@3+��)��.���+������M��+���`����+&�Y� k O < � �"�1P��4.���#:�����%���� !�H����+/���9��#/#/����K�� 6����� � �"�1P���'7<�4����7��+� û �;*-�� ! )./���*��Y����4�;���&�d�������'e���>`"����Et�.&��'���}5�?b�`"�Y�� �<=L O Pv�/7�������"�|���� k O < � �"�1Pf��E����0C�����'e`3�-5��-�� k O < � �"�1PX*-�� !#�./�����V��+/�(��+����-��+���']�!.������!D�5���+/�M�����-��������m�/�-*- ���,��+�����G��*>�� �����GD��������q�4�w��'��4D&��'1��/�|���� 6�Y�� ���sa���*-'�.��/����g��'�'"�����-�����)*-��#&��D��'�7�� ���-?m�M.��)��'��4�������+/ .����-��4�/'�5!'���*���'� ���%���� d�Y����4�s?[I[��*+;���4�-�������-*- �/�-��`�+/�*>+6�����8���M��'�'���*������"D��������J����'75H�4�97����*���#���D/ '�7�T59���&�B��+��V�2.� 9D:������������82�(�����;���4�������(��G��+��1�=5��=���- E?� @3+�� k O < � �"�1Pf��+/���-��+/�4' �E��F��'��4D&��'_a:`�+/ *+, !�������"��+&�Y��4�/�o���4�-�����m*-�4 !#/./���-�p��+��~��+����-��+���' �eaG���f*-�� ! !�-���������D:�Y0C��a����&�1*-�4 ! 9.��/�*��������s7�j���"��+/�[����+/�����-?$@3+���b2`3���� �<L O Ph��+/������+���']�X��)�������������'V���g����*>+f�����-���!���&�\���>{&�-*������+��B�����-����y �(*���#���D/ '�7����-�-?� O ���-�����$���-*-�����4�9��)b2`3���� �<TL O PE���#/����D&��D/�' ��=���*"��*�*-����2<����,���,��+/�d�=`3���� Ey �) !�2�/��'����F�/�02�����4�f���(']��D:����?\IA0C�-�`�+����E��+��H#����4D���D/ '�7�T5;���[�>K/��*-./����/�6�������8;��F' �������4a�7�[7���������������4a[��+&���~���~���4�����!*����~���-*- �/�G�/���)���Z��K��-*-./���7�EQ%`�7��+f�� 6��'�'3#����4D���D��'�7�T5�U>?p§}� k O < � �"�1Pjaj��+��d���4���2���'7`3��5��M���-'��-*����F��+��B�����8;`�+/�-�,����F*���#&��D/�' 7�}5,��M���I O �������+&���G��+��M��+����-��+���']��?� b2`3���� �<TL O P¡.������\�w�� !#/'��m����8C�-�i#/��������*-�4')������' '���`�����-�����,���q�=5/�/*+/���4�/�N-�4?ô@3+/����.���+���+/��g#���������*-��'9����*+�����-���(82����`��3��+��1�Y0��� ' ��D�'��B�����8��������G*-�� ! )./���*��Y���-�37�����*���������H���;��+/������+��>���-? k O < � �"�1P���'����,.������H����8C�-�/�B���� !#/���Y0C�)*-�4 ! 9.��/�*���������\� �*-��-��*>5�a�D�.��H7���J#���������*-�4'V�� !�����J���4#/+���=���*���������?9@3+/�9����8C�-�g !�-�������4�-�1*���������-�1*-�4 �< !7�� !�-�����,��/�|���� 6������4�����&�h��+��f*��4 !#�.���������+����-��+���']��a`�+��*+E���.������GD�56��+/�H���4�-�����"���!�/��*- �/�1��+/�-7�F��*>�� �����-?� k O < � �"�1Pm������'���`�7��+ O � � *��4���=����� �/���!�����82�V��+/���4./�4+*-�4 ! 9.��/�*���������sas���*>������������;��+/���2.� 9D:���1���3���484�-�p !�-�=<�������-�ZQWr'sut!vw)tyx{z7|�t}s�~�vw9�-U>?hb2`3���� �<TL O Pu./���-�d�Z�� !#/'�� !�2�/�-'���+&���V !����7x&�-�A��+��(�����-�����-y����-���/�-�/*�5���*-*-���������H�����+�� O � � *-�4�/�=���������-�G�����82�(��'�'��2*-������C�e?�M�/�G*����~D����C�4��'75v*��4 !#&�Y���G��+/�G���-��./'7���6��*+���-04�E+/�����G`�7��+��+���������*+���-04���gD�5 k O < � �"�1P����&� � b O QS��#/#/����K�� 6�����9��'��4��<��7��+� i�%��� � �"�1P�U��7�Y���Sa2�� �/*-�"��+/�"��K�#:����� !�-������`3������#:�����|���� !���`�7��+f#&�Y��� !�����>���)���)*�'��4���;���-?\�����./���;�E��+���`��9��+&�Y�)�4.�����#�<#/���4��*+w��d�-t2./�04��'��-���6���\��+/� � b O aV��*>+/��-02 �/�X#/������*���'�'�5~��+������ !�B����`3����/�-?M@3+�� k O < � �"�1Pq�=�������=���*���'�5g��./��#:�����|���� !�(��+��b2`3���� �<TL O P�?s@3+/�������-�-��5Z !����+/�2�g !#/'��- !�-�������,D:����+pD�5g.����������+��(�4�/�F./���������J�-0���' .������(��+�� k O < � �"�1PX�/ �!�/���"��*>+/��-0C���+��B���� !�1���-��./'����-?�@�+���F*����ED:�H��K�#/']��������;D�5G.���������-���/���G�/7�|<�%�>���-��*-���( �G��+/�B� !#/' �� !�-���������4�/�-?

7. CONCLUSIONS AND FUTURE WORK@3+/�E��#�#����C��*>+~��������2�/./*-���f+/�����E�/�-��'���`�7��+v��+��;IAK2���-���/���LM�-�/�����'��N-��� O '�' ��*-������4�GP$���4D/' �� RQSI$<TL O PVUAD����������4����+/�"��+���<�����>�� *-��'2 !�2���-'��j�����/�02 ������J���/' ��D:���$��H�=`3���� !�-?$@3+/�V#����-���-���������'��4����7��+� Ea/*���' '����Eb2`3���� �<TL O Pja�����' 04�-�(I$<TL O Pf��E���E��#�#/����K2<� 6�������;���&�;�/��=�����D�.������d�_����+���4�e?3@3+/�HIA<TL O Pv+&����D����-�;.��������,��+��H' 7������Y��./���J���6 !�2�/��'j��*-�-��������4�1���V���-����*+p�����E������*-.��J ��- !�������-��*>5q��7��.�������4�/�-a(`�+������Z��*��������G 9.��=�6*-���������&�����,��+��-7���*>�� ����������+/�-'�#;��,�� �����=�����F 6���������- !�-����?b�`"�Y�� �<=L O P�������-�&���6���fD:�Z�X��� !#�'��Z�����w�����-*>�� 04�p��'��4��<��7��+� E?$@3+��F�>K/#:����� !�-�����'�������.�'7���3��+/��`w��+&�Y����+��(#����4D&��D��'���=�� *�/��*-��� ���sa/D&�����-�G�4�d��+/�F���-�&���-��*�56�����d#:�4'75� !����#�+/��� ü !�2�/�-'��-a��'�'���`�����+/�d���4�������9���g 6��8C�����������4����D/'��;*-������/��&�Y�����f��*�����4�/�-?@3+��6b2`3���� �<TL O Po#��>���%���� !�1`��-'�'SaA��*+���-02����;���>`"������-a$���X��02<������4�4a:�4�/'�5,���C�c`��������H��+&���E��+/�H�����-�F����*+���-04���ED�5;�������-�-��5*-�-�������' �N����h��#/#/���4��*+s?û@3+��p��K��-*�./����4��*-��� !*-��-���,� !#����Y0C�-���+��F��0C������4�F����`3����d��G�4�4����dIA<TL O PX����=����/*-�-��`�7��+d���-0C�>���'������������-' �����-�;�����8��-?§}�\��+����%.���./����a$`��6��2�����&�Z���,*-�4 !#���������+��!#:�����|���� 6���/*-�6���b2`3���� �<TL O Pq`�7��+ � b O ����� k O < � �"�1Po��p�6`3�-5g��+����M��.��/���'�'j��'��4����7��+� !�(��G��+��1���� !�1��� )./']�Y����4�E���20�7���4�/ !�-���3`�7��+E��K2<��*>��'�5!��+/�F���� !�F������./#s?A@3+���3 6�-5!*��4�/x/�� ���+��F#����-���-�������-��.�'7���-a��*+���-04���BD�51�3���4.���+H*-�4 !#���������4�e? r �-�� ���-�-a�`3�V������-���M������2������<�/./*-���_���'�./�����V �9��+��3�� 9.�' �Y�� ���)���-�4����/��/�1��+��3*-�4 ! 9.��/�*�������4�*+&���/����'Sa:���4�-�����(�����8;#:����*��-#/����4�ea:���&�;���*-�� !#�'������B8��/��`�'����/�����D:��./�"����+��>���"*���#���D/�' 7����-�-?�@3+���3�_�� '�.����-�(*��4�������D�.����F���)�J������'7<��=���*!���&��' ��=5��B���3��'�'A��'��4�������+/ !�-?J@3+/�9]�����6 �F���d�-0���'�.&�Y���)��./������.�7����4�X��+����J�=`"���� �'��8C�d��'�������7��+� !�9�����d��D�'��!���g�/�-��'V`�7��+�_�� '�.����-�F�������'�5G`�7��+E��� !#�'��1 !�-*+&������� !�-?

8. REFERENCES�7���Jb�? O ' _a&b�?)�B���-�/ ��a&���&�Elp?�@��� )D:�4?�P$���-#����2*��-�����������-*+��/]t�./�-�3�%���F��*-*-��' �>���������!��+/�B�/*-��#E��'��4����7��+/ ¡�4�/��#/��?§T���h��s��avLvL�ux@w_�u��s;��t@�9v5��su�_�t@�m�aw9t!v�aw�z�tyx{suw�zu|D�_suxWw)t-Qsuw���v�Lvw��av�s�w�0��'t}s�w�su��su�_��0Q��vw9ty��zuw������)| tyx�0]��vw9t�9���Lt!v����a&#&���4�-�B��������&�����C��a&�M��`��V����8:a/���)a'�1b O a/�����C�2?O �"lôP$���-���-?

� �Y�HI�? r �4����D:����.sa:L9?&@3+����.�' ��N4a:�����Elp? � ���� ���/?e�9�]zu�a��aw9t!v |W| xY��vw��av����3��su� �=zuty�_��z7|et}s�0��tyx ¡]� x{z7|3�)�u�Lt!v����?�FK��|����m�F���0dP$���-���-aj�-ú4ú4ú2?� ���Jlp?:�3�� !#:���-a&I"? r ���&��D:����.sa:L9?&@3+����./']��N4a:���&�G/? � �-�/�-./D���./����? � 5��&�� ! *M��*+/���/./' ��/�!�����;�/�02��� ���E���' ��D:���F��;����*- ��'j������-*>���-?&§}�H0��7zr�tyxW¢uv�£5vL�)z�¢�x{su�>a�0C��' ./ !�������a&#����4���F�4���2ú4ù2ae��������?� ���=¤9?&���*-7���-'�'��d���&�Eb:?&b� !7��+s?2§T !#����Y0C���!���4.�������)`3����#/���%����/��=�����D�.������d�_��*�������5;*��4�������4'_?��_su�_�w�z7|Ds;��0��_t}suw�su��su�_�

0]��vw9ty��z�w����H�'| tyxy¥y0Q��vw9t¦�9���Lt!v����a:�/QS�CU>� ���4������ù4ù�aelE�-5���4���/?� �Y�§G/? � ��02��E�����;P�?�G�?:l;�2�/_?�§T !#&��*��"���$#/����D�'��- *-�-�������' �N��Y����4�g��E�/��=�����D�.������G*-�����=����� ���(�4#��� !�N��Y����4���'��4�������+/ !�-?�§T���h��s��avLvL�ux@w_�u��s;��t@�9v5��su�_�t@�m�aw9t!v�aw�zutyxysuw�zu|

�_suxWw)t5-Qsuw���v�Lvw��av�s�w�0��'t}s�w�su��su�_�50]��vw9ty��zuw���H�'| tyxq0]��vw)t¦�9���Lt!v����a&#&�����-�H�-�C�4���&���4ù4ù�a:�F��`¨�V����8:a/���)a�Bb O a������C��? O �"l�P$���-���-?

� ù��Jlp? � ���� ���/?¦©�r�tyxW��xYª�z�tyx{suw9«�¬Dv�zu�w)xWw��­z�w����§zuty�_�;z7|0�| ��su�x@t@�_����?&P[+ � ��+��-����-a&P���'������*-���*-�d��jl;�' ������aj��ú4ú4�2?� �Y�Jlp? � ���� ���/a�L9? � j�3������a:����� k ?&L1�� 9D&�Y��/�-'�' ��? O �2�*-�4'��4��5G�4#���� ! N-������4�e� O ����`� !�>���<�+��-.�����=�� *�?/§T�;P�?�G/?O �/�4�-'�����4a�®$?&l;�*+&��'���`��*-N4aelp?&b�*+����-����.��>��a9¯J?'������a&�����O ?�®���'�N���']�2a:�-�/7�������-a��h�;s��vLv��uxWw_�u��s;��t@�9v/-Qsuw��u�Lv�L�°s�w± ¢�su|²�_tyx{s�w�zu�a�H-Qsu��r��'t{zutyx{suw/a&0C�4'�./ !�H��a�#&���4�-��-���������-���C�2a�lE�-5�{���`3��� � ������'Sa�WX����+���/�����4� � ? �F?�a9�Bb O a

55

ù���ú;��ú�ú4ú�?&§TIVI[IqP$���-���-?� ���HP�?&�/�����������³G���?&����� O ? r ��N-N����s? � ��=�����D/./�����G !������������*+�����.�'��1��+����4.���+;*-���4#:����Y���0C�H !���� ������4�s�[���&��'75��������4#����#:�/y �(#:�����|���� 6���/*-�B��E�)*-�� !#�'���Kd��*-�-��������/?&§}�O ?&l;�� ����'���a&���/7������a��h�;s��avLvL��xWw_�u��s;��t@�9v§�9x�´ut@�

�aw9t!v�aw�z�tyx{suw�zu|5µ­su�L~��;�)sar¶suw�·�xW�Lty�xy¸ �'t!v���-Qs�w9�Lty��zux@w9t¹ v�zu�suw)xWw_�HºW·³- ¹�»'¼7¼7½ ¾ ¥]�5xWw�v t}vavw)t@�­�aw9t!v�aw�zutyxysuw�z7|-Qsuw���v�Lvw��av°suw�0��tyx ¡]� x{z7|��aw)t!v |W| x���vw��av³ºW���-�0�� »_¼�¼7½ ¾ a#&���4�-�H�-���a�&�4����a�G4.�'75E�����C��?

� ú�� � ?&LM����/�4�e?&@3+��B�����C�����N�������4�,����`�����8G��;����*- ��'j������-*>�*-�4'�������-�-?3�§z�ty�'��v>a����4�2� ���2��&�Y����aj��ú�ú4ù�?�7���Y�HnJ?/@M?:lE��+/�-�=`"�Y����sa�lp?&@j�� 9D:�4a�I"? r ��`"�������a¿G�?&P�?P����Y��*-�4a:���&�;P�?)¤V�����8�������+&�� E?/@���82 �/� � �"�1Pv���9��+��������'e`�����' ����IÀ!*-��-���(*-�4 !#/'������1���4'�./����4�/���%���F�/��=�����D�./���-� )./'7���<��-04�-���"��*+/���/./'� �/�/?�§}�ÂÁe�_x@�����aw9t!v�aw�z�tyx{suw�zu|D�_suxWw)t

-Qsuw���v�Lvw��av°suw�0��'t}s�w�su��su�_�50]��vw9ty��zuw����H�'|²tWx{z���vw9t�9���Lt!v����a&0C�4'�.� !�9��a�#&���4�-�F�2���������Y�2a&WX����+���������4�ea � �Fa�Bb O a)G4./'75E���4���/?&§=I[I[Iw���4 !#/./�����(b���*-����}5�?

�7�4�>�HnJ?�lE���'�' �>�1���&�m¤J? k ����������?�b2�4'�02����!����=���� D/./�����*-�4�/�=���������(�4#/��� !�N��Y�� ���E#/����D�'��- !�".�����/��*-���4#:����Y���0C� !����]�Y�� ���s?/§}������s��vLv��uxWw��u�°s;��t@�9v��aw)t!v�w�zutyx{suw�z7|D�_sux@w9t-Qsuw���v�Lvw��av°suw�0��'t}s�w�su��su�_�50]��vw9ty��zuw����H�'|²tWx{z���vw9t�9���Lt!v���L«¦Ã_Ä a/#����4���(�C�����������2a:�M�>`¨�V����8:a&��������?:�F��`�V����8:a/§TIVI[Iw���� !#�./���>�Fb���*-����}5�?

�7�Y��� � ?&l;����82'��4a:lp?:l; �/�/�-���/�����}a&����� � ?&b2*>+/ !�-*8:? O ���*-�4'�����5;��#/��� !�N�������4�G�%�������-���4.���*-��<�*-�4�/�=���������-�E#/���Y�=�-*����*+�����.�'������?¿� ±]±Q± Á9��zuw)�z�� tyx{suw)�°suw ± ¢�s7| �_tyx{suw�zu�a�-Qsu��r��'t{zutyx{suw/a:ù/Q_��U>� ���4���&�-�Cù�ae���4�C��?�7���Y�JP�?�G�?:l;�2�/_a/Wi?&b�+����sa�lp?&@j�� )D:��a����&�Elp?'�V�48C����?O ���4#���� O �=5���*+����4����.�������=���� D/./�����d*-�4�/�=����������4#��� !�N��Y����4�;`�7��+,t�.���'�7�T5d�4.&�Y��������-���-?30��atyx ¡¦� x{zu|

�aw9t!v |@| x���vw��av>as��ù2�4���-�Cú��&���4��a¿GC���2.&�Y��5E���4�4�2?�7�-���JP�?�G�?:l;�2�/_a/Wi? <=lp?&b�+/�-�sa�lp?&@j�� 9D:�4a������Elp?)�V�484���/?O �;���=5���*+����4����.���*��4 !#�'������1 !����+/�2�6�%���F�/��=�����D�.������*-�4�/�=���������(�4#/��� !�N��Y�� ���s?�§}�o��vL�Lsuw��2x@w9t!v�aw�zutyx{s�w�z7|7Å�suxWw)t

�Lsuw���v��vw��av�suw�0��_t}suw�s��³su�_��z���vw9ty��zuw��/�°�'| tyx{z���vw9t�L�u�Lt!v�°�a&#&���4�-�H�-ù������ù4��a:�F��`¨�V����8�a����)a'�1b O a����4�4�2?O �"l�PA���-���-?

�7�Y��� O ?�Pj����*-.E���&� r ?&�&��'7�� �/�4�-? O ��*-��' ��D/' �H !����+/���!�%��� )./'7��]���4�-����*-�����=���������(�4#��� !�N��Y����4�s?/§T�H�5xWw�vt!vavw)t@��aw9t!v�aw�z�tyx{suw�zu|¦-Qsuw���v�Lvw��av°suw�0��tyx ¡¦� x{z7|��aw)t!v |W| x���vw��v>aI[�/��2D�.����4+sa�b�*-����']���&��a O .��!�����C��?

�7��ùY�9L9?&I"?�n(�4D/ �����s?/nF�-��.�' ������4�E���[���0������4�E���$']��D:���F��������-*>�F����*-�������-�-?30�w)w9�)z7| ¹ v ¢�x{v�¨sU� ± w9t}su��s7|YsL�u��a�C��� ù��C���2ù4ùC�2a��-ú4úC��?�7�Y���JP�?:b2*-������_a O ?�����������-'�'�_aeb:?&�M8��� !������a/���&�Elp?�@��� )D:�4?O '�'���*�������/�!�����82�(��;��K2�����- !�F������ 6�-?�§T������s��avav��ux@w_�u��s;�

t@�9v]��su�_�at@��xWw)t!v�w�zutyx{suw�z7|�Å�suxWw9t¦�asuw���v�Lvw��v�s�w0��_t}suw�su��su�_��z���vw9ty��zuw��/�°�)| tyxyz���vw9t¦�L���Lt!v����a:#����4����4�����������/a:�M�>`¨�V����8:a/���)a'�Bb O a������C�2? O �"l�P$���-���-?

�7���Y� � ? r ?&b�+� !��5��"���&��Æ�?�@��Y��/�4�-? O �;��#�#�����K� 6�Y�� �����'��4����7��+� û�%���(��+��1�4���������' �N����,��������4�/ !�-���(#/����D�'��- E?�ozut@�)Ä3�h��sL�u��z��/Ä a&ù4��QS�CU>� �4ù���������/a���ú�ú4��?

�7��úY�9L9?&@3+/�����./']��N4a:I"? r ���&��D��-��.sa&���&�­G/? � �-���-./D:�4.����/?nF����#:�4�����M��+/���-��+/�4' �G���� ���%����*-�- !�-���F���&�;�� 02�����4�E���' ��D:�4.��F��;������-*>�F����*-�������-�-?&§T� ¹ su�7z7|3��s�� x}vty�msU��¬�s�w���suw�¿v�ax}v��£ÈÇ�£�x{su|YsL�ux{�azu|3��� x}vw��av��a&0C��'�.� !�B��ùC��a&#����4����C�����2���C�2ae�d�-ú4ú4�2?

56

Agent teamwork and reorganization: exploringself-awareness in dynamic situations

Kathleen KeoghSchool of Information Technology &

Mathematical SciencesThe University of BallaratMt. Helen VIC Australia

k.keogh@ballarat.edu.au

Liz SonenbergDepartment of Information Systems

The University of MelbourneParkville VIC Australia

l.sonenberg@unimelb.edu.au

ABSTRACTWe propose attributes that are needed in sophisticated agentteams capable of working to manage an evolving disaster.Such agent teams need to be dynamically formed and ca-pable of adaptive reorganization as the demands and com-plexity of the situation evolve. The agents need to have self-awareness of their own roles, responsibilities and capabilitiesand be aware of their relationships with others in the team.Each agent is not only empowered to act autonomously to-ward realizing their goals, agents are also able to negotiateto change roles as a situation changes, if reorganization isrequired or perceived to be in the team interest. The hierar-chical ’position’ of an agent and the ’relationships’ betweenagents govern the authority and obligations that an agentadopts. Such sophisticated agents might work in a collabora-tive team with people to self-organize and manage a criticalincident such as a bush-fire. We are planning to implementa team of agents to interface with a bush-fire simulation,working with people in real time, to test our architecture.

KeywordsHuman performance modeling, reorganization, simulation,multi-agent systems

1. INTRODUCTIONComplex and dynamic decision making environments suchas command and control and disaster management requireexpertise and coordination to improve chances for successfuloutcomes. Significant challenges include: high informationload detracting from human performance [11, 18], coordi-nation of information between parties involved needs to bewell organized [22], sharing situation awareness amongst allrelevant parties, and having an efficient adaptive organiza-tional structure than can change to suit the needs presentedby the dynamic situation [8, 11].

Using artificial agents as assistants to facilitate better coor-

dination and information sharing has the potential to sup-port studies of human decision makers and to improve dis-aster management training. Using disaster management do-mains as a ’playground’ for virtual agent teams has the po-tential to provide insight on the design and structures ofagent teams.

Exploiting a disaster simulation requires dynamic and com-plex team decision making task with an appropriate level offidelity [28]. Our collaborators have developed a networkedsimulation program: Network Fire Chief (NFC) [19] thathas been developed and used for training and research ofthe strategic management of a bush fire. NFC provides arealistic simulation of the fire disaster scenario. Using NFCalso provides us with the opportunity to compare the be-havior of our artificial agents with human agents engagedin the same simulation. We can draw on the data availabledescribing how people react to a simulation to inform ourdesign.

In this paper, we present preliminary analysis toward build-ing adaptive BDI agent teams with self-awareness and teamflexibility to enable dynamic reorganization. We will aug-ment NFC with agents that have access to fire, landscapeand resources information appropriate to the role they haveadopted, and appropriate teamwork infrastructure. Theagents will be able to work with humans to manage a sim-ulated bush-fire. In the remainder of this paper, we outlinethe requirements of such agents and team infrastructure andour preliminary architecture for their implementation. Weargue that self awareness in our artificial agents will em-power them to ’thoughtfully’ negotiate appropriate struc-tural reorganization of the team. Disaster Management pro-tocols demand that teams restructure when the complexityof a situation changes [2].

The remainder of this paper is structured as follows. InSection 2 we provide some background on the bush-fire in-cident control system and typical features of the teamworkrequired. In section 3 we provide some background on re-lated work in multi-agent teams and we describe the re-quirements of our sophisticated virtual agents. In Section 4we outline how we plan to integrate virtual assistant agentswith humans to improve the communication, shared situ-ation awareness and coordination between the parties in-volved.

57

2. DOMAIN BACKGROUND: BUSH FIREMANAGEMENT

Typical characteristics of a domain that might benefit fromsophisticated agent teamwork are: too large for any one in-dividual to know everything, necessary for communicationbetween agents (people or artificial agents) to update andshare situation awareness. Each agent needs to be awareof their own responsibilities and work autonomously to per-form tasks toward their goal. Agents need to work togetherin a coordinated and organized way. The nature of the dy-namic and emerging situation requires that teams self orga-nize and possibly re-organize during the life of the team.

The disaster management simulation is a well suited mini-world in which such sophisticated agents might be employed- responding as part of a team (of human and artificialagents) to an emerging disaster. In this disaster scenario,dynamic decision making and actions must be taken un-der extreme time pressure. Previously disaster simulationsystems have been developed and used for studies of agentteamwork and adaptive agent behavior (e.g., [7, 21]).

A persistent problem in disaster management is the coordi-nation of information between agencies and people involved[22]. An essential factor in coordination is to provide essen-tial core information and appropriate sharing of this infor-mation [8]. It is not clear what exact level of shared mentalmodel is required for effective teamwork. It may be thatheuristics are used based on communication between teammembers rather than explicit shared models [15]. Using ar-tificial agents to aid the flow of relevant information betweenhumans involved in the disaster management has been im-plemented using R-CAST agents [13]. These artificial as-sistants were shown to help collect and share relevant infor-mation in a complex command and control environment andto alleviate human stress caused by the pressure of time [13].These agents aided the coordination of information betweenpeople involved. Artificial agent teams themselves can havea team mental state and the behavior of a team is morethan an aggregate of coordinated individual members’ be-havior [25].

Human performance modeling and behavioral studies haveshown that information load can have a negative impact onperformance (e.g. [11, 18]). The skills required to coor-dinate an expert team need to be developed in a realisticand suitably complex simulation environment [28]. Disas-ter management training involves following protocols andpolicies as well as flexible and responsive interpretations ofthese in practise [1]. Using synthetic agents in a realisticsimulation to provide expert feedback and guided practisein training has been shown to be helpful [5, 28].

There are complex protocols available for incident controland coordination. These protocols define levels of com-mand and responsibility for parties and agencies involvedand the flow of communication. The organizational struc-ture changes based on the size and complexity of the in-cident. Simulating and modeling complex command andcontrol coordination has been useful as a tool for investi-gating possible structural changes that can help toward suc-cess. Entin and colleagues have investigated the effect ofhaving an explicit command position: intelligence, surveil-

lance, and reconnaissance (ISR) coordinator to help collab-orative teams in command and control [1]. A collaborativeteam that is capable of reorganizing structurally as well asstrategically during a problem scenario to adapt to a chang-ing situation might perform better than a team with a fixedstructure [9, 12]. Self-awareness and meta knowledge havebeen shown to be required in team simulation studies [27].In diaster management, it has been suggested that one im-portant mechanism needed for coordination is improvisationand anticipatory organization [22]. We speculate that agentsthat are capable of initiative and anticipation in terms oftheir coordination, need to be self-aware and aware of othersin the team to enable anticipatory behavior. Anticipatingconfiguration changes that might be required in the future,during team formation is critical toward reducing the timerequired to reform the team at that future time [17].

Protocols for fire management have been developed to de-fine the actions and responsibilities of personnel at the scene.In Australia, the ICS Incident Control System [4] has beenadopted (based on a similar system used in USA). Duringan incident, the ICS divides incident management into fourmain functions: Control, Planning, Operations and Logis-tics. At the outset of a fire disaster, the first person incharge at the scene takes responsibility for performing allfour functions. As more personnel arrive, and if the situa-tion grows in complexity, some of the functions are delegatedwith a team of people responsible for incident management.It may be that the initial incident controller is reallocatedto a different role if a more experienced incident managerarrives.

In a large incident, separate individuals are responsible foroperations, planning, logistics and control, and the fire areais divided into sectors each with a sector commander. In asmaller incident, the incident controller performs all func-tions, or may delegate operational functions to a operationsofficer.

In the period of a normal bush fire scenario, the manage-ment and control structure may be reorganized according toneed as the size and complexity of the fire changes. In arecent study [18] investigating reasons for unsafe decisionsin managing fires, two factors identified as impacting ondecision-making are of interest to the current work. Thesewere: 1. Shift handover briefings were not detailed enoughand 2. lack of trust in information passed on regarding thefire if there was not a personal relationship between the offi-cers concerned. We will revisit these factors in our plans forscenarios and trials in the current work. It might be possi-ble to recreate such factors in artificial simulations and tosupport the handover of information at the end of a shift byhaving a detailed handover to a new virtual assistant agentpotentially making extra information available to the newshift crew.

3. SOPHISTICATED SELF-AWARE AGENTSThe focus of the current work is to describe attributes neededin a sophisticated collaborative team of artificial agents ca-pable of emergent team formation with flexibility in terms ofthe roles adopted and an ability to reorganize and change/handoverroles during a scenario. We are interested to investigate ifthe BDI agent architecture can be successfully extended to

58

create more sophisticated team agents for a particular do-main. Unlike Teamcore agents [20] we are restricting ourinterest to a situation in which all the agents can be homoge-nous in design and can share access to a common workspace.We are interested to develop self-aware agents with a levelof autonomy allowing them to reorganize during a simu-lated disaster scenario. Following from the work of Fan andcolleagues [13], we are planning experiments to investigatewhether sophisticated BDI team agents can be used as as-sistants to aid relevant information sharing between humanoperators in a complex and dynamic decision making con-text. Unlike Fan, we plan that our assistant agents can takeon more than one role and may change roles during the sce-nario.

We have the added value that our agents will be interactingin a simulation system for which there is data available todescribe realistic human behavior. We can be usefully in-formed by a comparative analysis of artificial agent behaviorwith human agent behavior responding to elements in thesimulation [23].

3.1 Multi-agent Collaborative TeamsMulti-agent systems research has included work on team-work and architectures for collaborative agent teams. Sig-nificant effort is being invested in building hybrid teams ofagents working with people (e.g. [20, 26]) and fully au-tonomous agent teams (e.g., [7]). Heterogeneous agent teamshave been created by using special TEAMCORE agent co-ordinators to act as mediators between team members [20].

Sharing situation awareness of a complex task is a difficultcoordination problem for effective teamwork. Knowing whatinformation to pass to whom and when this can be helpfulis not a simple problem. Yen and colleagues have conductedresearch into the coupling of agent technologies to aid peo-ple in efficient decision making using distributed informa-tion in a dynamic situation [13]. They have successfullyimplemented agent assistants to aid humans share situationawareness in command and control situations.

The R-CAST architecture is based on recognition primeddecision making RPD, making decisions based on similarpast experiences. Each person involved in the commandand control simulation is assisted by one or more RPD-enabled agent. The agents may collaborate together withother agents and with their human partner. The effec-tiveness (quality and timely decision making) of the teamdepend on effective collaboration - sharing of informationproactively and in anticipation of the needs of others. Theartificial agents help by: i.accepting delegation from the hu-man decision maker to inform other agents and collaboratein making a decision; ii.the agent recognizing a situationand prompting their human partner, or iii.based on decisionpoints explicitly provided in a (team) plan followed by anagent. Each artificial agent has access to a domain deci-sion space based on cues (abstractions) of the informationavailable. The agents perform similarity matching and re-finement to choose the most relevant decision space. In theproject described by Fan [13], the artificial agents moni-tor for critical situations and inform human operators whenthese occur. The agents also have access to a shared map ofthe situation and can update icons on individual workspace

maps and on a shared general map if given approval. TheR-CAST agents have been shown to help collect and sharerelevant information in a complex command and controlenvironment and to alleviate human stress caused by thepressure of time [13]. The R-CAST agent team was fixed- each agent was assigned to a human counterpart for theduration of the scenario and each agent was limited to onetype of decision. (If a person was performing more than onefunction, they were supported by more than one R-CASTagent, each operating separately.) One of our focuses isto explore the dynamic nature of the environment and todesign agents that can change their role and adapt as theenvironment changes, as these are important features of ourdisaster management domain.

3.2 Team Reorganization and Autonomous dy-namic role adoption

Research into organizational structures has involved agentteams in simulations to test how and when re-organizationshould occur (See for example: [9, 12]). There has beensome agent research work in building adaptive agent teamsthat are capable of dynamic reorganization [16]. We are in-terested to progress this further by designing BDI agentsthat can negotiate their roles dynamically in an emergingteam. Reorganization has been described as 2 distinct types:structural and state reorganisation [16]. Some progress hasbeen made toward flexible strategic/state reorganization ofteams. Matson and DeLoach have implemented algorithmsfor reallocation of new agents to roles to respond to situ-ational changes (e.g. when agents are lost from a team)however they have not implemented structural reorganiza-tion in their agent team [16]. We are interested to provideour agents with some self-awareness and team awareness toenable the agents to decide upon structural reallocation ofthe roles required to fit the changing situation. It is hopedthat our experimentation will clarify the level of knowledgeand awareness needed to enable such reasoning.

The general Teamcore teamwork agent architecture is de-signed to rely upon team plans that are created at designtime. These team plans define a hierarchy of dependenciesbetween team and individual roles as well as a decompositionof team plans and sub-plans. There is no provision of op-portunity for negotiation between agents to handover/swaproles as the agents themselves are not given a level of selfawareness about the team structure nor team plan. Only theproxy agent is aware of current team plans, the actual do-main agents are given instructions from the proxy agent Inthe current project, we are interested in homogenous agentswho have a level of self awareness of their position and withinthe constraints of delegated authority rights, may be able toautonomously change roles or show initiative by perform-ing an urgent task without ’permission’ or delegation, oranticipate a future need. The ability for an agent to au-tonomously (within limits) take on initiative responsibilitiesor negotiate to handover to, or accept responsibilities cur-rently adopted by another agent are desirable in the emer-gency management domain [2]. Tambe and colleagues havesuccessfully implemented adjustable autonomy in agents toenable agents to transfer control to a human, however it isour interest to investigate agent properties that would en-able artificial agents to negotiate with other artificial agentsto handover roles. It is our goal to develop self-aware agents

59

that can exhibit initiative and reason without the aid of acontrolling or proxy agent manager, to self organize in re-sponse to the dynamic situation.

Collaborative agents require a meta level of additional self-knowledge in the agent to enable agents to negotiate. Agentsneed to know and possibly negotiate around their adoptedroles and what actions they are capable of performing. Anagent role can be defined statically at design time - in termsof goals to be performed or the role might be more flex-ible and negotiated dynamically - to enable more flexibleand adaptive team reorganization at run time. Providingthe infrastructure to enable an agent to be more flexibleand to enable the reorganization of teams requires a moresophisticated agent design than the BDI approach of itselfprovides and more resources. According to the domain andlevel of sophistication and reorganization needed, the de-cision to ’keep it simple’ or to include more complicatedstructures is a trade off between flexibility and extra re-sources and structure required. Agent roles can be definedto scope the sphere of influence an agent might have and toenable agents to balance competing obligations [24].

3.3 Relationship awarenessOrganizations have been described as complex, computa-tional and adaptive systems [6]. Based on a view of orga-nizational structure and emerging change in organizationswith time, Carley and Hill have suggested that relation-ships and connections between agents in a network impacton the behavior in organizations. Relationships and inter-actions are claimed to be important to facilitate access toknowledge. ”Whom individuals interact with defines and isdefined by their position in the social network. Therefore, inorder to understand structural learning, it is particularly im-portant to incorporate a knowledge level approach into ourconceptions of networks within organizations.” P.66 [6] Thiswork may suggest that for teams involving artificial agentsinvolved in a dynamic and emerging organizational struc-ture, it might well be worth investigating the significanceof relationship awareness to enable appropriate interactionsbetween agents. In the disaster management domain, thereis evidence that suggests that relationships between peoplehave an impact on their level of trust in communication(apart from the roles being performed) [18]. It is not in thescope of our research to investigate trust between agents,however it may be interesting to be able to create ’personal’relationship links between agents in addition to positionallinks due to role hierarchies and show the impact of these ina simulation.

3.4 Toward defining the sophisticated agentteam

Bigley and Roberts [2] conducted a study of the IncidentControl System as employed by a fire agency in USA. Theyidentified four basic processes for improving reliability andflexibility in organizational change: Structure Elaborating,Role Switching, Authority Migrating, and System Resetting.Structure elaborating refers to structuring the organizationto suit the situation demands, role switching refers to re-allocated roles and role relationships, authority migratingrefers to a semi-autonomous adoption of roles according tothe expertise and capabilities of the individuals available,

AGENT A1

'personalityIndividual preferences

BeliefsDesiresIntentions

Capabilities

AGENT A3

AGENT A2

Current Library

Plans

Task actions

Roles

Policies

Shared Workspace

Shared Situation Awareness

resourcesrequiredposition matrix

responsibilities,obligatiopns,goalsposition/authorityhierarchy tree

AllocatedRoles

Values

relationship links

Goals

Figure 1: The proposed BDI agent team architec-ture

system resetting refers to the situation when a solution doesnot seem to be working and a decision is made to start witha new organizational structure. These four processes caninform the agent team architecture. The agent teams struc-ture will be established so that common team knowledge isavailable and where appropriate, sub-teams are formed [24].

A proposed agent team architecture (based on BDI archi-tecture) is as follows: Dynamically allocate tasks (respon-sibilities (obligations), actions, goals) to a particular ’role’.Allow agents dynamically adopt, refuse, give up, change andswap roles. Maintain a central dynamic role library, accessi-ble to all agents in which roles are defined. Figure 1 showsthis architecture.

Agents require a level of self awareness: know their own ca-pabilities, know their current ’position’ in the team (basedon current role), know the responsibilities associated withthe role currently adopted (if any), know relationship link-ages existing between roles and (if any) ’personal’ relationsbetween agents, know their obligations, know their respon-sibilities for any given time period, know their level of dele-gated authority and what tasks can be done autonomously,without requesting permission or waiting for a task to bedelegated. Agents must adhere to published policies govern-ing behavior. All agents have access to a shared workspacerepresenting a shared mental model of the situation. In ad-dition, agents have their own internal beliefs, desires andintentions. Agents potentially could also have individualpreferences governing features such as willingness to swaproles, likelihood of delegating or asking for help etc.

This architecture allows for some domain knowledge to beencoded in plan libraries, role libraries and task descriptionsat design time. However, it allows for agents to update roleallocations and current shared mental models dynamically.There is no attention management module made explicit,but this and decision management processes (c.f. R-CASTarchitecture, [13] are important and will be provided by theunderlying BDI architecture.

Agents might be required to show initiative - by volunteeringto take on roles they are capable of performing or have par-ticular expertise with, if they have the time and resources to

60

devote to such roles; by taking action in an urgent situationwhen there is no time to negotiate or delegate.

3.4.1 Managing reorganization in the team3.4.1.1 Time periodsWork in time periods, such that for any time period, t, theteam structure is static, but then at a new time periodt +k, the environment has changed, significantly enoughto warrant reorganization of the team. (k is a variableamount of time, not a constant.) The leader controllingagent would decide that reorganization was required or couldbe prompted to reorganize by a human team member.

At the start of a new time period t’, the team leader couldcall a meeting and open the floor for renegotiation or roles,alternatively, two agents can at any time agree to handoveror swap roles and then note their changed roles in the cur-rent role allocation in shared workspace. A mechanism foragents being able to define/describe/be self aware of theirobligations and relationships is needed so that the agentscan (re)negotiate their roles and responsibilities allowing ateam structure to emerge in a dynamic way.

3.4.1.2 Coordination and ControlThis is a situation of centralized decision making, wherethere is an ultimate leader who has authority and a chain ofcommand hierarchy, c.f., [23]. The team members are locallyautonomous and responsible to make local decisions withoutneed of permission, using the local autonomous/Master styleof decision making (Barber and Martin, 2001, cited in [10])

One approach for the support and control of an agent teamis to use policy management to govern agent behavior. Thisenforces a set of external constraints on behavior - externalto each agent. This enables simpler agents to be used. Poli-cies define the ’rules’ that must be adhered to in terms ofobligations and authorizations granting permissions to per-form actions [3]. It is planned to have a set of governingpolicy rules defined in the central library.

To achieve coordination between agents, one approach is toalso control interactions via external artifacts - similar toa shared data space between agents, but with an added di-mension of social structure included [14]. This will hopefullybe achieved in our system with the shared workspace andproviding agents access to current role allocations includingrelational links.

When agents join the team they agree to accept a contractof generic obligations and some general team policies [3] aswell as some more specific obligations that are associatedwith specific roles. In addition to obligations (responsibili-ties accepted that must be adhered to), an agent may haveauthority to ask another agent to perform a particular taskor adopt a role, or authority to perform particular actions.These actions could be (for example) to drive a truck to alocation and turn on the fire hose to fight a fire at that loca-tion, or to order another agent (with appropriate authorityand access) to drive a truck to a location and fight a fire,or to accept a new role as a sector commander in a newlyformed sector.

Obligations are based on position in the hierarchy. E.g. if

Table 1: Example Position-Delegation-Action Ma-trix

Action Agent PositionP1 P2 P3

Act1 0 0.5 0.5Act2 1 1 1Act3 -1(3M,4I) -0.5 0.5

a leader (or agent in higher ranked position than you) asksyou to take on a role, you are obliged to agree, but if a ’peer’asks you to swap or handover a role, you may reject, or opennegotiations on this.

3.4.1.3 Delegation and Authority to act autonomouslyThe imagined organizational structure is such that thereis controlled autonomy enabling automatic decision-makingby agents on some tasks and requiring that other tasks bedelegated, coordinated and controlled by a ’leader’ agent.Examples of automatic decisions that might be authorizedas possible without involving permission from a more senioragent are: two peer agents agree to swap roles; or two agentsmight agree to work together to work more efficiently towardrealization of a particular task.

An agent’s position in the team hierarchy defines the level ofautonomy allowed to that agent to perform actions. Actionsare defined in terms of required agents and position levelsneeded to perform this action.

A Position-Delegation-Action Matrix could be defined asshown in table 1.

Each empty cell may contain a code to indicate the level ofautonomy afforded to perform an Action (Act n) for an agentwith the corresponding position Pm. Possible codes include: 0 - Never permitted, 1 - Always permitted, 0.5 - mayrequest permission and act alone on Act n with permission,-0.5 - may engage in teamwork to help others to performthis Act n, -1 - must engage in teamwork to perform thisAct n, cannot be done alone. In the latter two cases, whereagents might work as part of a team on an Action, thenthere needs a representation of the required minimum, Mnumber of agents and the ’ideal’ number of agents neededto successfully perform this task. This could be representedin parentheses (Act3 with agent in position P1 requires atleast 3 agents to perform, and ideally is performed by 4agents).

3.4.2 Roles and ResponsibilitiesBelow we describe some responsibilities that could be asso-ciated with generic roles. These roles will be elaborated infuture to include more specific responsibilities based on theprotocols defined in the Australian incident control system(discussed in the next section).

3.4.2.1 Example responsibilities associated with genericLeader role defined at design time

• Forward planning, anticipate resource needs for nearfuture (time t+k)

61

• Broadcast/request resource needs and invite other agentsto adopt the responsibility to fulfill these resource needs

• Accept an agent (A) ’s proposal to adopt a role (R)for time period (P: between time:ttn)

• Agree/negotiate with an agent on the list of responsi-bilities (RS) allocated to a role (R)

• Set a time for a (virtual) team meeting and set invi-tations/broadcast messages to some/all agents to par-ticipate

• Keep a mental picture of the current team structure:resources available, the ’position’ of these resources inthe team hierarchy

3.4.2.2 Example responsibilities associated with genericTeam Member role defined at design time

• Be aware of own capabilities (C) and access to re-sources

• Only volunteer/accept responsibilities set RS that arein agent’s current capabilities set (RS = C)

• Act reliably, i.e., don’t lie and always act within as-signed responsibilities and policies

• Have self-knowledge of own position in the team hier-archy, and know what delegated rights and authorityto act are available

• Be flexible : prepared to abandon an existing role infavor of a new role that has a higher priority

• Be prepared to handover/swap a role if asked by aleader or an agent with position of authority higherthan self

• Be prepared to negotiate with peers to swap/handoverroles if of team benefit

• Volunteer to take on a new role if you are capable andhave access to all needed resources

• When agent can not predict success, or experiencesfailure in attempting current responsibility, relinquishthat responsibility according to agreed policy

4. TEST SCENARIO4.1 Experimental designThe scenario planned for our experiment involves a team ofhuman sector commanders each managing a separate sectorof land under threat by a spreading bushfire. There is oneoverall fire controller. Each sector commander can commu-nicate with other commanders, but has access to informationupdates regarding the spread of fire their own sector only.The sector commanders choose when and how much infor-mation is passed on to the incident controller. Followingfrom the work of Yen [13], we plan to have a virtual as-sistant agent assigned to each human agent involved in themanagement of the scenario. These virtual assistants willhave read and write access to a shared workspace regard-ing the current state of the fire and awareness of their ownnetwork of relationships to other agents in the team. The R-CAST agents were shown to help collect and share relevant

information in a complex command and control environmentand to alleviate human stress caused by the pressure of time[13].

Each assistant will adopt one or more roles from the rolelibrary according to the role allocated to their human coun-terpart. If their human counterpart changes or delegatessome of their roles, it will then be necessary that the agentsnegotiate to update their roles so that they are still helpfulto the person they are paired with. Agent assistant roleswill include: Incident Controller, Sector Commander, Op-erations officer, planning officer, logistics officer. Initially,when the fire size is small, the incident controller will alsobe performing the role of operations officer, planning officerand logistics officer. As the fire grows and spot fires appear,some of these roles will be delegated to new personnel. Atthis stage, the agents will be asked to reorganize themselvesand dynamically update their roles accordingly.

In addition to the assistant agents, there will be additionalagents in monitoring roles. These agents will update theshared mental workspace with updates on situation aware-ness. The monitoring agents have limited information ac-cess, so there is distributed awareness across multiple agentsof the overall situation. These monitoring agents will moni-tor changes to the fire disaster in one sector. We might alsoengage specialized monitoring agents with specific roles toprotect particular resources in the landscape e.g. a house.

4.2 An exampleA fire is spreading across the landscape, each agent roleis either responsible for an appliance such as Fire Truck,Bulldozer or Helicopter, or is responsible for monitoring thelandscape in a particular area, or is acting as a dedicatedassistant to a human agent. Agents adopting the monitoringagent roles have limited information, so that there is distrib-uted awareness across multiple agents of the overall situa-tion. These monitoring agents are responsible for initiatinginformation flow to other monitoring agents and people - orperhaps for updating a central ’map’ or shared awarenessspace with abstractions summarizing significant data. Eachmonitoring agent role has visibility of one sector only.

Landscape is broken into 3 sectors; each sector has a hu-man sector commander. Each human sector commander ishelped by a monitoring agent that has visibility and aware-ness regarding that sector of landscape. In one sector, thereis a house on top of a hill. A fire is spreading toward thissector from an adjoining sector. Wind direction is encour-aging the spread of the fire and if it keeps traveling in thisdirection, it will take off up the hill and endanger the house.

There are a limited number of fire-fighting appliances: 2fire trucks, 1 bulldozer, 1 helicopter. The incident controlleris aware of all appliances and the sector they are located.Sector commanders are only aware of resources in their ownsector. Assistant agents are allocated to assist each sectorcommander, and in the roles corresponding to the four mainfunctions in the ICS. Special protective monitoring agentsare responsible for monitoring threat to a particular resource- e.g. house, tree plantation, etc. The fire begins in sector1, spreads toward sector 2. The house is in sector 2. Thehelicopter is at home base in sector 3. The house needs

62

protection. The agents and sector commanders will need tomobilize resources to stop the fire spreading and save thehouse.

5. DISCUSSIONThis design is yet to be implemented, although preliminaryfeasibility study has been conducted to test if agents wouldbe able to satisfactorily access the landscape and simulationinformation within NFC. This would enable our syntheticagents to automatically access the simulation environmentin a similar way to human agents would. It is planned thatdevelopment on our BDI agents will begin in 2006. It is notin the scope of this project to investigate agent coordinationand communication protocols, nor agent negotiation proto-cols. These aspects will be informed by existing research inthese areas.

It is not an aim of this project to replace human fire con-trollers with artificial agents, rather to use the fire fightingdomain as a good case study to implement and test our teamstructure in a controlled, but realistically complex, dynamicvirtual world. It is hoped that our agents will be sophisti-cated enough to be able to (at least partially) replace humanagents in the simulation training exercise and that we cancompare human behavior with artificial agent behavior toinform our design. It may be that our work provides agentsthat could assist humans in the real-time management ofdynamic disasters, however we make no claims that this willbe so.

It is our intention to implement agents to meet our proposedrequirements and interface these agents in the virtual simu-lation world of NFC and observe their collaborative behav-ior. Our particular interest initially is to see if the agents cancommunicate with each other in a way to provide assistanceto the humans involved in improving shared situation aware-ness. Also, we are interested to see how the agents performin team reorganization. In the initial stages, it is our inten-tion to create a simulation involving agents as assistants tothe key human personnel involved in the fire management.

In later simulations, we hope to be able to substitute vir-tual (expert) agents for human agents in the managementscenario and perhaps use such agents to aid with trainingexercises. It has been found that providing expert examplesof dynamic decision making in repeated simulations can helpimprove human performance [5, 28], there might be potentialfor our agents being used in training of incident managementteams with virtual disasters in NFC. There also possibilitiesto use the NFC as a playground for an entirely virtual teamand investigate the reorganizational and collaborative capa-bilities of our team agents within this mini-world.

This is work in progress. This paper describes our positionin terms of how sophisticated agents might be structured ina team. We are planning to create specialized agents whoshare access to a role library and share team goals. Wepropose that the agents require awareness of their own posi-tion and relationship to others in the team, be committed toteam goals, accept leadership and authority and be preparedto be flexible and adaptable - to handover responsibilities orswap roles if necessary. We are designing agents with anteam infrastructure to support dynamic reorganization.

6. ACKNOWLEDGEMENTSThe authors thank Dr. Mary Omodei and her team at La-Trobe University for their support with Network Fire Chiefsoftware and disaster management issues in the firefightingdomain.

7. REFERENCES[1] K. Baker, E.E. Entin, K. See, B.S. Baker,

S. Downes-Martin, and J. Cecchetti. Dynamicinformation and shared situation awareness incommand teams. In Proceedings of the 2004International Command and Control Research andTechnology Symposium, San Diego, CA, June 2004.

[2] G. A. Bigley and K. H Roberts. The incidentcommand system: High reliability organizing forcomplex and volatile task environments. Academy ofManagment Journal, 44(6):1281–1299, 2001.

[3] J Bradshaw, P Beautement, M Breedy, L Bunch,S Drakunov, P Feltovich, R Hoffman, R Jeffers,M Johnson, S Kulkarnt, J Lott, A Raj, N Suri, andA Uszok. Handbook of Intelligent InformationTechnology, chapter Making Agents Acceptable toPeople. Amsterdam IOS Press, 2003.

[4] M Brown. Managing the operation. Technical report,Security and Emergency Management ConferenceUNSW, 2004.

[5] Gonzalez C. Decision support for real-time dynamicdecision making tasks. Organizational Behavior andHuman Decision Processes, 96:142–154, 2005.

[6] K Carley and R Hill. Dynamics of organizationalsocieties: Models, theories and methods, chapterStructural Change and Learning within Organizations.MIT/AAAI Press, Cambridge, MA., 2001.

[7] Michael L. Greenberg David M. Hart Cohen, Paul R.and Adele E. Howe. Trial by fire: Understanding thedesign requirements for agents in complexenvironments. AI Magazine, 10(3):32–48, 1989.

[8] L Comfort, K Ko, and A. Zagorecki. Coordination inrapidly evolving disaster response systems. the role ofinformation. American Behavioral Scientist, 48(3):295– 313, 2004.

[9] F.J. Diedrich, E.E. Entin, S.G. Hutchins, S.P.Hocevar, B. Rubineau, and J. MacMillan. When doorganizations need to change (part i)? coping withincongruence. In Proceedings of the Command andControl Research and Technology Symposium,,Washington, DC., 2003.

[10] V. Dignum, F. Dignum, V. Furtado, A. Melo, andEA Sonenberg. Towards a simulation tool forevaluating dynamic reorganization of agents societies.In Proceedings of workshop on Socially InspiredComputing @ AISB Convention, Hertfordshire, UK,2005.

[11] E E Entin. The effects of leader role and task load onteam performance and process. In Proceedings of the6th International Command and Control Research and

63

Technology Symposium, Annapolis, Maryland, June2001.

[12] E.E. Entin, F.J. Diedrich, D.L. Kleinman, W.G.Kemple, S.G. Hocevar, B. Rubineau, and D. Serfaty.When do organizations need to change (part ii)?incongruence in action. In Proceedings of theCommand and Control Research and TechnologySymposium., Washington, DC, 2003.

[13] X. Fan, S. Sun, G.and McNeese M. Sun, B.and Airy,and J Yen. Collaborative rpd-enabled agents assistingthe three-block challenge in command and control incomplex and urban terrain. In Proceedings of 2005BRIMS Conference Behavior Representation inModeling and Simulation, pages 113 – 123, UniversalCity. CA, May 2005.

[14] N Findler and M Malyankar. Social structures and theproblem of coordination in intelligent agent societies.2000.

[15] J Hicinbothom, F Glenn, J Ryder, W Zachary,J Eilbert, and K Bracken. Cognitive modeling ofcollaboration in various contexts. In Proceedings of2002 ONR Technology for Collaborative Commandand Control Workshop, pages 66–70. PAG TechnologyManagement, 2002.

[16] Eric Matson and Scott A. DeLoach. Autonomousorganization-based adaptive information systems. InIEEE International Conference on KnowledgeIntensive Multiagent Systems (KIMAS ’05),,Waltham, MA, April 2005.

[17] R. Nair, M. Tambe, and S. Marsella. Team formationfor reformation in multiagent domains likerobocuprescue, 2002.

[18] M Omodei, J McLennan, G Cumming, C Reynolds,G Elliott, A Birch, and A. Wearing. Why dofirefighters sometimes make unsafe decisions? somepreliminary findings, 2005.

[19] M. M. Omodei. Network fire chief. La TrobeUniversity.

[20] D Pynadath and M Tambe. An automated teamworkinfrastructure for heterogeneous software agents andhumans. Autonomous Agents and Multi-AgentSystems, 7, 2003.

[21] N Schurr, J Maecki, M Tambe, and P Scerri. Towardsflexible coordination of human-agent teams. InMultiagent and Grid Systems, 2005.

[22] W Smith and J Dowell. A case study of co-ordinativedecision-making in disaster management. Ergonomics2000, 43(8):1153–1166.

[23] R. Sun and I. Naveh. Simulating organizationaldecision-making using a cognitively realistic agentmodel. Journal of Artificial Societies and SocialSimulation, 7(3), 2004.

[24] G Tidhar, A S Rao, , and L Sonenberg. On teamworkand common knowledge. In Proceedings of 1998International Conference on Multi-Agent SystemsICMAS98, pages 301–308, 1998.

[25] Gil Tidhar and Liz Sonenberg. Observations onteam-oriented mental state recognition. In Proceedingsof the IJCAI Workshop on Team Modeling and PlanRecognition, Stockholm, August 1999.

[26] T Wagner, V. Guralnik, and J Phelps. Achievingglobal coherence in multi-agent caregiver systems:Centralized versus distributed response coordination.In AAAI02 Workshop Automation as caregiver: TheRole of Intelligent Technology in Elder care., July2002.

[27] W Zachary and J-C Le Mentec. Modeling andsimulating cooperation and teamwork. In M J Chinni,editor, Military, Government, and AerospaceSimulation, volume 32, pages 145–150. Society forComputer Simulation International, 2000.

[28] W Zachary, W Weiland, D Scolaro, J Scolaro, andT Santarelli. Instructorless team training usingsynthetic teammates and instructors. In Proceedings ofthe Human Factors and Ergonomics Society 46thAnnual Meeting, pages 2035–2038. Human Factorsand Ergonomics Society, 2002.

64

Soft-Restriction Approach for Traffic Managementunder Disaster Rescue Situations

Hiroki Matsuihiroki.matsui@aist.go.jp

Kiyoshi Izumikiyoshi@ni.aist.go.jp

Itsuki NodaI.Noda@aist.go.jp

Information Technology Research InstituteNational Institute of Advanced Industrial Science and Technology (AIST)

Ibaraki, 305-8568, Japan

ABSTRACTIn this article, we investigate social behaviors of tra±c agentswith road-restriction information, and show the possibilitiesof soft-restriction to get an equilibrium where social bene¯tis improved.

Under disaster and rescue situations, tra±c resources be-come so tight due to damages to roads and large logisticalrequirements. If there are no restrictions, serious tra±c jamsoccur on most of major roads and intersections so that emer-gency and prioritized vehicles also get large delay to reachtheir destinations. A general way to guarantee tra±c ofthe emergency vehicles is to impose legal controls on someroads by which only approved vehicles can take them. Suchstrict methods may, however, also cause ine®ective situationwhere tra±c jams of unapproved vehicles block emergencyones. In order to overcome such dilemma, we introduce‘soft-restriction,’ an approach that imposes large penaltieson the unapproved vehicles taking regulated roads instead ofexcluding from there. Using a social multi-agent simulation,we show that the soft-restriction enables us to control tra±cin non-strict way by which distribution of vehicles reachesan equilibrium where both of emergency and normal vehiclessave their delay.

Keywordstra±c management, disaster situation, multi-agent simula-tion, social simulation

1. INTRODUCTIONUnder disaster and rescue situations, tra±c resources be-

come so tight because many roads are damaged by the dis-asters. In addition to it, the situations to bring many kindsof resources for rescue activities require unusual and hugetra±c. These activities are performed by both of public andprivate sections because the transportation capacity of thegovernment is limited.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.AAMAS’06May 8–12 2006, Hakodate, Hokkaido, Japan.Copyright 2006 ACM 1-59593-303-4/06/0005 ...$5.00.

The ¯rst task of the headquarters of the local tra±c centeris to ¯nd available roads to connect outside and inside of thedamaged area [6]. The information should be broadcastedfor the general public in order to avoid confusion.

Their second task is to determine restricted roads foremergency vehicles. If there are no restrictions, most oftra±c converges into major and useful roads and causes se-rious tra±c jams. As a result the emergency and prioritizedvehicles also get large delay to reach their destinations. Inorder to avoid such situations and to guarantee tra±c of theemergency vehicles, a general action the tra±c center takesis to impose legal controls on some roads by which onlyapproved vehicles can take. Because this kind of actionshave legal force, the restriction is applied in a strict wayby which all drivers who do not follow it get legal penalty.Such strict methods may, however, also cause another inef-fective situation where tra±c jams of unapproved vehiclesblock emergency ones.

This kind of social ine±ciency occurs because all peopletend to make the same decision based on the same informa-tion. Kawamura et al. [2] show that the total waiting timeincrease when all agents behave based on the same informa-tion in a theme park. Yamashita et al. [7] also show thattra±c congestion at the normal time also increases whendrivers make a decision of routing using the same tra±c in-formation. On the other hand, both of these works reportthat such social ine±ciencies are reduced when some agentsget di®erent information so that each agent becomes to havea varied policy to choose the common resources.

We try to introduce the same idea into the control of emer-gency tra±c under disaster and rescue situations. In stead ofthe strict legal restrictions, we design a ‘soft-restriction’ bywhich we encourage drivers to have varied decision-makingpolicies. The variation of the policies will let concentratedtra±c be diverted so that the congestion will be reduced.

In the rest of this article, we de¯ne a simple model of traf-¯c under disaster situations in section 2. Using the model,section 3 describes experimental setup and results of a multi-agent simulation. We also discuss various possibilities andissues to apply this approach to the actual situations in sec-tion 4, and conclude the summary of result in section 5.

2. MULTI-AGENT TRAFFIC MODELIn order to investigate and evaluate tra±c phenomena un-

der disaster situations, we design a model of multi-agenttra±c simulation. The model consists of a road network,

65

ST

V

OD

UD2

D1

Figure 1: Simple road network under disaster situ-

ation

tra±c agents, and a tra±c control center. We keep themodel simple enough in order to make it clear what is thefactor of social e±ciency and ine±ciency using ‘soft-restriction.’

2.1 Road NetworkThe simple road network which is used on our model is

shown in Figure 1. The all links except O-S, S-U and U-V

in the network have two lanes each way and the same speedlimit. The link O-S is wide and has three lanes each way andthe same speed limit as ones of most links in the network.The links S-U and U-V are narrow ones, so have only onelane each way and much lower speed limit than ones of theothers. Each lane of each link is not the exclusive lane tovehicles which go to speci¯ed link. In other words, vehiclescan go straight, turn right or left from any lanes. Currently,all intersections do not have signals.

The all vehicles in this network have the same origin O

and the same destination D.1 We assumed a disaster andrescue situation, the network has an emergency vehicularroute as the shaded link T-D1 on the network. The routeis an exclusive route for emergency and rescue vehicles, it isthe link with which the distance of the route is shortest insome routes of the OD in order to guarantee tra±c.

Public vehicles, which are not approved to use the emer-gency vehicular route T-D1, must take one of two detourroutes, wide route T-V-D2-D1 and narrow route S-U-V-

D2-D1-D. The main di®erence of these two route is that thewide route shares the link S-T with the emergency route.Therefore, vehicles taking the wide route (we refer such ve-hicles as wide-route vehicles afterward) may a®ect the tra±cof emergency vehicles more than vehicles taking the narrowroute (narrow-route vehicles). However, the narrow routehas not enough capacity to handle all public vehicles.

In this network, free travel time of the wide route is shorterthan one of the narrow route. In such a disaster situation,however, selecting the wide route is not always the best routedue to large tra±c volume. Figure 2 shows changes theaverage travel time2 of each type of vehicles according tochanges of ratio of the number of wide-route and narrow-route vehicles. In this graph, the average of the travel timesof emergency, wide-route, and narrow-route vehicles are sep-

1There are no vehicles on the opposite lanes of each linkfor simplicity. So we do not consider the e®ect of them onturning vehicles at intersections.2Travel times in the network are measured from O to D1or D2, not to D in order to ignore the loss to °ow togetherat D.

150

200

250

300

350

400

450

500

10:0 9:1 8:2 7:3 6:4 5:5 4:6 3:7 2:8 1:9 0:10

Ave

rage

Tra

vel T

ime

[s]

Ratio of Public Vehicles’ Routes (Wide:Narrow)

Emergency VehiclesPublic Vehicles (Wide Rt.)

Public Vehicles (Narrow Rt.)

Figure 2: Average travel times of vehicles in the

network

10

20

30

40

50

60

70

80

90

100

110

120

10:0 9:1 8:2 7:3 6:4 5:5 4:6 3:7 2:8 1:9 0:10

Sta

ndar

d D

evia

tion

of T

rave

l Tim

es

Ratio of Public Vehicles’ Routes (Wide:Narrow)

Emergency VehiclesPublic Vehicles (Wide Rt.)

Public Vehicles (Narrow Rt.)

Figure 3: Standard deviations of travel times of ve-

hicles in the network

The ratio of emergency vehicles is equal to the one ofall public vehicles in Figure 2 and 3.

arated. The vertical axis indicates the average travel timeper vehicle, and the horizontal axis shows ratio of wide-route/narrow-route vehicles, where left/right ends are thecase the most of vehicles take the wide/narrow routes, re-spectively. The both travel times of emergency and wide-route vehicles, which use the link S-T, increase when theratio of wide-route vehicles becomes large (left-side), be-cause the tra±c volume exceeds the tra±c capacity of thelink S-T. Especially the ratio is larger than 5:5, the delayof tra±c increases quickly. It is also the reason that thenumber of right-turning vehicles at the node T gets largerbecause turning right takes more time than going straightahead. On the other hand, the travel times of emergency andwide-route vehicles increase slowly in the right-side (the casemost of public vehicles take narrow route), because increaseof the number of right-turning vehicles at the node S causesa tra±c jam on the link O-S. In any cases, the travel time ofthe narrow-route vehicles changes little at all ratios, becausethey are a®ected little by vehicles of the other routes.

66

The state of system optimal(SO) by Wardrop’s secondprinciple3 [5] of tra±c °ow in this network is at the ratio ofpublic vehicles’ route Wide :Narrow = 4 : 6. The state ofpublic vehicle optimal is also around the ratio. The stateof user equilibrium(UE) by Wardrop’s ¯rst principle4 [5] isaround the ratio of public vehicles’ route Wide : Narrow =5.75 : 4.25.

It is also important under disaster and rescue situationsthat the travel times of emergency vehicles are stable be-cause it enables us to plan rescue activities easily. Figure 3shows changes the standard deviation(SD) of travel times ofeach type of vehicles according to changes of ratio of wide-route and narrow-route vehicles. The vertical axis indicatesthe SD of time of each type of vehicles, and the horizontalaxis shows the same ratio as one of Figure 2. As the averagetravel time, the SD of emergency vehicles’ travel time getslarger when the most of vehicles use the same route. It isthe reason that a number of turning vehicles at the nodeT or S interfere with emergency vehicles going straight. Inparticular the e®ect of the turning vehicles is large at thenode T because emergency vehicles not be able to moveduring turning vehicles on the outside lane of the link S-T

which has only two lanes. The SD of emergency vehicles’travel times takes the minimum value around the ratio ofpublic vehicles’ route Wide : Narrow = 4 : 6. The ratio isthe best from both viewpoints of the system optimal andthe °uctuation of emergency vehicles’ travel times.

2.2 Traffic Control CenterThe tra±c control center (CC) carries out tra±c manage-

ment policies in order to guarantee tra±c of the emergencyvehicles. It corresponds to a police or a local government inthe real world. As mentioned in Section 1, a traditional wayto control such tra±c is strict-restriction where no publicvehicles can use route S-T. This is, however, not always ap-propriate from the viewpoint of system optimal and the sta-bility of tra±c. In the road network shown in Section 2.1, thestrict-restriction is the case the ratio of wide-route/narrow-route vehicles is 0:10, where the average and SD of traveltime of emergency vehicles are larger than SO (the ratio =4:6). On the other hand, if CC does nothing, the situationwill fall into UE (the ratio = 5.75:4.24), which is worse thanstrict-restriction. In order to to control the ratio of vehi-cles and to make the situation close to SO, we assume thatCC can use two types of ‘soft-restriction’ methods: One ofthem is ‘penalties’ to public vehicles which select the wideroute. The penalties are given to the travel time of the ve-hicles. The way is direct and strong but cost highly becausesuch restriction requires much human resources to managethe vehicles. The other is a ‘sign’ to public vehicles. CCuses the sign to inform the vehicles probability of getting apenalty if they select the wide route before they select theroute. The way is not direct and weak but cost little.

The conditions to give penalties, the amount, with orwithout the sign etc. are dependent on simulation settings.

2.3 Traffic AgentsThere are two kinds of tra±c agents as vehicles in the

3At equilibrium the average travel time of all vehicles isminimum.4The travel times in all routes actually used are equal andless than those which would be experienced by a single ve-hicle on any unused route.

network on our model. One of them is emergency vehicleagents (EVA). The agents represent emergency vehicles orrescue vehicles like ambulances, vehicles to transport aidsupplies, ones to carry wrecks out and so on. This type ofagents can go through the emergency vehicular route. Thepurpose of the agents is to reach the destination earlier byusing the route. The agents have no alternatives of theiraction.

The other is public vehicle agents (PVA). The agents rep-resent public vehicles, not special ones which have o±cialmissions. Each agent can choose one of two routes, wide-and narrow-route, to travel from O to D. We assume thatagents are sel¯sh: each agents chooses a route by which itcan get the destination faster without penalties than by an-other. In order to make the simulation simple, the penaltyis measured as a loss-time of tra±c and added to the traveltime. Because each agent can not know actual travel timefor each route beforehand, we assume that agents uses eval-uation functions described below to make decisions.

The evaluation functions are trained by experiences ofeach agents. In order to the training possible, we assumethat the agents repeat the travels from O to D after eachtravel ends and learn which route is better with the follow-ing method based on their own experiences. The agents haveevaluated value of each route in each state. They decide oneof two route to travel based on the evaluated values.

Route Selection Mechanism and Information Source

We suppose that each agent has an evaluation function foreach route, which estimate travel time of the route in thecurrent situation. Using the evaluation function, the agentsselect a route based on ε-greedy action selection [4]. Namelythe agents select the route whose estimated travel time isshorter than the other with probability (1 − ε), otherwisethey select a route randomly.

We assume that each agent can use two types of infor-mation to calculate the evaluation function, sign state andlocal traffic situation. The sign state means whether the CCdisplays the ‘sign’ or not. By the sign state information,each agent can know the intention of the CC that indicatesglobal trends of road situation implicitly.

The local tra±c situation indicates the number of vehicleson each lane in the same road the vehicle is using. Wesuppose that vehicles tend to select a lane dependent ontheir next link; a vehicle tends to use the right lane if thevehicle will turn right, a vehicle tends to use the central orleft lane if the vehicle will go straight ahead at node S. Theaverage numbers of vehicles in cases that the ratio of PVAs’routes is ¯xed are shown in Table 1. As seen in this table,each agent can know current local trends of route selectionamong public vehicles by the local tra±c situation.

Evaluation function for each route

We investigate two cases of information availability, usingonly global information (sign state) and using both of globaland present local information (local tra±c situation).

1. Only global informationWhen an agent can use only global information, theevaluation function T consists of just a table of fourvalues, that is,

T (r, s) = Cr,s, (1)

where r is the route, w (wide) or n (narrow); s is the

state, with or without the sign; T (r, s) is the estimated

67

Table 1: Average number of vehicles on each lane of

the link O-S

Values in each cell mean “Average number of vehi-cles(Standard deviation of numbers of vehicles).”

ratio of Wide Rt. Narrow Rt.PVAs’ Rt. (central and left) (right)

10 : 0 86.1(6.4) 21.8(3.0)9 : 1 80.8(7.7) 22.7(3.3)8 : 2 76.7(7.4) 23.5(3.5)7 : 3 72.8(7.7) 24.6(3.7)6 : 4 68.0(8.0) 25.5(3.8)5 : 5 60.0(8.4) 27.1(4.2)4 : 6 51.7(7.4) 33.3(4.7)3 : 7 53.0(7.5) 37.8(3.9)2 : 8 53.4(7.5) 42.2(2.9)1 : 9 52.9(6.5) 42.2(2.9)0 : 10 51.9(6.4) 43.4(2.5)

travel time via route r at the state s; Cr,s is the eval-uated value of the route r at the state s.

2. With local informationWhen the local information is available, each agent willbecomes estimate travel time more precisely, becausethe local information provides analog value that tightlyrelated with the future ratio of route selection. Forthe simplicity, we suppose that each agent estimatesthe evaluation of each route by a linear function of thelocal information, that is,

T (r, s, Lr) = Kr,sLr + Cr,s, (2)

where Lr is the number of vehicles on lanes at theirroute decision, takes the sum of numbers of vehicleson the central and left lane if r is w, the numbers ofvehicles on the right lane; Kr,s and Cr,s are parametersto estimate travel time.

Learning evaluation function

As wrote above, we assume that each agent is sel¯sh. Thismeans that agents try to adjust evaluation function in orderto get more bene¯t from their route-selection. In order tore°ect this property, we suppose that each agent learns itsown evaluation function using resulted travel time.

After each travel, the agents update the evaluation func-tion with the actual travel time T by the following methods.

1. Only global informationThe agents update the estimated travel time via r atthe state s as follows,

Cr,s ← (1 − α)Cr,s + αT , (3)

where α is their learning ratio.

2. With local informationThe agents update the parameters Kr,s, Cr,s to esti-mate travel time. The agents update the parametersby minimal changes to decrease the error E = |T − T |by the steepest gradient algorithm. The parameters

Figure 4: Multi-agent simulation on Paramics

This image is the view from the above of the node Oto direction of the node V.

update as follows,

Kr,s ← Kr,s + αdLr, (4)

Cr,s ← Kr,s + αd, (5)

d =T − T

L2 + 1.

PVAs do not update other values than one of selected routeand the perceived state independent of type of evaluatedvalues.

3. SIMULATION WITH TRAFFIC MODEL

3.1 Traffic SimulatorWe constructed the model on a tra±c simulator Param-

ics [3, 1]. We show an image of our multi-agent simulationon Paramics in Figure 4. Paramics is a microscopic traf-¯c simulator that simulates behavior of each vehicle and wecan set properties of each vehicle. Therefore the simulator issuitable for use as the base of multi-agent model. We builtthe network on the simulator and implemented routing algo-rithms of agents, employed the default model of Paramics onthe other behaviors and properties of vehicles, for examplehow to change lanes, acceleration.

3.2 Simulation SettingsWe examined proposed ‘soft-restriction’ by multi-agent

simulation in various settings of penalties and the sign byCC. Common settings of our simulations in this section areas follows.

Traffic agent from the origin O

• The ratio of EVAs and the one of all PVAs are equal.

• The number of agents which leave the origin is about60 per minute.

Number of PVAs

• The number of PVAs is 400. It is decided based onthe number of PVAs in the network at the same time.PVAs repeat the travel from O to D one after another.

68

Learning method of PVAs

• The value of ε in ε-greedy is 0.1. All PVAs have thesame value and the value is not changed in the simu-lation.

• The learning ratio of each PVA is randomly given inthe range 0–1 at the start of the simulation. The ratiois not changed in the simulation.

• The estimated trip time Cr,s of PVAs which learn withonly global information and the parameters Kr,s andCr,s of PVAs which learn with local information areinitialized to 0 at the start of simulations.

Experimental period

• The duration of one simulation is 24 hours in the model.

Evaluation method

• We evaluate the policies of CC by the average traveltime of EVAs and the cost of penalties. The cost ofpenalties are de¯ned as follows,

Cost = p × n, (6)

where p is the penalty time to give at one penalty; nis the number of times giving penalty.

3.3 Simulation Results

3.3.1 No PenaltyAt ¯rst, we experimented with no penalty. Only emer-

gency route is set as tra±c management. As the result,the average travel times of EVAs were 252.0 seconds in thecase that PVAs learn with only global information, 279.5seconds in the case that PVAs learn with local information.The ratio of PVAs’ routes are shown in Figure 5. The ratiostabilized around the user equilibrium point independent ofthe learning type of PVAs in this setting.

3.3.2 Anytime with PenaltiesSecondly, we experimented the case that CC gave penal-

ties to all PVAs which selected the wide route. We show theratio of PVAs’ routes are shown in Figure 5 with a large p.In this setting, the ratio approaches Wide :Narrow = 1 : 95

as the penalty time p gets larger. The average travel timeof EVAs is about 205 seconds at the ratio.

3.3.3 With Penalties when EVAs Get DelayedIn this model, EVAs take much travel time with no penal-

ties. On the other hand, travel times of EVAs are short inthe case of penalties to all PVAs via the wide route. How-ever the cost of the policy is too high and the average traveltime of all agents is not short because the equilibrium is sofar from the ratio at the system optimum. Then we try toovercome the problem with “soft-restriction.” The essentialidea of this policy is that CC gives penalties to PVAs whichselect the wide route in only situations that the recent av-erage travel times of EVAs get longer than a threshold witha probability Pp. In the case that CC uses the sign, CCpresents the sign to PVAs in the situations the time of EVAsgets over the threshold, otherwise hides the sign.

5The ratio does not approach 0 : 10 because ε = 0.1.

0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100 120

Rat

io o

f PV

As

whi

ch s

elec

t rou

tes

Trials

no penalty (G)no penalty (G&L)

anytime with penalties (G, p = 200)anytime with penalties (G&L, p = 225)

Figure 5: Ratio of PVAs which select the wide route

in the cases of no penalty and anytime with penalty

G: The case that PVAs learn with only global infor-mation.G&L: The case that PVAs learn with local informa-tion.One trial means time in that all agent travel once.

We assumed that the threshold is 200 based on Figure 2and that CC consider to calculate the average travel timeto compare with the threshold of the last 20 EVAs whicharrives the destination. We experimented with the penaltyp in the range 20–200 and the probability Pp in the range0.2–1.0. The results of simulations with or without the signin each learning type of PVAs are as follows.

3.3.3.1 Case with only global information.At ¯rst, we carried out an experiment of the case that

PVAs learn with only global information. The relation ofthe average travel time of all EVAs and the cost of penaltiesper hour are shown in Figure 6 as the results. Figure 8 alsoshows changes of standard deviations of the travel time foreach case.

While the average times with and without the sign are al-most identical in the case that CC managed with lower cost,there is explicit di®erence between the average times withand without the sign in the case that with larger p. With-out the sign, the total cost increases and the average traveltime decreases smoothly when CC increases large penalty pand large probability Pp. On the other hand, with the sign,even if CC use the large penalty p and/or large probabilityPp, changes of the total cost and the average time stop inthe middle. This phenomenon is appeared as a horizontalline about 215 seconds in the ¯gure. This phenomena occursbecause the most of agents switch their choices from wide-to narrow-route only when the sign appears. This meansthat the tra±c situation switches drastically between theratio 10:0 and 0:10. Therefore, the standard deviation in-creases when the total cost increases in the case of ‘no-sign’in Figure 8.

3.3.3.2 Case with global and local information.The next experiment is the case when PVAs can use eval-

uation function with local information. The relation of the

69

300

280

260

240

220

200

180

160 0 5000 10000 15000 20000 25000 30000 35000 40000 45000

UE

Th

SO

Ave

rage

Tra

vel T

ime

of E

meg

ency

Veh

icle

s [s

]

Cost of Penalties / hour

No SignWith Sign

Figure 6: Average travel time of EVAs with ‘soft-

restriction’ in the case that PVAs learn with only

global information

300

280

260

240

220

200

180

160 0 20000 40000 60000 80000 100000 120000 140000 160000

UE

Th

SO

Ave

rage

Tra

vel T

ime

of E

meg

ency

Veh

icle

s [s

]

Cost of Penalties / hour

No SignWith Sign

Figure 7: Average travel time of EVAs with ‘soft-

restriction’ in the case that PVAs learn with local

information

30

35

40

45

50

55

60

0 5000 10000 15000 20000 25000 30000 35000 40000 45000

SD

of T

rave

l Tim

es o

f Em

egen

cy V

ehic

les

Cost of Penalties / hour

No SignWith Sign

Figure 8: Standard deviations of travel time of EVAs

with ‘soft-restriction’ in the case that PVAs learn

with only global information

30

35

40

45

50

55

60

0 20000 40000 60000 80000 100000 120000 140000 160000

SD

of T

rave

l Tim

es o

f Em

egen

cy V

ehic

les

Cost of Penalties / hour

No SignWith Sign

Figure 9: Standard deviations of travel times of

EVAs with ‘soft-restriction’ in the case that PVAs

learn with local information

average travel time of all EVAs and the cost of penalties perhour are shown in Figure 7 as the previous case. Figure 9also shows changes of the standard deviations.

In this case, the e®ect of the sign is clear and positive.While the travel time is reduced linearly when CC paysmore cost in the case of ‘no sign’, the similar e®ects on thetravel time can be realized with half of costs in the case of‘with sign’. In addition, similar to the case of only globalinformation, the total cost is saturated in the middle (about80000). This means that there is a boundary of the amountof penalty. This is good news for CC because CC needs notprepare more resources for the penalty than this boundary.

4. DISCUSSIONThe results of the simulations in the previous section tell

interesting features of this kind of tra±c as follows; Whenthe agents can use only global information (in other words,its own experiments), the ‘sign’ method has negative e®ectson the tra±c. On the other hand, when the agents can useinformation of the current information, the method works

well. This means that “showing more information” is notalway a good strategy. In this case, the sign service (showingthe abstracted status of the tra±c) by CC is only e®ectivefor agents who can use the local information. When PVAsperceive number of vehicles to decide routes, they seemsto use the sign state to recognize the ratio of other agentsclearly. The penalties by CC is unavoidable for PVAs ifthey select the wide route. Therefore PVAs tend to avoidthe wide route when making decision with the sign. CC alsosucceeds to save cost in both cases. However the phenomenawe have not analyzed them yet.

Note that the usage of the local information is not con-trollable by CC. The average travel time of case of with-local-information (Figure 6) is larger than ones of the caseof with-global-information (Figure 7). The ‘sign’ methodsucceeds to reduce the travel time in the same level of with-global-information case.

How can we apply the soft-restriction policies to the actualroad networks? In order to apply, we need to investigaterelation between of the actual penalty for agents and the

70

cost of CC in more detail. In this work we assume thatthe amount of the cost is proportional to the strength ofthe penalty. In general, however, actual penalty will berealized by punishing with a ¯ne. In the case using the ¯neas the penalty, the cost to impose penalty is constant notdependent on the amount. We need to analyze the result insuch case.

5. CONCLUSIONWe investigated social behaviors of tra±c agents with

road-restriction information under disaster and rescue sit-uations with a multi-agent tra±c model. We proposed soft-restriction with penalties in a road network under disasterand rescue situation. We found that it is e®ective to getan equilibrium where social bene¯t is maximized. We foundalso that suppling the information of restrictions is e®ectiveto save cost and achieve the purpose of management.

In this article, we use a very simple road network. Weneed to experiment and analyze with another large networkmodel in order to apply the actual road networks.

6. REFERENCES[1] G. Cameron, B. J. N. Wylie, and D. McArthur.

Paramics: moving vehicles on the connection machine.In Supercomputing ’94: Proceedings of the 1994ACM/IEEE conference on Supercomputing, pages291–300, New York, NY, USA, 1994. ACM Press.

[2] H. Kawamura, K. Kurumatani, and A. Ohuchi.Modeling of theme park problem with multiagent formass user support. In Working Note of the IJCAI-03Workshop on Multiagent for Mass User Support, pages1–7, Acapulco, Mexico, 2003.

[3] Quadstone Ltd. Paramics: Microscopic tra±csimulation. http://www.paramics-online.com/.

[4] R. S. Sutton and A. G. Barto. Reinforcement Learning:An Introduction. A Bradford Book, The MIT Press,1998.

[5] J. G. Wardrop. Some theoretical aspects of road tra±cresearch. In Proceedings of the Institution of CivilEngineers II, number 1, pages 325–378, 1952.

[6] T. Yahisa. Sonotoki saizensen dewa –Kotsukisei hamaho dewa nai! Tokyo Horei Publishing, 2000. (inJapanese).

[7] T. Yamashita, K. Izumi, K. Kurumatani, andH. Nakashima. Smooth tra±c °ow with a cooperativecar navigation system. In 4rd International JointConference on Autonomous Agents and MultiagentSystems (AAMAS), pages 478–485, Utrecht,Netherlands, 2005.

71

1

Enhancing Agent Capabilities in a Large Rescue Simulation System

Vengfai Raymond U* and Nancy E. Reed*#

*Dept. of Electrical Engineering and #Dept. of Information and Computer Sciences 1680 East-West Road, 317 POST, University of Hawaii

Honolulu, HI 96822 vengfai@hawaii.edu nreed@hawaii.edu

ABSTRACT This paper presents an enhanced and generalized model for agent behavior in a large simulation system, called RoboCup Rescue Simulation System (RCRSS). Currently the RCRSS agent model is not flexible enough to support mixed agent behaviors. Our solution extends the RCRSS and YabAPI development frameworks to create an enhanced agent model, the HelperCivilian (HC) [8]. The aim is to simulate situations in which agents can have multiple roles, and can change their capabilities during a simulation. By providing increased capabilities and configurations, a richer mix of agent behaviors becomes possible without the addition of a new agent class. Our experimental results demonstrate improved performance in simulations with higher percentages of Helper Civilians as opposed to those with the current civilian agents.

The HC population was configured and tested under different conditions, including the relative percent of HC out of a total of 100 civilian/HC agents. This HC model shows significant impact in the outcome of the rescue simulations. In addition, researchers can more easily configure a wider range of behavior in their agents. The result enables the simulation of more complex scenarios than was previously possible. In disasters, civilians are not all incapable of assisting with rescue efforts, since some are off duty medical or other trained personnel. Thus, our system enables simulations to more accurately reflect real situations.

Categories and Subject Descriptors D.2.11 [Software Architectures]: Domain-specific architectures and Patterns. I.2.11 [Distributed Artificial Intelligence]: Intelligent agents and Multiagent systems. I.6.7 [Simulation Support Systems]: Environment simulators.

General Terms Performance, Design, Simulation, Modeling.

Keywords Software Engineering, Agent Simulation Systems, Multi-Agent Systems, RoboCup Rescue, Disaster Management, Software Design and Architecture.

1. INTRODUCTION This paper presents a generalized model for civilian agents with the aim of increasing agent capabilities, and versatility and realism of simulations. This work is tested using the popular RoboCup Rescue Simulation System (RCRSS) [6]. Ideally, together with the enhanced agent development framework (ADF) and the enhanced environment simulator (RCRSS), agent developers can simulate more complex agent scenarios with higher realism and build agents more rapidly.

Our solution to enhancing the system is to generalize the base agent model while keeping it easy to use. Complex behavioral models can be rapidly developed, with the potential for application to similar distributed multi-agent simulation systems. Our enhanced base agent model is called Helper Civilian (HC) [8].

Implementation of the HelperCivilian necessitated extending several world model simulators, as well as the YabAPI agent development framework (YADF) [6]. With our enhanced agent model, a richer mix of agent behaviors becomes possible without the addition of new agent classes. We look forward to the possibility that our efforts could one day be used for saving lives.

The rest of this paper is organized as follows. The next section briefly introduces the existing RCRSS. The next section describes the details of our enhanced agent model, the HelperCivilian. Applications of the HC model for creating agent behavior and the readiness of the enhanced architecture to support agent learning are described next. Experimental results are described in section 4, including details of experimental conditions. The last two sections summarize the paper and briefly describe avenues for future work.

2. BACKGROUND The RCRSS project has brought researchers around the world together to improve and strengthen a common testbed. Not only has RCRSS provided a common tool for researchers to study rescue strategies and multi-agent systems, but also it promotes the spirit of collaboration through annual competitions and the exchange of ideas through message boards, email, and workshops [6]. We believe that this project can add to multi-agent research

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

AAMAS’06, May 8-12, 2006, Hakodate, Hokkaido, Japan. Copyright 2006 ACM 1-59593-303-4/06/0005...$5.00.

72

2

and improve awareness that enables people to be better prepared in the event of a disaster.

Figure 1. The RoboCup Rescue System (RCRSS) architecture.

Figure 1 shows the architecture of the RCRSS [7]. Version 0.43 was the standard when this project started. At this writing, version 0.48 is available, and rapid development continues. Simulators for building collapse and fire destruction are the basis for the environment, while simulators for water damage (floods or tsunami) or complex multi-level buildings have yet to be developed.

Four human agent roles are currently supported – the civilian, medical technician, fire-fighter, and policeman [6]. Civilians (bystanders in the simulation) are currently defined as having none of the capabilities present in the other agents (first aid, emergency medical services, police that can clear blocked roads, and fire fighters). Radio communication is also not available to civilians. The YabAPI framework (see Figure 2) is written in Java, and is easy to use [3], however only a small set of agent behaviors are available. Our HelperCivilian model generalizes and extends the agent model in YabAPI. We aim to simulate more complex and realistic scenarios than are currently possible.

Figure 3. RCRSS package organization

Figure 2. The architecture of the YabAPI agent framework.

73

3

3. ENHANCED AGENT MODEL Our enhanced agent model, the HelperCivilian [8], is generalized from the agents in YabAPI. We kept the ease of interaction to enable users to rapidly develop new agent behaviors. The HC model is powerful partially because the agents’ capabilities are configurable for each scenario. They are also designed to enable behaviors to be modified at run-time. The aim of on-line changes is to enable learning or adjustable autonomy in the future. A medical technician, for example, may be able to teach a civilian enough to aid in victim care.

Figure 3 shows the organization of the RCRSS package [1]. The modules are developed in the Java and C++ programming languages. To integrate the HC model with RCRSS, we studied and identified the program elements that would need modification, including object definitions and system behaviors. During this process, we found more than 39 files that needed modification. The issues addressed included:

Create the new agent type, and integrate with the rest of the system

Where to instantiate the agents and how to synchronize the states of the agents

Enable additional skills in new human agents both at design time and run-time

Enable skill ‘learning’ where one agent adds capabilities from another during runtime

Modify the Viewer [2], GIS [1], JGISEdit [9] and kernel simulator modules to support the new agent model

The HC generalized model allows the experimenter to configure each agent in a scenario with or without any combination of capabilities. A non-trained HC agent is identical to the current Civilian agent whose sole ability is to move away from damage and wait for rescue. A fully trained HC agent includes all basic capabilities such as Sense, Hear, Say and Move as well as Rescue and Clear – advanced capabilities that are currently only available in the Police and Ambulance agent models. Table 1 summarizes the capabilities of the agents in our extended platform [3].

Table 1. Capabilities of RCR agents.

Type Capabilities HelperCivilian Sense, Hear, Tell, Move, Rescue, Clear

Civilian Sense, Hear, Say, Move Ambulance

Team Sense, Hear, Say, Tell, Move, Rescue, Load, Unload

Fire Brigade Sense, Hear, Say, Tell, Move, Extinguish Police Force Sense, Hear, Say, Tell, Move, Clear Ambulance

Center Sense, Hear, Say, Tell

Fire Station Sense, Hear, Say, Tell Police Office Sense, Hear, Say, Tell

The HelperCivilian agent design is shown in UML format in Figures 4, 5, and 6. In Figure 4, the class diagram models the relationship between the HelperCivilian class and the entity classes within the world model [3].

Figure 5 shows further details about the HC class, including attributes, methods, and class associations. In order for HC agents to be configurable, a special variable skills is needed. The agent definitions are in the files objdef.h, objdef.enum.h, basic.hxx, and HelperCivilian.inl [1].

Implementation of the HC class within the YabAPI framework required additional design and implementation effort, as illustrated in Figure 6. The implementation spans three Java packages yab.io.object, yab.agent.object, and yab.agent [4]. The HelperCivilianAgent is the class designated to be derived by end users. Action and query function API’s are provided. A default behavior is also implemented for debugging purposes.In order to enable clear and rescue capabilities in HC agents, the functions rescue and clear needed updating in the misc sub-simulator. Appropriate instantiations and constants have been put in place to finish integrating HelperCivilian agents with the RCRSS.

The HC agent attribute skills is a 4-byte integer used as a bit vector [8]. Bit 0 (RESCUE) and Bit 1 (CLEAR) are reserved for the current advanced capabilities. There is ample room for expansion with other capabilities. Proper synchronization is required to ensure that each module in the distributed environment sees the attribute skill in an identical manner. The table below lists the values for the attribute along with the equivalent agent roles.

Value Clear Capability

Rescue Capability

Equivalent To

0 No No Civilian 1 No Yes Medical Tech 2 Yes No Police 3 Yes Yes Police / Medical Tech

With the introduction of attribute skills, it is possible to configure an agent’s capabilities at design time and modify them at run-time. Hence, a richer mix of agent behaviors is possible without the addition of new agent classes. Potentially, we can develop methods such as skill training, through which agents “learn” new skills from another agent at run-time.

4. EXPERIMENTAL DESIGN Our experiments contained 100 civilian agents (with none, R, C or RC capabilities) and 25 other agents (police, fire, and ambulance) in each simulation, as shown in Table 2. The scenarios differed in that there were varying percentages of the populations of (original) civilians (skills = 0) and enhanced civilians (skills > 0). All other environment variables remained constant. All enhanced civilians had the same skill(s) during each set of experiments with percentages of trained civilians ranging from 0% to 100%. Three sets of experiments were conducted, one with only Rescue (R) capabilities, one with only Clear (C) capabilities, and the third with both Rescue and Clear (R&C or RC) capabilities. The percentage of HC agents went from 0 – 100%, in increments of 20%. Agents were randomly selected for enhancement.

74

4

Figure 4. The enhanced world model.

Figure 5. The HelperCivilian Agent description.

75

5

Figure 6. YabAPI modifications for the enhanced model.

Each simulation ran for 300 simulation cycles, the standard length. The simulation environment and the computing platform are listed in Tables 2 and 3, respectively. The standard Kobe city map provided the environment.

Table 2. Simulation parameters Element Type Element Location Distribution

(N = Node, B = Building, R = Road)Road segments 820 Node 765 Building 730 AmbulanceCenter 1 PoliceOffice 1 FireStation 1 Refuge 7 Civilian 1 (N:1,B:0,R:0) HelperCivilian 100 (N:2,B:86,R:12) AmbulanceTeam 5 (N:5,B:0,R:0) FireBrigade 10 (N:10,B:0,R:0) PoliceForce 10 (N:9,B:0,R:1) FirePoint 4

Table 3. Computing platform

Processor Intel Celeron M 1.3Ghz Memory 512MB RAM OS Platform Redhat Linux 9.0 RCRSS Version 0.44

To evaluate our simulation performance, we used the evaluation rules from the 2003-05 competitions, as shown next [5]:

Evaluation Rule 2003-05: V = (N + HP) * SQRT (NB) Where, T = current simulation cycle (time) N = number of surviving agents HPini = total agent health point at time = 0 HPfin = total agent health point at time = T NBini = total area of non-burned buildings at time = 0 NBfin = total area of non-burned buildings at time = T HP = HPfin / HPini NB = NBfin / NBini

The total score (V) depends on the number of living agents (N), their health point ratio (HP), and the ratio of non-burned to total buildings (NB). The contribution from HP to V in the above Evaluation Rule is much less than the NB weight. As we are

yab.io.object yab.agent.object yab.agent

76

6

focusing on the health condition and evacuation efficiency of the population, and use the same fire-fighter agents and simulation conditions, we decided to look at the HP component separately. In order to analyze the different HP scores, the viewer console was altered to also print V, N, HP, and NB for each simulation cycle.

5. EXPERIMENTAL RESULTS Figures 7, 8, and 9 show the experimental results as measured by the official score (V), the number of surviving agents (N), and the agent health point ratio (HP), respectively. The results confirm our expectations. The population with both Rescue and Clear (RC) capabilities outperformed both the Rescue (R) only capability and the Clear (C) only capability agents. All enhanced agent populations (RC, C and R) outperformed the pure civilian populations with no extra capabilities (skills = 0).

0

10

20

30

40

50

60

70

80

0 20 40 60 80 100

Trained HC Population (%)

Off

icia

l Sco

re (

V)

at T

=30

0

Rescue

Clear

R&C

Figure 7. Score (V) versus percentage of trained population.

Figure 7, shows the results of simulations having 0% to 100% enhanced civilian agent populations (left to right). The 0% enhanced population score in each line is the same (as it should be), at approximately 30 .For the Rescue (R) and Clear (C) populations (diamond and square) individually, the total score increased from 30 to 45 with 100% capabilities. With both Rescue and Clear (R&C) capabilities in each agent, the final total score reaches 70. Thus, the enhanced agents made a significant improvement in the total score (V) over pure civilians, with a gain of between 50 and 130 percent.

0

20

40

60

80

100

120

0 20 40 60 80 100

Trained HC Population (%)

No.

of

Surv

ivin

g A

gent

s (N

) at

T=

300 Rescue

Clear

R&C

Figure 8. Number of surviving agents (N) versus percentage of

trained population.

Figure 8 shows the results using the number of surviving agents as the metric. At the start, each scenario had 125 agents, 100 civilian and 25 others (police, fire, and ambulance). The percentage of civilians with additional capabilities again ranged from 0% to 100% (left to right). With 100% enhanced Rescue (R) or Clear (C) civilians, the final N increased from approximately50 to 70. With 100% combined Rescue & Clear (R&C) civilians, the number of survivors increased from approximately 50 to 110 (out of a total of 125). This again shows a 40 % to 120% increase in agent survival when all of the 100 civilians have maximum capability.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 20 40 60 80 100

Trained HC Population (%)T

otal

Hea

lth P

oint

Rat

io (

HP)

at

T=

300 Rescue

Clear

R&C

Figure 9. Agent health point ratio (HP) versus percentage of

trained population.

Because the HP is not highly weighted in the Evaluation Rule, we calculated it individually. We wanted more detailed information about the results of the enhanced agents than V and N provide. The results are shown in Figure 9. In simulations with 100% enhanced civilians with either Rescue (R) or Clear (C) capabilities, the average Health Point Ratio increased from 30 to 40%. When the civilian population included 100% enhanced civilians, all having both Rescue and Clear (R & C) capabilities, the HP increased to approximately 65%. Thus the increased ability of the enhanced civilian agents resulted in an increase of up to a 115% as compared to the pure civilian population.

6. SUMMARY By creating a general and flexible agent model, we aimed to simulate situations where human agents could have flexible capabilities. As expected, the enhanced agents, in particular the agents with both Rescue and Clear capabilities clearly improve the results, by all measures examined. We found that when 100% of the civilians had Rescue and Clear capabilities, the official score (V), the number of survivors (N) and the overall agent health point (HP) values increased by 130%, 120%, and 115% respectively. We have demonstrated improved performance in simulations, the agents created are easily configured with any set of skills desired, a broader range of scenarios can be simulated, and the simulations can more closely reflect human behavior.

77

7

7. FUTURE WORK Our extended agent model allows new behaviors to be simulated and potentially supports adjustable autonomy and agent learning through skill training. We look forward to further improvements, including development of agent training and collaboration scenarios, expanding support for more capabilities in the HC model, and extending the representation to reflect multiple levels of mastery for one or more skills.

One weakness of the current structure of RCRSS code became apparent during this project. We needed to alter too many files, and often in similar ways to integrate the new agent model. If all simulators and sub-simulators could communicate with the human agents using shared code, the resulting system would be easier to develop and maintain.

8. ACKNOWLEDGMENTS This work was supported by the Electrical Engineering, and the Information and Computer Sciences Departments at the University of Hawaii. The authors wish to thank the faculty and graduate students for helpful conversations and providing encouragement during this project. REFERENCES [1] Koto, Tetsuhiko (2004). RoboCup Rescue Simulation

System Basic Package: Version 0.44. Retrieved April 2004, from http://www.notava.org/rescue/rescue-0_44-unix.tar.gz

[2] Kuwata, Y. (2001). LogViewer Source Code.

Retrieved May 2004, from http://homepage1.nifty.com/ morecat/Rescue/download.html

[3] Morimoto, Takeshi (2002). How to Develop a RoboCupRescue Agent, version 1.00. Retrieved Jan 2004, from http://ne.cs.uec.ac.jp/~morimoto /rescue/manual/manual-1_00.pdf

[4] Morimoto, Takeshi (2002). YabAPI: API to Develop a RoboCupRescue Agent in Java. Retrieved Jan 2004, from http://ne.cs.uec.ac.jp/~morimoto/rescue/yabapi

[5] RoboCupRescue Simulation League 2005 Web Site. RoboCup 2005 Rescue Simulation League Rules. Retrieved August 8, 2005, from http://kaspar.informatik.uni-freiburg.de/~rcr2005/ sources/rules2005.pdf

[6] RoboCup Rescue Web Site. Retrieved May 2004, from http://www.rescuesystem.org/robocuprescue

[7] Takahashi, Tomoichi (2001). Rescue Simulator Manual version 0.4. Chubu University. Retrieved Jan 2004, from http://sakura.meijo-u.ac.jp/ttakaHP/ Rescue_index.html

[8] U, Vengfai Raymond (2005). Enhancing Agent Capabilities in a Large Simulation System. Master Thesis. Dept. of Electrical Engineering, University of Hawaii, Dec. 2005.

[9] University “La Sapienza” (2002). JGISEdit - Multi-Platform Map Editor and Initial Conditions Setting Tool. Retrieved September 2004, from http:// www.dis.uniroma1.it/~rescue/common/JGISEdit.jar

78

Requirements to Agent based Disaster Simulations fromLocal Government Usages

Tomoichi TakahashiMeijo University

Shiogamaguchi, Tempaku, NAGOYA 468-8502, JAPAN

ttaka@ccmfs.meijo-u.ac.jp

ABSTRACTThe agent based approach has been accepted in various ar-eas and multi agent systems have been applied to variousfields. We are of the opinion that a multi agent system ap-proaches constitute one of the key technologies in disasterrescue simulations, since interactions with human activitiesshould be implemented within them. In joining RoboCuprescue community, we have recognized that rescue agents’behavior has been analyzed ad hoc and evaluated by em-ploying various standards. In this paper disaster simulationsystem and its components are discussed from local govern-ment usages. With RoboCup Rescue simulation data, wediscuss issues that disaster management system will havewhen it is applied for practical usages. These discussionwill delve into future research topics in developing practicaldisaster simulation systems.

1. INTRODUCTIONApproaches to use agent technology for simulating social

phenomena on computers are promising ones. Agent basedapproach has been accepted in various areas and multi agentsystem (MAS) has been studied in various fields [13][6]. Thepurposes are (1) to expand possibilities of MAS, (2) to sup-port modeling one social activity, (3) to use simulation re-sults in daily life, etc.

Disaster and rescue simulation is one of social simulations.We are thinking that multi agent system approach is one ofkey technologies in disaster rescue simulations, since inter-actions between human activities and disasters can be im-plemented in them. We have proposed RoboCup RescueSimulation System as a comprehensive rescue and disastersimulation system [7]. Not only rescue agents but also dis-aster simulations have been improved by doing RoboCupRescue simulation competitions every year [11]. And vari-ous related researches have been presented using the system.The application tasks or fields are different in structure andsize. Some of the agents’ abilities have domain-specific fea-tures while the other have task independent ones.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.AAMAS’06 May 8–12 2006, Hakodate, Hokkaido, Japan.Copyright 2006 ACM 1-59593-303-4/06/0005 ...$5.00.

A. Farinelli et al. proposed a project that uses RoboCupRescue to support real time rescue operations in disasters[5].They also tested the robustness of rescue agents by changingtheir sensing abilities. N. Schurr et al. have presented asystem to train fire office [10].

When MAS are applied to disaster related matters, namelyputting simulation results to practical use, there are manyissues that were not expected when it was designed. Thispaper discusses system requirement from viewpoints of lo-cal governments that will use the simulation for their ser-vices. The work is organized as follows; section 2 describessystem architecture of disaster rescue simulation using MASapproach, section 3 highlights factors on evaluations of simu-lation results, and section 4 discusses qualitative and quanti-tative evaluation using RoboCup Rescue Simulation results.Finally, we summarize open issues in practical applicationsand future research topics from agents based social system.

2. DISASTER SIMULATION SYSTEM

2.1 agent based disaster simulationIn scientific and engineering fields, a following step has

been repeated to further their advancement.

guess → compute consequence → compare experiment

Simulations have been used as experiments tools. They havesome advantages to physical experiments. The advantagesare that it does not cost time and money and simulationscan be repeated with changing parameters, etc. In disasters,we cannot do experiments physically on real scale and alsohardly involve human as one factor of the experiment. Hu-man behaviour ranges from rescue operations to evacuationactivities.

The behaviour has been analyzed as an area of social sci-ence. Agent-based simulation makes it possible to simulatedisasters, human actions and interactions between them.

2.2 usage as disaster management systemVarious kinds of disaster have been suffering and will suf-

fer us. Form 2003 to 2005, five earthquakes with more 1,000deaths were reported. Among them, there was the tsunamicaused by earthquake at Northern Sumatra. [1] HurricaneKatrina at 2005 September was also a disaster. Table 1shows disasters, disaster simulations and assumed usages ofsimulations.

It is hoped that disaster simulation can predict damagesto civil life. Fig. 1 shows systems from a simple agent todisaster management system,

79

Environment

agent

interacation

communiaction

interacation_1

agent_1

agent_n

Environment

interacation_2

communiaction

disaster simulation_m

disaster simulation_1

interacation_3

GIS

agent_1

agent_n

d. simulation_m

d. simulation_1

communiaction

agent_1

agent_n

human operations

interacation_2

interacation_1

Environment

GIS

Environment

(a) single agent (b) multi agents

DisaterRescue System

physical sensors

physical world

(d) disaster management system(c) disaster rescue simulation

Figure 1: MAS Architecture to Disaster Management System

1. basic agent model: An agent moves for one’s goals withintegration with environment. A typical task is that afire engine finds fires and extinguishes them as soon aspossible.

2. multi agent system model: Agents have common goals.They try to make it cooperatively such as fire enginesextinguish fires as a team. Sometimes, the fire enginesextinguish fires with help of fire offices. The latter casecorresponds to a heterogeneous agent system.

3. disaster environment modeling: Houses are burnt by fireand smoke from the houses decrease the sight of firefighters. Building collapse hurts civilians and blocksroads with debris. Disasters impact the environmentand agents. Disaster simulators are required to changetheir properties properly.

4. disaster management system: This corresponds to an im-age that disaster-rescue simulation is used at local gov-ernments emergency center. In such a system, datacomes from real world and results of the system willbe used to support local government’s rescue planning.

2.3 evaluation of disaster simulation resultsIt is required that simulation results are evaluated by com-

paring with other methodologies. It is important to makeclear followings so that simulations will be use as experi-ments in compare experiment in 2.1; (1) what targets ofthe simulations are, (2) under what conditions the simula-tions are done, (3) what computational models of simula-tions are, (4) there is some criteria that assures simulationresults , etc.

To clarify discussions on MAS, we present MAS systemas

S = {G,Ag, Σ, E ,Ac, C}

where G is a purpose of S. It is not necessary the same asaims of agents work.

In a case of disaster management system, G is a taskthat emergency centers try to solve at disasters. A rescueagent team is composed of fire brigade agents and ambu-lance agents. Ag is a set of agents who may have their owngoals such as fire engine agents extinguish fire and ambu-lance agents help injured and bring them to hospitals. It isdefined recursively as composite of heterogeneous agents asAg = {a | a is an agent, or a set of agents}.E is an environment where agents act. Road networks,skyscrapers, undergrounds or theaters are involved in E .Σ is a set of simulators or physical sensors. {s1, s2, . . . , sl}.Some of them are disaster simulators that change propertiesof the environment, E . Others simulate human conditionsand change injury of agents. The injuries lead to death attheir worst without any rescue. These health properties areprohibited to be changed by themself. Ac is a set of actionsor protocols that agents can use. Using them, agents cancommunicate with each other or interact with E .C represents communication channel among agents and be-tween E . Seeing is a channel to get information of E , hearingand speaking are a communication channel among agents.So voice, telephone or police radio are different channels.

3. PROBLEMS OF DISASTER MANAGE-MENT

3.1 system targets: G

It is assumed that disaster rescue simulation system isused at local governments. They plan so that people willlive safely at disasters. The usages are different betweenbefore and after disasters. Before disaster, they estimatedamages of their town and make disaster prevention plansfor expected disasters. They will use the disaster manage-ment system to confirm the plans will decrease damages tocivil lives. After disasters, emergency center will be set upto evacuate people, deliver relief goods, etc. Outputs of the

80

Table 1: Disasters, necessary simulation components and purposescomponents usage

disasters simulators data items to be evaluated time to used

natural earthquake fire GIS human lives before disastertsunami *smoke *analysistyphoon collapse facilities damages after disasterflood *building, mudslide * public property *planning

man-made terror human activity life lines * private property*traffics*evacuation

Table 2: Quantitative or qualitative standardquantitative factor qualitative factor

Ag number of agentssocial hierarchy (fire man, fire office)agents’ character (friendly, selfish, ..)act with (without) prior knowledge

Σdisaster (fire, collapse simulation,..)life line (traffic, communication, ..)

models and precision of each disaster

E area size of target resolution of GIS, 2, 3 dimensional undergroundmall

Acinteraction model with environments, interactionmodel among agents

commands, protocol communication band

C sensing powerpartial view, hearing of voice with errors (imper-fect world)

disaster rescue simulation are required to ones that are as-sured for making decisions.

The scheme will change into

plan →compute damages → compare past data| {z }

as a estimation of models

→ compare plans| {z }

usage as disaster management tool

.

While agent approaches are micro level simulations, localgovernments will use simulation results G in a macro scale.Namely how many people will be saved and how much dam-ages will be decreased rather than how good a rescue teamwill do at disaster are expected at local government.

3.2 computational model:Σ, E

Table 2 shows items of MAS’s components from quanti-tative and qualitative factors. In order to make simulationresults close to real one, it is necessary to improve each com-ponents in quantitative and qualitative ways.

Typical quantitative parameters are number or scale ofobjects. Take the number of agents as an example. Thenumber of agents is proportional to the number of processesor threads, communication among them costs resources onnetwork. In a similar way, GIS data for wider area requiresmore memory.

When earthquakes occur, people evacuate, call for rescues,confirm safety of families etc. Fire brigades go into action.Such human activities are also simulated. In evacuation,human move inside and outside houses, on foot or by cars.Traffic simulations support such human movements. Whenthe finer traffic model are used, the more data on road andthe complicated signal controls are required[4].

The improvements of components are issues of not onlyagent technology but also software technology.

3.3 human behavior model:Ag,Ac, C

At disasters, many kinds of agents are involved such as firepersons, police persons, victims, etc. Modeling the agentsand relationship among them are also one of qualitative is-sues. Characters involved in the simulation are representedas Ag. Ag is required to present not only their functionsbut also their organization.

Simulation of evacuation from burning theaters is a casethat agents of single kind are involved. When fire rescuescome in, they will become a heterogeneous agent system.The fire engines are given directions by the fire office, andthen there is a hierarchy among fire agents. And fire enginesand fire office have different functions. When implementingthese agents, their fundamental functions are inherited fromhuman behavior such as they can

• see only circumstances around them not the wholearea, hear and speak agents around them(Ac),

• communicate strange agents at distance via telephoneor radio transmission (C),

• follow instructions, however take emergency actionshastily in some cases.

So these situations specify framework of agent system as

• environment is dynamic and partial observable,

• input is imperfect and has errors,

• agents’ goals are partially satisfied,

• output is stochastic.

3.4 evaluation standard:F (S)

Simulation result F (S) are to be of help at emergencycenters. Disaster simulation results should be comparableto real data, and also be analyzed systematically. Data can

81

only get form past disasters cases. Simulation models havebeen improved so that the simulated results correspond withthe past data from quantitative or qualitative points. Mostof disaster simulations are modeled to match these data.

Human is involved as key components in disaster and res-cue simulation. Simulations involved human activities alsorequired to verify the results or the models with data orexperiments as well as.

4. OPEN DISCUSSIONS BASED ROBOCUPRESCUE SYSTEM’S DATA

With several examples, F (S) and other components arediscussed from local government usages.

4.1 competition ranking systemIn RoboCup Rescue Simulation league, following score for-

mula has been used in ranking teams [9].

V = (P +H

Hint) ×

r

B

Bmax(1)

where P is the number of living civilian agents, H is HP(healthpoint, how much stamina agents have) values of all agentsand B is the area of houses that are not burnt. Hint andBmax are values at start, the score decreases as disastersspread. Higher scores show that rescue agents operate bet-ter. Participants develop rescue agents to increase V .

Fig. 2 shows scores of semi-final games at RoboCup 2004.Six teams (from team A to F) out of 18 teams advanced tothe semifinals and performed rescue operations under dif-ferent disaster conditions and areas (from 1 to 6, in thehorizontal axis). Vertical scales on the figure are V s thatare normalized with the high score.

Table 3 shows three teams’ score at the same map withdifferent sensing conditions. The simulation results at col-umn r1 are those where the visual ability of agents was setat half the normal visual ability s. In column r2, the hearingabilities of agents were set half of s. Performances in col-umn r3 are those set with both visual and hearing abilitiesat half.

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6

ABCDEF

Figure 2: Scores at RoboCup 2004 games

disscusion 1: The three teams were developed by differ-ent universities. Their search, rescue and evacuation tacticsare different. Fig. 2 indicate that one performance at one

Table 3: Changes of rescue performances for sensingabilities

sensing conditionnormal(s) r1 r2 r3

team X 78.92 78.92 79.92 78.91team Y 97.69 35.41 83.49 90.87team Z 88.24 83.30 51.45 45.76

disaster situation does not guarantee performances at oth-ers. Table 3 shows team X is a robust team, while otherteams are sensitive for changes in sensing abilities.

Which feature, robust or efficient, is positive for rescueagents? What standard is used to evaluate agents’ perfor-mance and simulation results as consequence of them?

4.2 task allocation problemsOtha et al. showed cooperation among fire brigade agents

improve their efficiency and assigning fire brigades to ig-nition points properly reduce total damages under simpleconditions[8]. Kobe (1/100) map (numbers of edges, nodes& buildings are 125, 119 and 99 respectively.) was used intheir experiments.

discussion 2: Fire offices have enough fire engines toextinguish fires at normal times. The target of rescue plan-ning is how fast and efficiently they can extinguish them. Atdisasters, situations are different. Fires break out simultane-ously at some places. Consequently fire engines’ powers areinadequate to extinguish all fires. They change their targetsto protecting houses from fires from extinguishing fires 1 .

1. In cases of disasters that fire department changes theirpolicy of rescue, robustness in agent seems to be re-quired rather efficiency. Is it calculable to evaluateagent’s ability under the same standards ?

2. From decision supporting view points, it is one way tosimulate disasters with different parameters and com-pare the rescue activities. How simulation results willbe verified ?

4.3 simulation results comparisonTable 4 showed burned rates for sixteen wards in Nagoya

city where our university is. GIS network column shows thenumber of nodes and edges of maps that used in simulation.The network data with 1:25.000 scale is available as opendata.[2] These networks are bigger than maps contained inRoboCup Rescue Simulation Packages.

column A : Data in column A are cited from report byNagoya City Fire Bureau. The data are ones estimatedby a macro model based on the past fires. The valuesare ratio of estimated burnt out houses for an earth-quakes with magnitude 7 to 8.build.: the number of wooden houses in each ward,ig.p: the number of expected ignition points,burn: burned houses rate without fire rescues,burn F: burned houses rate with fire rescues.They used the same macro fire model to calculateburn: and burn F, the difference is the number of ig-nition points by fire fighting at the initial stage.

1from interview at Tokyo fire department

82

Table 4: Simulation results and target GIS dataGIS network A B C

ward node edge build. ig.p burn burn F build. burn F build. ig.p burn burn FChikusa 5,581 3,711 32,156 0 0.0% 0.0% 1,692 63% 9,924 7 2.05% 1.74%Higashi 2,420 1,690 14,761 1 0.1% 0.1% 757 76% 4,283 3 2.09% 1.38%

Kita 6,069 3,870 39,302 22 3.9% 3.4% 1,651 31% 9,541 7 1.51% 0.99%Nishi 6,430 4,122 44,773 58 5.8% 4.9% 1,419 71% 10,468 7 1.97% 1.74%

Nakamura 6,044 3,766 41,769 45 5.1% 4.5% 1,431 61% 8,994 6 2.16% 1.15%Naka 2,026 2,093 18,726 5 0.9% 0.5% 905 95% 5,396 4 2.03% 1.12%

Showa 3,795 2,456 28,464 0 0.0% 0.0% 1,186 84% 6,325 4 0.78% 0.58%Mizuho 4,053 2,563 30,092 2 0.5% 0.1% 1,062 94% 6,656 4 0.65% 0.47%Atsuta 2,609 1,760 17,580 3 1.3% 1.0% 641 90% 4,309 3 4.32% 1.42%

Nakagawa 9,449 6,154 58,612 31 2.6% 1.7% 1,952 39% 17,327 13 0.93% 0.87%Minato 7,127 4,892 38,694 0 0.0% 0.0% 1,378 35% 15,269 18 1.32% 1.24%Minami 5,718 3,710 43,318 1 0.0% 0.0% 1,404 39% 10,157 7 2.11% 1.71%

Moriyama 6,651 4,413 39,821 0 0.0% 0.0% 1,422 36% 13,077 13 1.80% 1.22%Midori 8,945 5,996 53,445 0 0.0% 0.0% 1,831 23% 18,301 15 1.11% 1.06%Meito 5,612 3,724 27,554 0 0.0% 0.0% 1,556 46% 10,740 8 2.27% 1.66%

Tenpaku 5,986 3,951 29,584 0 0.0% 0.0% 1,553 27% 11,259 9 2.03% 1.79%correlation between the number of buildings in A 0.83 0.85

column B & C : Data in column B and C are results ofRoboCup Rescue Simulations (Ver.46). Housing dataare personal properties so they are not available asopen data like road network. They are generated byprograms under conditions that the number of gen-erated houses is proportional to real numbers. Datain column B data is used to show correlation betweensimulation results and environment’s change[12].

Difference between column B and C are(1) Scale of map in C is real and B is 30:1.(Fig.3)(2) The number of ignition points is 5 for all maps incolumn B, while the numbers are set to proportional toareas in column C. It may be natural to use the samenumber as column A, however, no fire is supposed athalf of ward.

There are correlation between map sizes and simulation re-sults and also correlation between macro level data (A) andmicro level simulation results (B & C).

discussion 3: Three sets of data are presented. Whichone does local government use as assured ones? In otherwords, do they feel it sufficient to use simulations results toshow setting open spaces such as parks in built-up areas areuseful to prevent fires?

1. E in column C are ones more real than column B.

Does it mean values in column C are assured ones?

2. Damages in column C are less than ones in column Band the order of values become close to ones in columnA. Followings are considered to be causes;(1) Spaces between building become wider.(2) Fire engines takes more time to go to fire points.These are instinctively understood. Does it indicatethat fire simulation and traffic simulators are well-designed ones?

4.4 effect of disaster simulatorsSimulations results are combination of disaster simula-

tion and agent actions. RoboCup Rescue simulation pack-age had used a fire simulator that was designed based on

Figure 3: Rescue simulation of Chikusa ward(above:Column B, below:Column C, They are simi-lar figures, scales are different. )

KobeAawaji disasters till version 0.46. Form version 0.47,they have used a newly developed fire simulator. It is basedon physical model of heat development and heat translation[3]. The new fire simulators can simulate preemptive coolingthat will protect buildings from catching fires, however, ithas not verified with real data.

discussion 4: Two fire simulators output different val-

83

i) Nishi & Nakamura wards

ii) composite map

iii) clipped one

Figure 4: Flow mage generation from open data to simulation target map.

ues. And introducing physical model makes simulators mul-tifunctional ones. Setting simulation components more real,fine ones require more computational resources.

Is there appropriate resolutions for applying simulationresults to decision supports ?

4.5 effect of disaster areaDisasters occur beyond the confines of local governments’

administrative districts. Fig.4 shows an outline of suchcases. The left figure shows two awards, Nishi and Naka-mura in Table 4. The middle map is a map that is combinedwith two maps. The right one is middle-down part wherehouses are built densely.

discussion 5: Table 5 shows simulation results have cor-relations with Nagoya fire Bureau’s data. The number ofignition points are set to the number of column A. And sim-ulation results at composite maps show similar trend.

Damages are severe at the clipped area and rescue oper-ations are done at this area. Is it sufficient area for simula-tions? It seems reasonable that simulations are done underconditions that agents have knowledge beforehand. Howwell do they know?

Table 5: Ignition points set eqaul to estimatedEarthquake

ward No. ignitions no fire brigade fire brigade

Nichi 30 (night) 8.53% 8.08%58 (day) 13.40% 12.96%

Nakamura 22 (night) 8.90% 8.45%45 (day) 15.64% 15.23%

correlation with data* 0.89 0.92Clipped area 7.20% 7.14%

*: from same report used in column A (Table.4)

5. SUMMARYSince we proposed RoboCup rescue system as one of agent

based social simulations, we have asked from research sideand practical aspects; (1) what specific and different pointsdo they have in agent research themes, (2) what points dothey serve as practical tools.

In future, MAS will deal with more agents and wider areasimulations. It will provide tools to do experiments thatwould otherwise be impossible. They should be tested withreal data, and the simulation results should also be ana-lyzed systematically. A universal method to evaluate agentswill be one of the key issues in applying agent approachesto social tasks. We have no practical evaluation methodsfor social agents so far. The social activities are composedof various kinds of tasks and their evaluations depend onseveral task dependent parameters.

In this paper disaster simulation system and its compo-nents are discussed from local government usages. WithRoboCup Rescue simulation data, we discuss some of issuesthat disaster management will have in applying practical us-ages. These discussions will delve into future research topicsin developing disaster estimation to a practical one.

6. ACKNOWLEDGMENTSThe authors wish to express their appreciation to the

RoboCup Rescue community that provides fine software en-vironments and the organization that provides GIS data.

7. REFERENCES[1] http://neic.usgs.gov/neis/eqlists/eqsmajr.html.

[2] http://zgate.gsi.go.jp/ch/jmp20/jmp20 eng.html.

[3] http://kaspar.informatik.uni-freiburg.de/ nuessle/.

[4] J. L. Casti. Would-Be Worlds: How Simulation isChanging the Frontiers of Science. John Wiley andSons, 1997.

84

[5] A. Farinelli, G. Grisetti, L. Iocchi, S. L. Cascio, andD. Nardi. Robocup rescue simulation: Methodologiestools and evaluation for practical applications. InRoboCup Symposium, 2003.

[6] N. R. Jennings and S. Bussmann. Agent-based controlsystems. IEEE Control Systems Magazine, 23(3):61–74, 2003.

[7] H. Kitano, S. Tadokoro, I. Noda, H. Matsubara,T. Takahashi, A. Shinjou, and S. Shimada. Robocuprescue: Search and rescue in large-scale disasters as adomain for autonomous agents research. In IEEEInternational Conference on System, Man, andCybernetics, 1999.

[8] M. Ohta, T. Koto, I. Takeuchi, T. Takahashi, andH. Kitano. Design and implementation of the kerneland agents for robocup-rescue. In Proc. ICMAS2000,pages 423–424, 2000.

[9] RoboCup2004.http://robot.cmpe.boun.edu.tr/rescue2004/.

[10] N. Schurr, J.Marecki, N. Kasinadhuni, M. Tambe,J.P.Lewis, and P.Scerri. The defacto system for humanomnipresence to coordinate agent teams: The futureof disaster response. In AAMAS 2005, pages1229–1230, 2005.

[11] C. Skinner and M. Barley. Robocup rescue simulationcompetition: Status report. In Int. SymposiumRoboCup, 2005.

[12] T. Takahashi and N. Ito. Preliminary study to userescue simulation as check soft of urban’s disasters. InWorkshop: Safety and Security in MAS (SASEMAS)at AAMAS05, pages 102–106, 2005.

[13] G. Weiss. Multiagent Systems. The MIT Press, 2000.

85

Planning for Bidding in Single Item Auctions

M. Utku TatlıdedeBogazici UniversityPK 34342 Bebek

Istanbul, TURKIYE

tatlided@boun.edu.tr

H. Levent AkınBogazici UniversityPK 34342 Bebek

Istanbul, TURKIYE

akin@boun.edu.tr

ABSTRACTMarket based systems which use single bid auction usuallysuffer from the local minima problem. In many of the multi-agent problem domains, acting greedily in each step is notsufficient to solve this problem in an optimal way. Thereare alternatives such as combinatorial auctions, clusteringof tasks and task decomposition but all have their disadvan-tages. We propose that by taking a simple plan into accountwhile bidding in auctions, the agent is capable of exchang-ing multiple items in single item auctions. The proposedapproach is tested against two common market based algo-rithms in robotic exploration task. The tests are held in asimulator environment that models a grid world. It is shownthat, our approach increases the performance of the system.Due to its generality, this approach can readily be adoptedto the disaster management domain.

1. INTRODUCTIONMulti-agent systems have gained importance and have beenimplemented in many fields during the last decades sincethey are more reliable and fault tolerant due to the elimi-nation of single point of failure and faster due to parallelism.Among many coordination paradigms proposed, market basedcoordination is a promising technique which is well suitedto the requirements of the multi-agent systems [4].

Although multi-agent systems have been developed for solv-ing different types of problems, these problems share somecommon characteristics. Gerkey and Mataric [6] developeda taxonomy based on three attributes of the problem def-inition. Here after we will refer to this taxonomy as GMtaxonomy. First, the problem is categorized as either sin-gle robot (SR) or multi robot (MR) depending on whetherthe task can be achieved by one or more robots. The nextcategorization is done regarding whether a robot is capableof only a single task (ST)or more (MT). Finally, the prob-lem is categorized as instantaneous (IA) if all the tasks areknown by the agents initially. The counterpart definition ofthe assignment property is time extended (TA) which de-

ATDM ’06 Hakodate, JAPAN

scribes the problems where the tasks are discovered duringthe course of action. These definitions help to define prob-lems in the multi-agent domain.

Among many other applications of multi-robot systems, thedisaster management domain is special, since it provides nu-merous challenges. The main purpose of Robocup RescueSimulation League in Robocup [9] is to provide a researchplatform for emergency decision support by integration ofdisaster information, prediction and planning. The simu-lation covers the immediate aftermath of an earthquake inwhich the buildings have collapsed, fires have started dueto gas explosions, roads are blocked by debris and civiliansare injured and buried in buildings. During the initializa-tion of the simulation, the map of city is sent to the agents.The deliberative rescue agents should coordinate for taskssuch as finding civilians, opening roads, putting out the firesand rescuing the civilians before their health points reach tozero. In the RoboCup Rescue Simulation league the civilianfinding task is an instantaneous assignment exploration tasksince all the buildings are known at startup.

Among many coordination paradigms, the market-drivenapproach has gained popularity. The idea of a market-drivenmethod for multi-robot teams is based on the interactionof the robots among themselves in a distributed fashion fortrading work, power - information and hence providing ”Col-laboration by competition-cooperation”. In general, thereis an overall goal of the team (i.e., building the map of anunknown planet, harvesting an agricultural area, sweepingburied land mines in a particular area, etc...). Some entityoutside of the team is assumed to offer a payoff for that goal.The overall goal of the system is decomposed into smallertasks and an auction is performed for each of these tasks. Ineach auction, participant robots (which are able to commu-nicate among themselves) calculate their estimated cost foraccomplishing that task and offer a price to the auctioneer.At the end of the auction, the bidder with the lowest offeredprice will be given the right to execute the task and receivesits revenue on behalf of the auctioneer.

There are many different possible actions that can be taken.A robot may open another auction for selling a task that itwon from an auction, two or more robots may cooperativelywork and get a task which is hard to accomplish by a singlerobot, or, for a heterogeneous system, robots with differentsensors/actuators may cooperate by resource sharing (forexample, a small robot with a camera may guide a large

86

Figure 1: RoboCup Rescue Simulation League, Kobe Map.

Figure 2: Market scenario.

robot without a vision system for carrying a heavy load).The main goal in the free-markets is to maximize the overallprofit of the system. If each participant in the market triesto maximize its profit, as a result of this, the overall profitfor the system is expected to increase.

The general structure of the process would be better under-stood by referring to the following scenario: Suppose thereare two robots and two tasks in the environment. The costsof tasks calculated by the robots are as in Figure 2. So ro-bot 1 would take task 1 and robot 2 would take task 2, andimplement them. This would cost 50 for robot 1 and 75 forrobot, totally 125 for the team. But suppose robot 2 hasmore processing power and calculates that if robot 1 takesboth tasks, this would cost the team only 70, and the restof cost, 55 units would be leaved as profit, so it could offerrobot 1, task 2 and share some of the profit with it. So bothindividuals would gain more profit, and the job is still moreprofitable for the whole team.

2. RELATED WORKOne of the well known applications of multi-agent systemsis exploration which is applicable to many areas due to its

generality. The most popular problems are exploration ofplanets (e.g. Mars) and finding civilians in a territory. Themodel of exploration problem is SR-ST according to GMtaxonomy since the robots can move and explore withoutany help. Task assignment can either be IA or TA dependingon the domain or problem setup.

Centralized solutions do not satisfy the communication androbustness requirements. All agents communicate with thecenter which introduces the single point of failure problem;moreover if the communication with the center is lost or isnoisy, the performance degrades sharply even to non func-tioning level. The exploration task must be completed inany condition even only one robot survives. The majorityof the research is based on single item auctions [10] [5]. In[10] initially, all targets are unallocated. The robots bid onall unallocated targets. The bid for each target is the dif-ference between the total cost for visiting the new targetand all targets and the total cost for visiting only the tar-gets are already allocated to the robot. These total costsare computed using a TSP insertion heuristic. The robotwith the overall lowest bid is allocated the target of that bidand then is no longer allowed to bid. The auction contin-ues with the remaining robots and all unallocated targets.After every robot has won one target, all robots are againallowed to bid, and the procedure repeats until all targetshave been allocated. Finally, single targets are transferredfrom one robot to another, starting with the target trans-fer that decreases the total cost the most, until no targettransfer decreases the total cost any longer.

However single item auctions are not sufficient for ensuringsystem optimality. Single item exchanges between robotswith or without money lead to poor and suboptimal solu-tions in some but apparently possible cases. A simple sce-nario is depicted in Figure 3. The robots auction for thetasks which costs the lowest for them. Therefore after the

87

Figure 3: A task allocation scenario.

first auction round R1 adds B1 to its task list and R2 addsB3 to its task list. In the second round R1 will add B2 be-cause it becomes the best alternative for the task. After allthe targets are finished B2 will not be exchanged because itis best handled after B1. The auction rounds are finishedwhen all the agents are assigned to tasks. But after thatsingle exchanges cannot remedy the problem. The solutionis allowing some of the agents not to be assigned to any taskas our work suggests.

Golfarelli et al [7] clusters the tasks and assigns them toagents. Money is not defined in the system so the agentsare only allowed to swap tasks to increase the system perfor-mance. Although an increase in the performance is achieved,the system is not optimal. However, historical record showsus that if people could have met all their needs by barter,money would not have been invented.

In combinatorial auctions the targets are auctioned in com-binations so as to minimize the cost by grouping the targetsand allocating in an efficient way. One of the implemen-tations [1] uses GRAPHCUT algorithm to clear auctions.The major drawback of the combinatorial auction methodis its time complexity. In [2] the GRAPHCUT is outper-formed by an algorithm called PrimAllocation. Althoughthe main idea was to demonstrate the Prim allocation theyrepresent another one which is called Insertionallocation.In Prim allocation each robot only submits its best (lowest)bid to the auctioneer, since no other bid has any possibilityof success at the current round. The auctioneer collects thebids and allocates only one target to the robot that sub-mitted the lowest bid over all robots and all targets. Thewinning robot and the robots that placed their bid on theallocated target are notified and are asked to resubmit bidsgiven the remaining targets. The bids of all other robotsremain unchanged. The auction is repeated with the newbids, and so on, until all targets have been allocated. Theinsertion allocation is same as prim allocation except thatthe agent generates a path to its selected target and bidaccordingly. The results of this work show that both primand insertion allocation is better than combinatorial auctionimplemented by GRAPHCUT .

Another approach which is in fact a combination of sin-gle auctions and combinatorial auctions is clustering [3] andtask decomposition techniques [11]. In Dias et al’s work [3],the tasks are clustered and traded as clusters therefore par-tially eliminating the inefficiencies of single item exchanges.The size of the clusters is the main problem and even theclusters are traded. The approach only removes the intra-

Figure 4: Deadlock during task allocation.

cluster inefficiencies however the inter-cluster trading is stillachieved as single bid auctions. Dias further defines oppor-tunistic centralization where a leader role coordinates a teamin a centralized fashion in order to increase the system’s per-formance. But this approach is limited by communicationquality. The task decomposition and clustering have a com-mon point that both try to trade more than one task at atime. Zlot et al [11] decompose a task into subtasks as anAND-OR tree and any branch can be traded at any level.The decomposition’s communication requirements are highand there is no general way to decompose tasks.

3. PROPOSED APPROACHMarket based systems which use single bid auction usuallysuffer from the local minima problem. In many of the multi-agent problem domains, acting greedily in each step is notsufficient to solve the problem in an optimal way. Thereare alternatives such as combinatorial auctions, clusteringof tasks and task decomposition but all have their disad-vantages as mentioned in the previous section. We proposethat by taking a simple plan into account while bidding inauctions, the agent will be capable of exchanging multipleitems in single item auctions.

The proposed approach is implemented in multi-robot explo-ration task. The agent simply plans a route that covers allthe known targets by using the simple TSP insertion heuris-tic. Each agent auctions for the lowest cost target which isin fact the closest target to the agent. Other agents bid inthe auction according to their plan cost. The plan cost isthe distance between the agent and target if the target is theclosest target or the distance to the previous target in theplan. For example, In Figure 3 R1 constructs the path andits cost as B1:6, B2:4 and B3:8, in total 18. R1 constructsthe path and its cost as B3:4, B1:4 and B2:4, in total 12.Both agents start auctions for targets which are closest tothem, B1 and B3 respectively. R1 looses the auction be-cause the R2 bids 4 for B1 whereas R1 bids 6. B1 stays forthis time step. R2 wins the auction because it bids 4 for B3and R1 bids 8. Apparently to solve the problem optimallywe need to stop R1 if time is not our first priority.

Since the robots are set idle by the algorithm to minimize re-sources, deadlocks occurs when agents’ plans overlap. Thissituation is very common but can be detected and solved ina way to increase optimality. In Figure 4 R1 constructs itsroute as B1:4, B2:2 and B3:2 whereas R2 ’s route is B3:3,B2:2 and B1:2. Neither of the robots win the auction be-cause both bid better prices for the initial targets. This sit-uation is called a deadlock in which agents will stay forever

88

if not handled. In this case the agents detect the deadlockand calculate the total path cost to solve the deadlock situ-ation. R1 ’s plan cost is 8 and R2 ’s plan cost is 7 thereforeR2 wins all the tasks. If the plan costs are also equal thenthe agent with the smallest id wins the auction. The pseudocode for the algorithm is given in Figure 5.

1. check whether the target is reached

2. plan current tasks

3. bid for the lowest cost item

4. if auction won allocate task

5. else

(a) if deadlock detected solve according to totalplan cost

(b) else stay

Figure 5: Market plan algorithm pseudo code

In multi-agent systems, robustness is a very important issuefor most of the problem domains. Robot and communicationfailures commonly occur in physical implementations. Ourapproach handles such failures in a straightforward mannersince a robot always performs its best to cover all tasks.There is no need to track other robot failures because allthe tasks remain in the task queue until it is announced tobe finished. All the tasks will be finished even with only onerobot and no communication.

4. EXPERIMENTS AND RESULTSIn the experiments, in addition to the proposed market planalgorithm, simple market and market with re-planning istested. The two algorithms are described in below sections.

4.1 MarketThe algorithm is the simplest version of the single bid auc-tion. Agents auction the targets and the winner is not al-lowed to participate in any other auction unless it finishesthe assigned task. The bid is the distance between the agentand the target.

1. check whether the target is reached

2. if robot is idle

(a) bid for the lowest cost item

(b) if auction won allocate task

(c) else remove task from the list and re-auction

3. else continue execution of the allocated task

Figure 6: Market algorithm pseudo code

4.2 Market Re-planningRe-planning is a vital issue for all market based coordina-tion algorithms that should be implemented especially for

dynamic environments. The restriction of the market algo-rithm is that the agent can not participate in any other auc-tion until it finishes the assigned task. Re-planning removessome of the inefficiencies in the market algorithm becauseit allows the agent to act more greedily in the TA case. Ineach step, the agent auctions for the lowest cost target.

(a) check whether the target is reached

(b) bid for the lowest cost item

(c) if auction won allocate task

(d) else re-auction auction for other tasks

Figure 7: Market re-planning algorithm pseudo code

4.3 Experimental SetupThe test environment is specially developed for testing agentinteraction in grid worlds. It is implemented in JAVATMandcurrently prints the cell occupancies for each time step. Themessaging subsystem supports only broadcast messages andis implemented by a common message queue in the simula-tor. Th three algorithms are tested for 1000 times for in-stantaneous (IA) and time extended (TA) task assignmentproblems in the test environment. In each run robots andthe targets are placed randomly in the map. During initial-ization, the objects are not allowed to be in already occupiedcells. Randomness is needed for testing the algorithms indifferent scenarios. However, since the runs are random thismay yield biased results according to the unfair distributionof hard and easy problems. Therefore we implemented apseudo random number generator [8] that is fair enough togenerate randomized problems but always initiate the sameproblems for the robots.

4.4 ResultsThe results for a 10x10 grid world environment with twoagents and ten targets are given in Tables 1 and 2 bothfor the IA and TA cases respectively. Total time, costs forthe individual robots and total cost parameters are collectedand their average and standard deviation are reported. TheIA case is easier than the TA case because the tasks areinitially known so as to give the agents chance to constructnear optimal paths. In the IA case, market (M) and mar-ket with re-planning (M-R) algorithms perform almost thesame because their only difference is whether re-planning isenabled or not. The market with plan algorithm (M-P) isthe best in terms of cost because it uses a task allocationplan in order to correctly bid in the auctions. The total taskcompletion time is increased since the algorithm halts one ofthe agents to decrease system resource usage. This behav-ior is normal and functions as desired. In the TA case thetargets are randomly added to the world for every simula-tion second. Time extended assignment of tasks makes theproblem harder because the agent uses incomplete and sub-ject to change world knowledge in the auctions. The marketwith re-planning algorithm is better than the market algo-rithm because it can greedily allocate new coming low costtasks whereas the market algorithm completes its currentcontract before auctioning any other task. The market withplan algorithm is again the best performer because of its

89

Figure 8: a)Market algorithm runs on the sample scenario. R1=18, R2=15 and Total=33 cost. b)MarketRe-planning algorithm runs on the sample scenario. R1=20, R2=12 and Total=32 cost. c)Market planningalgorithm runs on the sample scenario. R1=10, R2=15 and Total=27 cost.

better bidding schema and re-planning ability which is vitalfor TA tasks.

Table 1: Results for the Instantaneous Task Assign-ment(IA) Case

Algorithm TimeCosts

Robot1 Robot2 TotalM 19.78±3.53 15.38±4.71 15.10±4.72 30.47±4.54

M-R 19.75±3.54 15.31±4.73 15.09±4.77 30.39±4.51M-P 27.19±5.00 17.80±9.60 11.99±9.50 29.79±3.99

Table 2: Results for the Time Extended Task As-signment(TA) Case

Algorithm TimeCosts

Robot1 Robot2 TotalM 24.67±3.60 19.03±5.01 18.72±4.98 37.75±5.19

M-R 23.90±3.77 18.43±5.17 17.67±4.86 36.10±5.17M-P 28.24±5.63 21.15±8.00 14.53±7.84 35.68±5.25

The actions taken by the agents according to market, marketre-plan and market plan in an instance of IA task are de-picted in Figure 8. The behavior of the robots are almost thesame for the market and market re-plan algorithms, howeverin the market with plan algorithm the targets are very wellshared by the robots to effectively explore all the targets ina cost efficient way.

5. CONCLUSIONWe developed a new algorithm to enable multi-item ex-change in a single item auction. In contrast to other ap-proaches, the proposed approach is domain and task inde-pendent therefore can be used in every domain. For exam-ple, in a heterogeneous task environment our approach canstill work effectively by bidding according to the plan. How-ever, clustering or decomposing heterogeneous tasks can notbe achieved easily whereas every agent has a time ordering oftasks internally. The agents must coordinate different typesof actions and can achieve it by making plans that take ad-vantage of different tasks available for them. The disastermanagement domain is the primary target for us because ofits complexity and social value.

6. FUTURE WORKThe results presented in this work is limited to two robotsand ten targets which are assigned instantaneously or ina time extended way. Unfortunately the optimal solutionsare not presented. Due to the simplicity of the problem(ceiling effect) the results are very close. The main purposeof this work is implementing the approach in a heterogeneousproblem as SR-MT or MR-MT. Agent and communicationfailures are not considered in the test setups. In near futurewe plan to achieve all targets in the grid world simulationsas the test domain and RoboCup Rescue Simulation as theapplication domain.

7. REFERENCES[1] M. Berhault, H. Huang, P. Keskinocak, S. Koenig,

W. Elmaghraby, P. Griffin, and A. Kleywegt. Robotexploration with combinatorial auctions. InProceedings of the IEEE/RSJ InternationalConference on Intelligent Robots and Systems, pages1957–1962. IEEE, 2003.

[2] M. Berhault, M. Lagoudakis, P. Keskinocak,A. Kleywegt, and S. Koenig. Auctions withperformance guarantees for multi-robot taskallocation. In Proceedings of the IEEE/RSJInternational Conference on Intelligent Robots andSystems. IEEE, September 2004.

[3] M. B. Dias and A. T. Stentz. Opportunisticoptimization for market-based multirobot control. InIROS 2002, page 27142720, September 2002.

[4] M. B. Dias, R. M. Zlot, N. Kalra, and A. T. Stentz.Market-based multirobot coordination: A survey andanalysis. Technical Report CMU-RI-TR-05-13,Robotics Institute, Carnegie Mellon University,Pittsburgh, PA, April 2005.

[5] B. Gerkey and M. Mataric. Sold! auction methods formulti-robot coordination. IEEE Transactions onRobotics and Automation, 18(5):758–768, 2002.

90

[6] B. Gerkey and M. Mataric. A formal analysis andtaxonomy of task allocation in multi-robot systems.International Journal of Robotic Research,23(9):939–954, 2004.

[7] M. Golfarelli, D. Maio, and S. Rizzi. Market-drivenmultirobot exploration. In Proceedings of the UKPlanning and Scheduling SIG Workshop, pages 69–82,1997.

[8] R. Hamming. Mathematical methods in large-scalecomputing units. Mathematical Rev., 13(1):495, 1952.

[9] T. Takahashi, S. Tadokoro, M. Ohta and N. Ito. AgentBased Approach in Disaster Rescue Simulation - fromTest-bed of Multiagent System to PracticalApplication Fifth International Workshop onRoboCup, 2001.

[10] R. Zlot, A. Stentz, B. Dias, and S. Thayer. A freemarket architecture for distributed control of amultirobot system. In Proceedings of the InternationalConference on Intelligent Autonomous Systems, pages115–122. IEEE, 2000.

[11] R. M. Zlot and A. T. Stentz. Complex task allocationfor multiple robots. In Proceedings of the InternationalConference on Robotics and Automation. IEEE, April2005.

91

Section 3

Agent-Based Simulation (Tools and Experiments)

92

Cooperating Robots for Search and Rescue Jijun Wang

School of Information Sciences University of Pittsburgh 136 N. Bellefield Ave. Pittsburgh, PA 15260

412-624-9426 Jiw1+@pitt.edu

Michael Lewis School of Information Sciences

University of Pittsburgh 136 N. Bellefield Ave. Pittsburgh, PA 15260

412-624-9426 ml@sis.pitt.edu

Paul Scerri Robotics Institute

Carnegie Mellon University 5000 Forbes Ave.

Pittsburgh, PA 15213 (412) 268-2145

pscerri@cs.cmu.edu

ABSTRACT Many hypothesized applications of mobile robotics require multiple robots. Multiple robots substantially increase the complexity of the operator’s task because attention must be continually shifted among robots. One approach to increasing human capacity for control is to remove the independence among robots by allowing them to cooperate. This paper presents an initial experiment using multiagent teamwork proxies to help control robots performing a search and rescue task. .

Categories and Subject Descriptors D J.7 : Computers in Other Systems

General Terms Multiagent Systems, Experimentation, Human Factors

Keywords Multiagent Systems, Multirobot Systems, Human-Robot Interaction.

1. INTRODUCTION Many hypothesized applications of mobile robotics require multiple robots. Envisioned applications such as interplanetary construction [4] or cooperating uninhabited aerial vehicles [8] will require close coordination and control between human operator(s) and cooperating teams of robots in uncertain environments. Multiple robots substantially increase the complexity of the operator’s task because she must continually shift attention among robots under her control, maintain situation awareness for both the team and individual robots, and exert control over a complex system. In the simplest case an operator controls multiple independent robots interacting with each as needed. Control performance at this task has been investigated both in terms of average demand on human attention [1] and for simultaneous demands from multiple robots that lead to bottlenecks [5]. In these approaches increasing robot autonomy allows robots to be neglected for longer periods of time making it

possible for a single operator to control more robots. Providing additional autonomy by enabling robots to cooperate among themselves extends automation to human control activities previously needed to coordinate the robots’ actions. Automating this function should decrease the demands on the human operator to the extent that attention being devoted to a robot involved coordination with other robots. If substantial efforts were required for coordination automation should allow improvements in performance or control of larger teams.

1.1 Teamwork Algorithm

The teamwork algorithms used to coordinate the simulated robots are general algorithms that have been shown to be effective in a range of domains [10]. To take advantage of this generality, the emerging standard approach is to encapsulate the algorithms in a reusable software proxy. Each team member has a proxy with which it works closely, while the proxies work together to implement the teamwork. The current version of the proxies is called Machinetta [8] and extends the successful Teamcore proxies [7]. Machinetta is implemented in Java and is freely available on the web. Notice that the concept of a reusable proxy differs from many other ``multiagent toolkits'' in that it provides the coordination algorithms, e.g., algorithms for allocating tasks, as opposed to the infrastructure, e.g., APIs for reliable communication. The Machinetta software consists of five main modules, three of which are domain independent and two of which are tailored for specific domains. The three domain independent modules are for coordination reasoning, maintaining local beliefs (state) and adjustable autonomy. The domain specific modules are for communication between proxies and communication between a proxy and a team member. The modules interact with each other only via the local state with a blackboard design and are designed to be ``plug and play'', thus, e.g., new adjustable autonomy algorithms can be used with existing coordination algorithms. The coordination reasoning is responsible for reasoning about interactions with other proxies, thus implementing the coordination algorithms. The adjustable autonomy algorithms reason about the interaction with the team member, providing the possibility for the team member to make any coordination decision instead of the proxy. For example, the adjustable autonomy module can reason that a decision to accept a role to rescue a civilian from a burning building should be made by the human who will go into the building rather than the proxy. In

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. AAAMAS ‘06, May 8-12, 2004, Future University, Hakodate, Japan. Copyright 2004 ACM 1-58113-000-0/00/0004…$5.00.

93

practice, the overwhelming majority of coordination decisions are made by the proxy, with only key decisions referred to human operators. Teams of proxies implement team oriented plans (TOPs) which describe joint activities to be performed in terms of the individual roles to be performed and any constraints between those roles. Typically, TOPs are instantiated dynamically from TOP templates at runtime when preconditions associated with the templates are filled. Typically, a large team will be simultaneously executing many TOPs. For example, a disaster response team might be executing multiple fight fire TOPs. Such fight fire TOPs might specify a breakdown of fighting a fire into activities such as checking for civilians, ensuring power and gas is turned off and spraying water. Constraints between these roles will specify interactions such as required execution ordering and whether one role can be performed if another is not currently being performed. Notice that TOPs do not specify the coordination or communication required to execute a plan, the proxy determines the coordination that should be performed. Current versions of Machinetta include state-of-the-art algorithms for plan instantiation, role allocation, information sharing, task deconfliction and adjustable autonomy. Many of these algorithms utilize a logical associates network statically connecting all the team members. The associates network is a scale free network which allows the team to balance the complexity of needing to know about all the team and maintaining cohesion. Using the associates network key algorithms, including role allocation, resource allocation, information sharing and plan instantiation are based on the use of tokens which are ``pushed'' onto the network and routed to where they are required by the proxies. For example, the role allocation algorithm, LA-DCOP [9], represents each role to be allocated with a token and pushes the tokens around the network until a sufficiently capable and available team member is found to execute the role. The implementation of the coordination algorithms uses the abstraction of a simple mobile agent to implement the tokens, leading to robust and efficient software.

We have recently integrated Machinetta [2] with the USARsim simulation to provide a testbed for studying human control over cooperating teams of robots. This paper reports our first tests of the system and does not yet fully exploit the richness and complexity of coordination that are available.

1.2 Experimental Task In this experiment, participants were asked to control 3

simulated P2DX robots (Figure 1) to search for victims in a damaged building. Each robot was equipped with a pan-tilt camera with a fixed 45 degrees of FOV and a front laser scanner with 180 degree FOV and resolution of 1 degree. The participant interacted with the robots through our Robots Control System (RCS). Status information, camera video, laser scanning range data, and a global map built from that data were available from each robot. The participant controlled the robot to explore the building and search for victims by issuing waypoints or teleoperating the robot and panning/tilting the camera,. Once a victim was identified, the participant marked its location on the global map.

Challenges to mobility encountered in real robotic search and rescue tasks were simulated in our experiment by obstacles including chairs, bricks, and pipes. Transparent sheets of plastic and mirrors were introduced to cause perceptual confusion and increase task difficulty. The camera’s FOV was restricted to 45 degrees to reflect typical limitations. As with real robotic system, there are uncertainties and delays in our RCS. Range data had simulated errors, the map was based on probabilistic data and some obstacles such as a chair or desk might be lost on the map because of inaccuracies in laser detection. Walls especially thin ones were also subject to loss due to errors in range data. There are also slight delays in video feedback and response to commands.

94

The RCS could work in either auto or manual mode. Under auto mode, the robots could cooperate in a limited way to automatically explore the environment. In manual mode, the robots had no automatic exploration capabilities and stopped after completing their commands. The experiment followed a repeated measures design in which participants controlled in both manual and auto modes. Order of presentation was counterbalanced and participants explored the same sequence of environments. The robots’ location, orientation and the users’ actions were recorded and timestamped throughout the experiment. The final map with marked victims was also saved. Demographic information and posttest survey were also collected.

1.3 The Robot and Environment Simulation In this experiment, we used USARSim [11], a high-fidelity simulation of urban search and rescue (USAR) robots and environments. USARSim supports human-robot interaction (HRI) by accurately rendering user interface elements (particularly camera video), accurately representing robot automation and behavior, and accurately representing the remote environment that

links the operator’s awareness with the robot’s behaviors. It was built based on a multi- player game engine, UnrealEngine2, an so is well suited for simulating multiple robots. USARSim uses the Karma Physics engine to provide physics modeling, rigid-body dynamics with constraints and collision detection. It uses other game engine capabilities to simulate sensors including camera video, sonar, and laser range finder. The experiment uses USARsim’s model of the NIST Yellow Arena [3]. The victims are evenly distributed within the arena and may appear as partial or whole human bodies . Victims were designed and placed to make the difficulty of finding them roughly the same. Two similar arenas (Figure 2) are used in the experiment. The two arenas were constructed from the same elements but with different arrangements.

1.4 The Robot and Environment Simulation

a) Arena 1 b) Arena 2 Figure 2. The Arenas.

95

USARSim

Driver

Machinetta Proxy

Driver

Machinetta Proxy

Driver

MachinettaProxy

User Interface

Machinetta Proxy

Comm Server

Robot 1 Robot 3 Robot 2

Figure 3. System Architecture.

The Robots Control System is based on Machinetta [2], a multiagent system based on teamwork proxies. The system’s architecture is shown in Figure 3. Each virtual robot connects with Machinetta through a robot driver. The driver parses the robot’s sensor data and transfers them to the Machinetta proxy. It also has limited low-level autonomy to interpret the proxy’s plan as robot commands; control the robot to avoid obstacles; and recover the robot when stuck. The user interface is connected to Machinetta as well to create a RAP (Robot, Agent and Person) system. There are embedded cooperation algorithms in Machinetta that can coordinate the robots and people through the Comm Server that exchanges information among the Machinetta proxies.

When the system works in manual mode, cooperation among the robots eliminated. When it runs in auto mode, the robot proxy is allowed to analyze the range data to determine what nodes the robot team needs to explore and how to reach those nodes from the current position (generating the paths). By exchanging these nodes and route information through Machinetta, a robot proxy can accept and execute a plan to visit a node by following a path (a series of waypoints).

Through the user interface, the operator can also directly control the robots’ cameras, teleoperate them or issue waypoints to the robots. Robots are controlled one at a time with the selected robot providing a full range of data while the unselected ones provide camera views for monitoring. On the user interface (figure 3), each robot is represented by a unique color. The control component’ background color is set to the currently selected robot’s color to help the users identify which the robot they are controlling. The components of the interface are:

• Robots List (the upper left component)

Figure 4. The Robots Control System.

The Robots List was designed to help the user monitor the robots. It lists the robots with their names, states, camera video and colors. It is also used to select the controlled robot. Camera video for this component is updated at a low frame rate.

• Map (left bottom component)

This component displays the global map created by the robots. It is intended to help the user maintain situational awareness. On this component, blue indicates unexplored areas; white shows an unoccupied area that has been explored and black shows obstacles within an explored area. Areas with gray color may or may not contain objects. Dark gray indicates that an area contains an object with high probability.

• Video Feedback (upper center component)

The currently selected robot’s video is displayed on this component. The picture is updated frame by frame with high frequency. The camera’s pan and tilt angles are represented by the crosshair on the video. The ‘reset’ button re-centers the camera. The ‘zoom’ feature was disabled for this experiment to provide a fixed FOV.

• Teleoperation (upper right component)

This component includes two sub-panels. The “Camera” panel is used to pan, tilt or center the camera. The “Wheels” panel is a simulated joystick that controls the robot’s movement. When the user uses the joystick, the robot will automatically clear its exploring path and enter teleoperation mode. In the auto condition after the user finishes teleoperating, the robot will return to auto mode and attempt to generate a new path, in the manual mode the robot remains stopped. A teleoperation episode is terminated when the user clicks the “Auto” button or 6 seconds has passed without operator input.

• Mission (bottom center component)

This component displays the current exploration situation on a “you-are-here” style map. The upper direction of the map is always the camera’s direction. The range data is displayed as bold green line overlaid on the map. The red cone emitted from the

96

Table 1 Sample Demographics

Age Gender Education

19 20~35 Male Female Currently Undergraduate

Complete Undergraduate

Order 1 1 6 1 6 4 3

Order 2 1 6 4 3 6 1

Total 2 12 5 9 10 4

robot marks the area shown in the video feedback. Combining the cone with video feedback can provide the user with better situation awareness and sense of distances. The path the robot is trying to follow is also shown on the map. With this component, the user can create a new path by issuing a series of waypoints, modify the current path by moving waypoints, or mark a victim on the map. When the user begins to mark a victim, the robot pauses its action until the user finishes the mark operation.

Table 2 Participants Experience

Computer Usage (hours/week)

Game Playing (hours/week) Mouse Usage for Game Playing

<1 1-5 5-10 >10 <1 1-5 5-10 >10 Frequently Occasionally Never Order 1

1.5 Procedure This experiment compared robot team control performance under auto and manual modes. Participant demographics were collected at the start of the experiment using an on-screen questionnaire. Standard instructions explaining how to use the interface were followed by a ten minute practice session in which participants following instructions practiced each of the operations available in the two modes and finished after searching for and finding a victim in auto mode. Order of presentation was counterbalanced with half of the participants assigned to search for victims in Arena-1 in auto mode and the other half in manual. After 20 minutes the trial was stopped. Participants were given brief instructions reminding them of significant features of the mode they had not used and then began a second 20 minute trial in Arena-2. At the conclusion of the experiment participants completed an online survey.

14 paid participants recruited from the University of Pittsburgh community took part in the experiment. The participants’ demographic information and experience are summarized in tables 1 and 2.

2. Results

2.1 Overall Measures 2.1.1 Subjective Measures Participants were asked to rate to what extent autonomy helped them find victims. The results show that most participants (79%) rated autonomy as providing either significant or minor help.

Only 1 of the 14 participants (7%) rated autonomy as making no difference and 2 of the 14 participants (14%) judged autonomy to make things worse.

0 2 1 4 3 4 0 0 6 1 0

Order 2 0 0 6 1 3 3 1 0 2 5 0

Total 0 2 7 5 6 7 1 0 8 6 0

Outcome of autonomy

36%

43%

7%

14%

Significant HelpMinor HelpNo DifferenceWorse

Figure 5. Outcome of autonomy

Figure 6. Victims found by participants.

97

2.1.2 Performance Measures 2.1.2.1 Victims Comparing the victims found by the same participant under auto mode and the victims found under manual mode using a one tail paired t test, we found that participants found significantly more victims in auto mode than in manual mode (p=0.044) (Figure 6).

2.1.2.2 Explored Ratio The explored ratio is the percentage of the area scanned by the robots. A one tail paired t-test was used to compare auto and manual modes. Participants were found to explore wider areas under auto mode than in manual mode (p=0.002).

2.2 Distribution of Attention among Robots Measuring the distribution of attention among robots as the standard deviation of the total time spent with each robot no difference (p=0.232) was found between auto and manual modes. However, we found that under auto mode, the same participant switched robots significantly more frequently than under manual mode (p=0.027). The posttest survey showed that most

participants switched robots based on the Robots List component. Only 2 of the 14 participants (14%) reported switching robot control independent of this component.

We also found that switches in control among robots led to finding more victims. Figure 9 shows the regression of victims found on the number of switches in attention among the robots (R2=0.477 p=0.006).

2.3 Forms of Control

Switches vs. Victims

0

5

10

15

20

25

30

0 20 40 60 80 100 120 140 160

Switching times in both arenas

Vict

ims

foun

d in

bot

h ar

enas

Participants had three forms of control to locate victims: waypoints, teleoperation, and camera control. No difference was found between auto and manual modes in the use of these forms of control. However, in the auto mode, participants were less likely to control waypoints (p=0.004) or teloperate (p=0.046) during any single control episode.

Comparing the victims found with control operations (waypoints and teleoperation), we found an inverted U relationship between control operations and the victims found (figure 12). Too few or too much movement control led to fewer found victims.

Victims Linear (Victims)

Figure 9. Switches vs. Victims.

Figure 7. Explored Ratio.

Figure 10. Waypoints controls in one switching. Figure 8. Switching Times.

98

2.4 Trust and Capability of Using Interface In the posttest we collected participants ratings of their level of trust in the system’s automation and their ability to use the interface to control the robots. 43% of the participants trusted the autonomy and only changed the robot’s plans when they had

spare time. 36% of the participants reported changing about half of the robot’s plans while 21% of the participants showed less trust and changed the robot’s plans more often. A one tail t-test, indicates that the total victims found by participants trusting the autonomy is larger than the number victims found by other participants (p=0.05). 42% of the participants reported being able to use the interface well or very well while 58% of the participants reported having difficulty using the full range of features while maintaining control of the robots. A one tail t test shows that participants reporting using the interface well or very well found more victims (p<0.001) based on a one tail t-test. Participants trusting the autonomy reported significantly higher capability in using the user interface (p=0.001) and conversely participants reporting using the interface well also had greater trust in the autonomy (p=0.032).

3. Discussion

Figure 11. Teleoperations in one switching.

This experiment is the first of a series investigating control of cooperating teams of robots using Machinetta. In this experiment cooperation was extremely limited primarily involving the deconflicting of plans so that robots did not explore or re-explore the same regions. The presence of simple path planning capabilities and limited autonomy in addition to coordination in the auto condition prevents us from attributing our results solely to the presence of a coordination mechanism. In future experiments we intend to extend the range of coordination to include heterogeneity in sensors, mobility, and resources such as battery power to provide richer opportunities for cooperation and the ability to contrast multirobot coordination with simple automation.

Although only half of the participants reported trusting the autonomy or being able to use the interface well, the results showed that autonomy helped the operators explore more areas and find more victims. In both the conditions participants divided their attention approximately equally among the robots but in the auto mode they switched among robots more rapidly thereby getting more detailed information about different areas of the arena being explored.

The frequency of this sampling among robots was strongly correlated with the number of victims found. This effect, however, cannot be attributed to a change from a control to a monitoring task because the time devoted to control was approximately equal in the two conditions. We believe instead that searching for victims in a building can be divided into a series of subtasks involving things such as moving a robot from one point to another, and/or turning a robot from one direction to another with or without panning or tilting the camera. To effectively finish the searching task, we must interact with these subtasks within their neglect time[6] that is proportional to the speed of movement. When we control multiple robots and every robot is moving, there are many subtasks whose neglect time is usually short. Missing a subtask means we failed to observe a region that might contain a victim. So switching robot control more often gives us more opportunity to find and finish subtasks and therefore helps us find more victims. This focus on subtasks extends to our results for movement control which suggest there may be some optimal balance between monitoring and control. If this is the case it may be possible to improve an operator’s performance through training or online monitoring and advice.

Operation vs. Victims

0

5

10

15

20

25

30

0 20 40 60 80 100 120 140

Control Times

Vict

ims

Waypoints Control# Teleoperation# Total#

Figure 12. Robot Controls vs. Victims.

4. ACKNOWLEDGMENTS This project is supported by NSF grant NSF-ITR-0205526.

5. REFERENCES [1] Crandall, J. and M. Goodrich. Characterizing Efficiency of

Human Robot Interaction: A Case Study of Shared-Control Teleoperation. in proceedings of the 2002 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2002.

[2] Farinelli, A., P. Scerri, and T. M. Building large-scale robot systems: Distributed role assignment in dynamic, uncertain domains. in AAMAS'03 Workshop on Resources, role and task allocation in multiagent systems. 2003.

99

[3] Jacoff, A., Messina, E., Evans, J. Experiences in deploying test arenas for autonomous mobile robots. in Proceedings of the 2001 Performance Metrics for Intelligent Systems (PerMIS) Workshop. 2001. Mexico City, Mexico.

[4] Kingsley, F., R. Madhavan, and L.E. Parker. Incremental Multiagent Robotic Mapping of Outdoor Terrains. in Proceedings of the 2002 IEEE International Conference on Robotics & Automation. 2002.

[5] Nickerson, J.V. and S.S. Skiena. Attention and Communication: Decision Scenarios for Teleoperating Robots. in Proceedings of the 38th Annual Hawaii International Conference on System Sciences. 2005.

[6] Olsen, D. and M. Goodrich. Metrics for evaluating human-robot interactions. in Proc. NIST Performance Metrics for Intelligent Systems. 2003.

[7] Pynadath, D.V. and Tambe, M., An Automated Teamwork Infrastructure for Heterogeneous Software Agents and Humans. Journal of Autonomous Agents and Multi-Agent Systems, 7, 2003, 71-100.

[8] Scerri, P., et al., Coordinating large groups of wide area search munitions, in Recent Developments in Cooperative Control and Optimization, D. Grundel, R. Murphey, and P. Pandalos, Editors. 2004, Singapore: World Scientific. p. 451-480.

[9] Scerri, P.; Farinelli, A.; Okamoto, S.; and Tambe, M.. Allocating tasks in extreme teams. In Proc. of the fourth international joint conference on Autonomous agents and multiagent systems, 2005.

[10] Tambe, M., Towards Flexible Teamwork. Journal of Artificial Intelligence Research, 1997. 7: p. 83-124.

[11] Wang, J., Lewis. L., Gennari J. A Game Engine Based Simulation of the NIST Urban Search & Rescue Arenas. in Proceedings of the 2003 Winter Simulation Conference. 2003. New Orleans.

100

Participatory Simulation for Designing Evacuation Protocols

Yohei Murakami Department of Social Informatics

Kyoto University Kyoto, 606-0801, Japan

+81 75-753-5398

yohei@kuis.kyoto-u.ac.jp

Toru Ishida Department of Social Informatics

Kyoto University Kyoto, 606-0801, Japan

+81 75-753-4820

ishida@i.kyoto-u.ac.jp

ABSTRACT In evacuation domain, there are evacuation guidance protocols to make a group of evacuees move smoothly. Each evacuee autonomously decides his/her action based on the protocols. However, the protocols sometimes conflict with evacuees’ goals so that they may decide to violate the given protocols. Therefore, protocol design process has to consider human’s decision making on whether or not to follow the protocols so as to control every evacuee more smoothly. To address this problem, we introduce participatory simulation where agents and human-controlled avatars coexist into protocol design process. It allows us to validate protocols at lower cost than demonstration experiments in the real world, and acquire decision making models from log data. In order to refine the protocols based on the acquired models, we have designed and implemented the agent architecture separating decision making from protocol execution.

Categories and Subject Descriptors I.2.11 [Artificial Intelligence]: Distributed Artificial Intelligence – multiagent systems.

General Terms Design

Keywords protocol design, multi-agent simulation, evacuation simulation

1. INTRODUCTION Disaster-prevention hardware, such as fire protection equipments and refuge accommodations and so on, is improved based on the lesson learned from frequent disasters. In contrast with the hardware, disaster-prevention software such as evacuation methods has no advancement. This is because a lot of subjects are necessary to validate designed evacuation methods and the cost to conduct the demonstration experiments is very high. Moreover,

such experiments are too complex to reproduce the results. The non-reproducibility causes difficulties in analyzing problems occurring in the experiments.

One of the approaches to solving these problems is multi-agent simulation [9]. Multi-agent simulation is a simulation to monitor macro phenomena emerging from interactions between agents which model actors such as humans. Once models are acquired from existing documents and past research data, we can reproduce simulation results as many times as needed. Besides, multi-agent simulation enables us to estimate the effectiveness of designed evacuation methods by assigning the methods to agents as their interaction protocols. That is, we view the protocols as behavioral guidelines of the agents during evacuation.

However, even when the simulation results show the effectiveness of the evacuation protocols for simulated evacuees, the problem of validating whether they are effective for real evacuees still remains. In order to solve the problem and develop more effective evacuation methods, we need participatory approach which is a method of bringing potential evacuees and leaders into the simulation. Therefore, we aim to design evacuation methods by using participatory simulation where agents and human-controlled avatars coexist. In this simulation, we can check the effectiveness of the designed evacuation methods by providing the protocols for not only agents but also potential evacuees controlling avatars. In order to accomplish our goal, we set up the following research issues.

Establishment of protocol design process: Humans may not follow the given protocols, since they are more autonomous than agents. Thus, we need protocol refinement process considering human’s decision making about whether or not to follow the given protocols. To construct more valid protocols, we have to modify protocols after verifying human’s decision making, internal models, obtained from participatory simulations.

Realization of autonomy under social constraints: To simulate human autonomous behavior under a given protocol, which is a social constraint, agent architecture needs a decision making mechanism independent of a given protocol. This mechanism coordinates a proactive action and an action prescribed by a given protocol, and realizes the selfish behavior that the agent violates its given protocol.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ATDM’06, May 8, 2006, Hakodate, Hokkaido, Japan. Copyright 2006 ACM 1-59593-303-4/06/0005...$5.00.

This paper will first describe what an interaction protocol is, and then clarify the target protocol we try to design. Next, we will

101

propose our protocol design process and agent architecture necessary to realize the proposed process. Finally, we will check usefulness of our approach by applying it to refine “Follow-me method,” a new evacuation method proposed by a social psychologist.

2. Interaction Protocols In the area of multi-agent, an interaction protocol is often employed as a means to control interactions among agents for any purpose. Interaction protocols can be roughly divided into two groups based on the goal of the protocol; interaction protocol for coordinating agents and interaction protocol for avoiding conflicts among agents.

The former is given to agents having joint goal or intention. By strictly following the prescribed protocols, agents can exchange a sequence of messages without considering the details of implementation of other agents. This conversation leads joint actions of multiple agents, such as consensus building and coordination. A Contract net protocol is representative of such coordination protocols in multi-agent domain. Foundation for Intelligent Physical Agents (FIPA), standards body for multi-agent technology, tries to develop specifications of standard coordination protocols in order to construct multi-agent systems in open environment like the Internet.

On the other hand, the latter is given to the society consisting of agents with different goals. The protocols define the minimum behavioral guidelines which constituent agents should obey. By sharing the protocols, every agent can expect other agents’ behavior and avoid the conflicts between agents. For example, left-hand traffic is this type of protocol. The protocol plays an important role that prevents collisions of cars. Such a protocol acts as a social constraint, so it often conflicts with agents’ goals. As a result, agents sometimes violate social constraints. That is, we can not always force agents to follow social constraints. For instance, a car with a pregnant woman can move the right side of a road to avoid a traffic jam.

In evacuation domain, there are humans with different goals; evacuee’s goal is to go out the nearest exit as early as possible and leader’s goal is to guide evacuees toward a correct exit. Hence, this paper focuses on the latter, and we try to design evacuation protocols. In this section, we introduce the existing protocol description languages and point out the problems in applying them to describe the latter protocol. Then, we explain scenario description language Q which we employ in order to describe evacuation protocols.

2.1 Protocol Description Language There are some representative protocol description languages such as AgenTalk [7] and COOL [1] based on finite state machine, and IOM/T [3] equal to interaction diagrams of AUML [12] which is modeling language focusing on sequence of message exchanges between agents. AgenTalk presents clear interfaces between agents and the given protocols, called agent-related functions, for specifying general protocols to adapt to each application domain. Functions specific to each agent are implemented as call-back functions which are invoked from protocols using the agent-related functions. COOL provides continuation rules that say the subsequent protocols for agents to

handle several protocols given to them. IOM/T generates skeleton classes from described protocols. As mentioned above, these protocol description languages provide just three types of choices for agents; whom to interact with, what protocols to employ, and what message content to send to other agents. This is because these languages are designed so that protocols described in them can define every interaction between agents and completely control the agents for coordination among them. That is, the protocols described in these languages limit agent’s autonomy. Therefore, we employ scenario description language Q, a protocol description language to request agents to do something. This language delegates to agents decisions about whether or not to follow requests. Therefore, it enables agents to do actions other than protocols and violate protocols.

2.2 Scenario Description Language Q Scenario description language Q is a protocol description language to define the expected behavior of agents [6]. In Q, protocols are represented by finite state machine and called scenario. Scenarios consist of cues, requests to observe environment, and actions, requests to affect environment. Q scenarios are interpreted by Q interpreter which is designed to connect with legacy agents. All Q interpreter can do is to send request messages to agents and receive the result messages from the agents. It does not consider how to execute the requests. If protocol designers respect the agent autonomy, the level of abstraction of cues and actions is high. On the other hand, if they require the degree of preciseness of protocols, the vocabularies are defined concretely. In fact, Q is employed in realizing evacuation simulation [9] and socio-environmental simulation [14], both of which need complex social interactions.

3. Protocol Design Process The existing protocol development process consists of the following five steps: analysis, formal description, validation, implementation, and conformance testing. In this process, correctness of the designed protocol is assured by checking whether a deadlock exists in the protocol or not, and whether the protocol can terminate or not [5][8]. This process is realized by assuming that every agent complies with interaction protocols and the protocols describe every interaction between them.

On the other hand, validation of protocols as social constraints takes more than verifying the correctness of the protocols, such as deadlock freeness, liveliness, and termination, since they describe only partial interaction between humans, and delegate decision making on what to do to humans. In order to check the validity of the protocols, we have to also consider human decision making. This is why participatory simulation where agents and human-controlled avatars coexist is effective to refine the protocols. Moreover, by acquiring agent’s internal model about decision making from participatory simulation results, we can modify the protocols efficiently without conducting participatory simulation many times. However, the effectiveness of the protocols strongly depends on the internal models, so we have to verify the acquired internal models whenever we obtain them. The protocol refinement process defining criteria about verification of the internal models is needed.

102

In this section, we describe overview of our protocol refinement process referring to Figure 1. The section 5 provides details of each step while designing evacuation protocols

Step1: Creating Protocols

(1) Extract agent's action rules from the existing documents and data of previous experiments, and construct agent's internal models (M1).

(2) Describe the initial protocols (P1) by using existing documents and experts knowledge.

(3) Conduct multi-agent simulation. The system designers check if its result (R1) satisfies their goal (G). If it does not satisfy the goal, they repeatedly modify both of the agent's internal models and the protocols until the result of simulation is closely similar to the goal. In addition, let S be a simulation function whose arguments are agent's internal models and protocols.

Step2: Validating Protocols

(1) Replace some of the agents with human-controlled avatars given the same protocols as in step 1 (P1). This participatory simulation enables us to store log data which are impossible to record in the real experiments.

(2) Compare the result of the participatory simulation (R2) and the result in step1 (R1). System designers check if the protocols (P1) are valid for the real users.

(3) Finish this protocol refinement process if R2 is similar to R1. Otherwise, go to the step 3.

Step3: Modifying Agent Models

(1) Modify the agent's internal models (M3) using log data obtained by participatory simulation.

(2) Conduct multi-agent simulation using the modified agent's internal models and the protocols (P1).

(3) Compare the result of the multi-agent simulation (R3) and that of the participatory simulation (R2). System designers verify the modified agent's internal models. If they check the verification of the models, go to step 4. Otherwise, they repeatedly modify the agent's internal model until R3 is closely similar to R2.

Step4: Modifying Protocols

(1) Modify the protocols (P2) in order to efficiently control a group of agents based on the agent's internal model (M3) and satisfy the goal.

(2) Conduct multi-agent simulation using the modified protocols (P2).

(3) Compare the result of multi-agent simulation (R4) and the ideal result (R1). The system designers verify the modified protocols. If they check the verification of the modified protocols, go to step 2 again in order to confirm if the modified protocols are valid for the real users. Otherwise, they repeatedly modify the protocols until R4 is closely similar to R1.

4. Agent Architecture with Social Constraints In contrast to the existing interaction protocols whose goals are to realize joint actions by multiple agents, our target protocols are to accomplish individual actions without conflicts with other agents. Such a difference of attitudes towards the protocols changes agent architecture. In the existing agent architecture, the given protocols are embedded so that the behavior of traditional agents can be strictly controlled by the protocols. To construct such agent architecture, there are two approaches; implementing the protocols as agent's behavior [2], and deploying a filtering function between each agent and its environment in order to control interactions [4]. On the other hand, agent architecture with social constraints has to realize decision making on whether an agent follows the constraints so that it can achieve its own goals. Therefore, agent's decision making needs to be separated from interpretation of protocols. This architecture enables agents to deviate from the protocols by dealing with requests from the protocols as external events like their observation. In the following sections, we discuss design and implementation of agent architecture with social constraints.

4.1 Design of Agent Architecture In this research, we need agent architecture that enables agents to select either proactive action or action described in the protocols. Implemented based on this architecture, agents can autonomously decide to perform next action according to their local situation. Figure 2 shows the agent architecture with social constraints, which we design.

Figure 1. Protocol Design Process.

103

Sensor

PriorityTable

Environment

Social Constraint

ActionRules

ActionSelection

Self-Action

ConflictResolution

PlanSelection

Actuator

PlanLibrary

Event Symbolization

Protocol

Result

Agent

RequestRequest Ruled-Action

Interpreter

Internal Model

In this architecture, observations which an agent senses are symbolized as external events, which are passed into the action selection. At the action selection, an executable action rule is chosen from a set of action rules depending on the received events. Action declared in an action rule is sent to the conflict resolution as proactive action the agent wants to perform.

On the other hand, a protocol given to the agent is interpreted by the interpreter that also sends a request of sensing and acting to the agent. If the agent observes an event as described by the protocol, the external event is passed into the interpreter outside the agent as well as action selection within the agent. Interpreter interprets the given protocol and requests the agent to perform an action subsequent to the observation. The action prescribed in the protocol is also sent to conflict resolution.

The conflict resolution chooses only one action from the set of actions received from both of the action selection and the interpreter, depending on priorities of the actions. The chosen action is realized by employing the corresponding plans in the plan library. The effect of executing plans is given to the environment by its actuators. Information concerning the chosen action is kept in conflict resolution until the action is completed. This is used to compare priorities between the ongoing action and the new received action. If the priority of the new received action is higher than one of the ongoing action, the agent stops the ongoing action and starts to execute the new action instead of it.

In this way, the agent's behavior depends on a set of action rules leading proactive action, a table of priorities between actions used at conflict resolution, and a plan library storing how to realize the agent's action. Therefore, we define these three components as agent's internal model. Especially, a table of priorities between actions is the most important component to determine the agent's personality; social or selfish. If the proactive action is superior to the action prescribed in the protocol, it means a selfish agent. On the contrary, if these priorities are reversed, it means a social agent.

4.2 Implementation of Agent Architecture To implement agent architecture with social constraints, we employ scenario description language Q and production system for description of a protocol and construction of decision making, respectively. The merit of the separation between protocol description and model description is that protocol designers and a-

Env

ProtocolProtocol

ProtocolProtocolProtocol

Protocol

Interpreter

QScenario

Agent

FreeWalkProduction System

Matching

WM PM

ActuatorsConflict resolution

SensorsEvent symbolization

Requests

ResultsMessagehandler

QInterpreter

Figure 3. Implementation of Agent Architecture. Figure 2. Agent Architecture with Social Constraints. gent modelers have only to describe objects of their own interest.

This implementation is shown in Figure 3.

In this implementation, three components indicating agent's internal model, a set of action rules, a table of priorities, and a plan library, are represented as prioritized production rules. All the rules are stored in production memory (PM). However, execution of plans depends on an intended action as well as external events, so it is necessary to add the following condition into the precondition of plans: “if the corresponding action is intended.” For example, a plan like “look from side to side and then turn towards the back” needs the condition: whether or not it intends to search someone. By adding another condition concerning intention into the precondition of plans, this implementation controls condition matching in order not to fire unintended plans. Therefore, when any action is not intended, only the production rules representing action rules are matched against the stored external events. The successfully matched rule generates an instantiation to execute the rule, and then it is passed into the conflict resolution.

On the other hand, a Q scenario, a protocol, is interpreted by Q interpreter. The interpreter sends a request to the agent according to the given Q scenario. The request is passed into the agent through the message handler which transforms the request message to a working memory element in order to store it in the working memory (WM). Prepared in the PM, the production rule denoting “a requested action is intended to perform” can make an instantiation of the rule and send it to the conflict resolution.

Finally, the conflict resolution selects the action whose priority is highest. Thus, if the above production rule ``a requested action is intended to perform'' is superior to other production rules representing action rules, the agent socially behaves complying with the protocol. Conversely, if production rules corresponding to action rules are superior to the above action rule to follow the request, the agent selfishly behaves ignoring the request.

However, although such priorities enable an agent to resolve a conflict between concurrently applicable rules, it is impossible to control instantiations generated while executing another instantiation. For example, this architecture cannot avoid executing action whose priority is lower than ongoing action. Therefore, we need to design the production rules considering data dependency between the rules. Especially, we focus on intention, because every plan execution depends on generated intention. We have to consider the case where the intention to do

104

+

+

+ +

Rule to follow requests

Plan

α

- +Action rule

+

++Plan

β- +

Action rule

γ

+

-

External event

Internal event(Intended action)

External eventExternal event

Internal event(Intended action)

Priority of a production rule

High

LowInitiating actuator/sensor

Initiating actuator/sensor

α β

+ +-

α βα β

+ +-

βγ

+ +-

βγ βγ

+ +-

γ α

+ +-

γ αγ α

+ +-

Rules to delete aninternal event

action whose priority is lower than ongoing action is generated, and the reverse case.

In the former case, we add a new condition; “if there is no intention generated by the more-prioritized rules in WM”, into the precondition of the less-prioritized rules. Because intention to perform action is kept in WM until the action is completed, the new condition blocks generating instantiation whose action is less prioritized than the ongoing action, while performing the ongoing action.

In the latter case, the problem that various intentions are in WM occurs. This state enables every plan to realize these intentions to fire all the time. Therefore, the production rule that deletes the WME denoting intention, whose action is less prioritized than others, is necessary.

Figure 4 shows the data dependency among production rules which compose social agents. Circles and squares mean production rules and WMEs, respectively. Arrow lines represent reference and operation towards WMEs. Specifically, an arrow line from a square to a circle represents reference to the data, while a reverse arrow line represents operation of the data.

5. Design of Evacuation Protocols In disaster-prevention domain, simulations can contribute to evaluation of contingency planning and analysis of second disaster, since it is difficult to conduct experiments in the real world. Traditional simulations ignore the differences between people and treat everyone as uniform bits with the same simple behavior. These simulations are employed in order to evaluate construction of a building. Human action is, however, predicted from a numerical analysis of just spatial position without considering social interactions such as guidance although social interaction is extremely common and strongly influences the responses seen in real-world evacuations. Therefore, we conduct evacuation simulations considering social interactions in order to design evacuation protocols. As a first step to addressing the problem of designing evacuation protocols, we simulated the controlled experiments conducted by Sugiman [13]. He established a simple environment with human subjects to determine the effectiveness of two evacuation methods: the “Follow-direction method” and the “Follow-me met-

Figure 5. Ground Plan of the Experiment and Initial Position of Subjects [13].

hod.” In the former, the leader shouts out evacuation instructions and eventually moves toward the exit. In the latter,

the leader tells a few of the nearest evacuees to follow him and actually proceeds to the exit without verbalizing the direction of the exit. Sugiman used university students as evacuees and monitored the progress of the evacuations with different number of leaders.

Figure 4. Data Dependency for Social Agent Model.

The experiment was held in a basement that was roughly ten meters wide and nine meters long; there were three exits, one of which was not obvious to the evacuees as shown in Figure 5. The ground plan of the basement and the initial position of subjects are also shown in the figure. Exit C was closed after all evacuees and leaders entered the room. At the beginning of the evacuation, Exit A and Exit B were opened. Exit A was visible to all evacuees, while Exit B, the goal of the evacuation, was initially known only by the leaders. Exit A was treated as a danger. Each evacuation method was assessed by the time it took to get all evacuees out.

5.1 Step1: Creating Protocols In our past research, we succeeded in double-checking the result of the previous fire-drill experiment by multi-agent simulation [9]. However, the previous simulation employed the simplest agent's internal model, which only followed the given protocols. That is, every interaction was described as interaction protocols. Therefore, we have to redesign interaction protocols with an appropriate degree of abstraction so that participants can easily understand them in the next step.

At first, we redescribe interaction protocols the same as those employed in the real experiments, in the form of finite state machine. In “Follow-me method” condition, each instruction given a leader and an evacuee in the real experiments is as follow.

Leader: While turning on emergency light, put his white cap on. After a while, when the doors to this room are opened, say to an evacuee close to him “Come with me”, and subsequently move with the evacuee to Exit B.

Evacuee: When the doors to this room are opened, escape from the room while following direction from leaders with a white cap on.

We try to extract action rules from the above instructions, and construct finite state machine by assigning each state to concurrently applicable rules. The generated finite state machines for a leader and an evacuee are shown in Figure 6 and Figure 7, respectively.

105

Table 1. Rules for Evacuation Scenarios.

Agent Rule (Plan) When the leader goes out from the room, he checks if the target evacuee also goes out from it. (Plan) If the target evacuee is out of the room, the leader goes to the next exit. (Plan) If the target evacuee is within the room, the leader walks slowly so that he/she can catch up with him.

Leader (Follow-me)

(Plan) If the target evacuee goes out from the room, the leader picks up the pace and moves toward the next exit. (Action rule) The evacuee looks for a leader or an exit. (Action rule) If the evacuee sees the exit open, he goes to the exit. (Action rule) If the evacuee sees a leader walk, he follows him. (Action rule) If the evacuee sees another evacuee close to him move, the evacuee follows him. (Plan) In order to look for a leader or an exit, the evacuee looks from side to side. (Plan) If the evacuee observes that someone he follows goes out from the room, he walks towards the exit.

Evacuee (M1)

(Plan) If the evacuee also goes out from the room, he follows the same target again. (Action rule) If the evacuee sees the people around him walk, it also walks towards the same direction. (Action rule) If the evacuee sees congestion in the direction of his movement, he looks for another exit. (Plan) In order to look for a leader or an exit, the evacuee turns towards the back.

Evacuee (M3)

(Plan) In order to look for a leader or an exit, the evacuee turns on the same direction as the people around him.

On the other hand, the difference between the previous protocols and the redescribed protocols is an agent's internal model. An agent's internal model consists of a set of action rules, which generates intention to perform a proactive action, and a set of plans, which realize the intention to do an action. Hence, we have to classify the left rules into two sets; action rules and plans. The criterion to classify the rules is what purpose the rule is used for. The rule that realizes the same goal as the given protocol is an action rule, while the rule that realizes other behavior is a plan. In the case of an evacuee, the following rules are action rules to realize evacuation; “go to the exit in his view,” “look for a leader,” and “follow someone close to him.” These action rules

generate intention to perform actions; “go,” “look for,” and “follow.” The means to realize these actions is a plan. Action rules and plans in “Follow-me method” condition are shown in Table 1. Note that leaders have no action rules since they completely obey their protocol.

Next, we conduct multi-agent simulation with the acquired protocols and agent's internal models. By simulating in three-dimensional virtual space like FreeWalk [11], it is easy to realize participatory simulation in the next step.

5.2 Step2: Validating Protocols In the second step, we conduct participatory simulation by replacing some agents with human-controlled avatars. Participatory simulation enables us to record various data impossible to collect in the real experiments.

Figure 6. Protocol for “Follow-me method”.

The purpose of participatory simulation is validation of the protocol described in the previous step. To accomplish this purpose, we instruct subjects on the evacuation protocol before participatory simulation, and then check if the result of participatory simulation satisfies the original goal. If it does not satisfy the goal, we have to modify the agent's internal model to more valid one by noting the difference between results of simulations.

In fact, we conducted participatory simulation by replacing twelve evacuee agents with subject-controlled avatars and instructing the subjects on the evacuation protocol. The other eight agents including four leaders and four evacuees were still the agents having been used in the previous step. We collected the results in the four leader “Follow-me method” condition and the four leader “Follow-direction method” condition, respectively.

Figure 7. Protocol for Evacuees.

In consequence, only the result of participatory simulation in the four leader “Follow-me method” condition was different from the result of multi-agent simulation. Figure 8 shows the situation reproduced on two-dimensional simulator. As shown in the figure, the circled evacuee avatar looked around it in the early stage, and after emergence of congestion, it walked towards the wrong exit in order to avoid the congestion.

106

The above result implies that there is another agent's internal model other than the model constructed so far. In the next step, we try to extract the new internal model from log data obtained by participatory simulation.

5.3 Step3: Modifying Agent’s Internal Models In third step, we modify the agent's internal model using log data obtained by participatory simulation. Specifically, we refine the internal model of the avatar taking unpredictable behavior, by interviewing with the subject while showing him his captured screen, acquiring an internal model by machine learning, and reproducing the situation in participatory simulation by log data. Validity of the modified internal model is checked by comparing the result of multi-agent simulation with the modified agent model and one of participatory simulation. Modifying the agent model is repeated until the result of participatory simulation is reproduced by multi-agent simulation with the modified agent model.

In participatory simulation, we actually captured two subject's screens on videotape and then interviewed with them while showing the movie to them. Figure 9 shows the interview with the subject. Showing the movie enables the subjects to easily remember what they focused on at each situation, and how they operated their avatars. At the interview, we asked the subjects three questions; “what did you focus on?” “what did you want to do?” and “what did you do?” Table 1 classifies the acquired rules into action rules and plans depending on the same criterion as in step 1.

However, it costs us high to interview in such a style, and so it is unrealistic to interview with every subject. Therefore, we also propose the method that can support acquirement of agent models by applying hypothetical reasoning [10]. Hypothetical reasoning can assure correctness of the acquired models because they are acquired as logical consequence. On the other hand, the validness

Subject

Interviewer

Figure 9. Interview with Subjects.

of the model is realized by winnowing the acquired models through a question and answer system

5.4 Step4: Modifying Protocols In the fourth step, we modify the protocols in order to accomplish system designer's goal that the protocols can control the modified agent model correctly. Specifically, modifying the protocols is repeated until the result of multi-agent simulation consisting of the agent models acquired in the previous step satisfies the system designer's goal.

Figure 8. Results of Participatory Simulation.

In fact, we modified “Follow-me method” protocol by adding new state with the following transition rules; “if the leader finds an evacuee walk towards the reverse direction, he tells the evacuee to come with him.” The correctness of this protocol was checked if the simulation satisfied the goal that every evacuee goes out from the correct exit. The modified protocol is shown in Figure 10.

Finally, we will conduct participatory simulation using the refined protocols in order to validate the protocols. Until the result of participatory simulation satisfies the original goal of the protocol designer, this cycle, from step2 to step4, is repeated.

Figure 10. Refined Protocol for “Follow-me method”.

107

6. Conclusions In order to design evacuation protocols by using multi-agent simulation, agents need decision making independent of their protocols. With the assumption that evacuees may follow or not follow evacuation guidance, we tackled the following issues.

Establishment of protocol design process: Although participatory simulation is very effective to validate protocols, it costs us to conduct it because of the fact that it needs several subjects. Therefore, we proposed the protocol refinement process that can not only validate the protocols but also acquire the models of participants for protocol refinement. This process defines criteria of verification and validation for agent models and protocols so that any system designers can reflect subjects' feedback in protocol design regardless of their ability. In fact, we applied our proposed method to improving evacuation guidance protocols and validated its usefulness.

Realization of autonomy under social constraints: Unlike multi-agent simulation, subjects controlling avatars are so autonomous that they sometimes violate the given protocols depending on their situation if they justify the violation. This kind of autonomy is also important to examine practical evacuation guidance protocols in the real world. Therefore, we developed agent architecture separating decision making from interpretation of the given protocols by using scenario description language Q and a production system. Considering priorities of production rules and data dependency between the rules, we can realize social agents strictly complying with the given protocols and selfish agents sometimes violating the protocols.

ACKNOWLEDGMENTS The authors would like to thank H. Nakanishi and T. Sugiman for making this work possible. FreeWalk and Q have been developed by Department of Social Informatics, Kyoto University and JST CREST Digital City Project. This work has been supported by a Grant-in-Aid for Scientific Research (A)(15200012, 2003-2005) from Japan Society for the Promotion of Science (JSPS).

REFERENCES [1] Barbuceanu, M., and Fox, M.S. COOL: A Language for

Describing Coordination in Multi Agent Systems. In Proceedings of the First International Conference on Multi-Agent Systems, pp. 17-24, 1995.

[2] Bellifemine, F., Poggi, A., and Rimassa, G. Developing Multi-agent Systems with JADE. Intelligent Agents VII. Agent Theories Architectures and Languages, Springer-Verlag, pp. 89-103, 2000.

[3] Doi, T., Tahara, Y., and Honiden, S. IOM/T: An Interaction Description Language for MultiAgent Systems. In

Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 778-785, 2005.

[4] Esteva, M., Rosell, M., Rodriguez-Aguilar, J.A., and Arcos, J.L. AMELI: An agent-based middleware for electronic institutions. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 236-243, 2004.

[5] Huget, M.-P., and Koning, J.-L. Interaction Protocol Engineering. Communications in Multiagent Systems, Springer-Verlag, pp. 179-193, 2003.

[6] Ishida, T. Q: A Scenario Description Language for Interactive Agents. IEEE Computer, Vol. 35, No. 11, pp. 54-59, 2002.

[7] Kuwabara, K. Meta-Level Control of Coordination Protocols. In Proceedings of the Second International Conference on Multi-Agent Systems, pp. 165-173, 1996.

[8] Mazouzi, H., Fallah-Seghrouchni, A.E., and Haddad, S. Open Protocol Design for Complex Interactions in Multi-agent Systems. In Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 517-526, 2002.

[9] Murakami, Y., Ishida, T., Kawasoe, T., and Hishiyama, R. Scenario Description for Multi-Agent Simulation. In Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 369-376, 2003.

[10] Murakami, Y., Sugimoto, Y., and Ishida, T. Modeling Human Behavior for Virtual Training Systems. In Proceedings of the Twentieth National Conference on Artificial Intelligence, pp.127-132, 2005.

[11] Nakanishi, H. FreeWalk: A Social Interaction Platform for Group Behaviour in a Virtual Space. International Journal of Human Computer Studies, Vol. 60, No. 4, pp. 421-454, 2004.

[12] Odell, J., Parunak, H.V.D., and Bauer, B. Representing Agent Interaction Protocols in UML. Agent-Oriented Software Engineering, Springer-Verlag, pp. 121-140, 2000.

[13] Sugiman, T., and Misumi, J. Development of a New Evacuation Method for Emergencies: Control of Collective Behavior by Emergent Small Groups. Journal of Applied Psychology, Vol. 73, No. 1, pp. 3-10, 1988.

[14] Torii, D., Ishida, T., Bonneaud, S., and Drogoul, A. Layering Social Interaction Scenarios on Environmental Simulation. Multiagent and Multiagent-based Simulation, Springer-Verlag, pp. 78-88, 2005.

108

Agent Modeling of a Sarin Attack in Manhattan

Venkatesh MysoreNew York University

715 Broadway #1012New York, NY, USA

mysore@cs.nyu.edu

Giuseppe NarzisiUniversity of CataniaV.le A. Doria 6, 95125

Catania, Italy

narzisi@dmi.unict.it

Bud MishraNew York University

715 Broadway #1002New York, NY, USA

mishra@nyu.edu

ABSTRACTIn this paper, we describe the agent-based modeling (ABM),simulation and analysis of a potential Sarin gas attack at thePort Authority Bus Terminal in the island of Manhattan inNew York city, USA. The streets and subways of Manhattanhave been modeled as a non-planar graph. The people at theterminal are modeled as agents initially moving randomly,but with a resultant drift velocity towards their destinations,e.g., work places. Upon exposure and illness, they choose tohead to one of the hospitals they are aware of. A simplevariant of the LRTA∗ algorithm for route computation isused to model a person’s panic behavior. Information abouthospital locations and current capacities are exchanged be-tween adjacent persons, is broadcast by the hospital to per-sons within its premises, and is also accessible to personswith some form of radio or cellular communication device.The hospital treats all persons reaching its premises andemploys a triage policy to determine who deserves medicalattention, in a situation of over-crowding or shortage of re-sources. On-site treatment units are assumed to arrive atthe scene shortly after the event. In addition, there are sev-eral probabilistic parameters describing personality traits,hospital behavior choices, on-site treatment provider actionsand Sarin prognosis. The modeling and simulation were car-ried out in Java RePast 3.1. The result of the interaction ofthese 1000+ agents is analyzed by repeated simulation andparameter sweeps. Some preliminary analyses are reportedhere, and lead us to conclude that simulation-based analy-sis can be successfully combined with traditional table-topexercises (as war-games), and can be used to develop, test,evaluate and refine public health policies governing catas-trophe preparedness and emergency response.

Categories and Subject DescriptorsI.6.5 [Simulation and Modeling]: Model Development—Modeling methodologies; I.6.3 [Simulation and Model-ing]: Applications; J.4 [Social and Behavioral Sciences]:Sociology; J.3 [Life and Medical Sciences]: Health

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.AAMAS’06May 8–12 2006, Hakodate, Hokkaido, Japan.Copyright 2006 ACM 1-59593-303-4/06/0005 ...$5.00.

General TermsExperimentation, Security, Human Factors, Verification

KeywordsTerrorism, Emergency Response, RePast, LRTA∗

1. INTRODUCTIONNew York University’s Center for Catastrophe Prepared-

ness and Response (CCPR) was founded in the wake of thecataclysmic terrorist attacks on the World Trade Center inNew York city. As part of its Large Scale Emergency Readi-ness (LaSER) project, mathematical models of the dynam-ics of urban catastrophes are being developed to improvepreparedness and response capabilities. The need for emer-gency response planning has been reinforced by the recentstring of natural calamities and controversies over the non-implementation of suggested plans (for example, see the hur-ricane Katrina disaster predicted and analyzed well-beforethe event [11]). Conventional policy planning relies largelyon war-gaming, where the potential disaster scenario is en-acted as a table-top exercise, a computer simulation or anactual full-scale rehearsal using actual resources and play-ers. It has been repeatedly observed that “disaster planningis only as good as the assumptions on which it is based” [3].

Agent Based Modeling (ABM) is a novel technique forsimulating and analyzing interaction-based scenarios [9], withits recent application to disaster management. The first sce-nario we investigated was the 1998 food poisoning of a gath-ering of over 8000 people at a priest’s coronation in MinasGerais, Brazil leading to 16 fatalities [7].Multi-agent mod-eling was explored for this problem by allowing simplistichospital and person agents to interact on a 2-dimensionalinteger grid. Counter-intuitive and unanticipated behaviorsemerged in the extremely parameter sensitive system, imme-diately suggesting a potential use for such agent-simulation-based analysis of catastrophes. This paper provides a morethorough and practical example of how a large-scale urbancatastrophe can be modeled, how real data about maps, sub-ways and hospitals can be integrated, how person, hospitaland on-site responder behavior can be modeled, and howsimulations can be analyzed to yield tangible non-trivial in-puts that a team of expert policy makers and responderscan utilize, in conjunction with conventional approaches.

Specifically, we picked the nerve gas agent Sarin and thecity of Manhattan to demonstrate our tools and techniques.Our choice was based on the literature available about asimilar attack executed in Matsumoto in 1994 and in Tokyo

109

in 1995 [8, 10, 4]. More importantly, by altering the pa-rameters describing the conditions after the attack and theprognosis, the scenario can easily be extended to any eventinvolving a one-time exposure (e.g., chemical agent, bombexplosion, food poisoning). Communicable diseases, radio-logical releases and events requiring evacuation or quaran-tine can be captured using additional layers of behavioraland evolutionary complexity.

2. SIGNIFICANCE OF THE SCENARIO

2.1 Sarin and other Nerve Gas AgentsSarin is a volatile odorless human-made chemical warfare

agent classified as a nerve agent [10, 4]. Most nerve agentsdiffuse because of air currents, sink to lower areas and canpenetrate clothing, skin, and mucous membranes in humans.Though Sarin presents only a short-lived threat because ofquick evaporation, clothing exposed to Sarin vapor can re-lease Sarin for several minutes after contact.

2.2 Sarin Attacks in JapanThe Aun Shinrikyo cult members initiated Sarin gas re-

lease in Matsumoto, Japan on June 27/28, 1994 leading to 7deaths and injuring over 200. A larger scale attack was ex-ecuted, less than a year later, on March 20, 1995. The loca-tion was a portion of the Tokyo subway system where threetrain lines intersected and the time was morning rush hour,when the subway was extremely crowded with commuters.Following the attack, all commuters voluntarily evacuatedthe stations. Emergency Medical Services (EMS) were no-tified 14 minutes after the event. Police blocked free ac-cess to subway stations within an hour. The Japanese SelfDefense Forces decontaminated subway stations and trains,and confirmed Sarin as the toxic-agent, three hours afterthe attack. This 1995 terrorist attack led to 12 fatalitiesand about 5,500 sickened people [8]. The kinds of questionsthat analyses can try to address become clear when someof the problems faced in this scenario are considered: (1)overwhelming of communication systems, (2) misclassifica-tion and delayed characterization of attack agent, (3) sec-ondary exposure, (4) shortage of hospital resources, (5) lackof mass casualty emergency response plan, (6) absence ofcentralized coordination, and (7) overwhelming of the med-ical transportation system.

2.3 Increased Preparedness in ManhattanThe sensational terrorist attack on the Twin Towers of

the World Trade Center on September 11, 2001 has madeNew York city an accessible urban location for analyzingthe problems with the emergency response system, warrant-ing well-funded research programs to aid policy developmentand evaluation. Manhattan, a 20 square mile borough ofNew York city, is an island in the Hudson River account-ing for 1.5 out of the 8 million residents and about 2.9out of the 8.5 million daytime population. For many rea-sons, besides the fact that it has become a target of ter-rorist attacks, Manhattan poses many challenges, servingas an excellent test-bed for verifying assumptions and refin-ing policies about response to large-scale disasters in urbansettings. These include: its geographical isolation, tremen-dous population density (e.g., a day-time population almostdouble that of the resident population), extensive publictransportation system including subways, buses, trains and

Figure 1: Snapshots of the Manhattan model

ferries, its almost vertical structure, its renowned linguis-tic, ethnic, and socioeconomic diversity, its asymmetric dis-tribution of medical facilities, its proximity to nuclear andtoxic-chemical facilities, its ports and airports as an inter-national point of transit and entry, etc. (The model canbe seen in Figure 1. The color code employed is: person– green(health=1.0), red (health=0.0); hospital/responder– unused (white), inactive (grey), available (blue), critical(pink), full (orange). The streets are black and the subwayshave the New York subway color codes.)

3. MODELING THE SARIN ATTACKIn this section, we describe the different aspects of our

model, the sources of information, the assumptions, thecomputational approaches and algorithmic issues. Most be-havior is probabilistic and most parameters are normalizedand initialized uniformly in the range (0, 1).

3.1 Manhattan: Topology and TransportationWe pick the 42nd Street Port Authority Bus Terminal,

one block west of Times Square, as the site of Sarin attack.On a typical weekday, approximately 7,200 buses and about200,000 people use the bus terminal leading to an averageflux of over 133 people per minute.

3.1.1 Graph Representation of the MapThe Geographic Information Systems (GIS) street map

and the pictorial subway map of Manhattan were obtainedfrom publicly available data sources. The information wasconverted into a graph with 104,730 nodes (including 167subway stops) under the following assumptions: (1) Eachnode represents a location (in real latitude-longitude) wherethe road curves or where there is a choice of edges to travelon; (2) Each edge represents a straight-line segment of anywalkway or a subway; (3) All people and vehicles are con-strained to move only along the edges of the graph; (4) Thearea between streets housing buildings and the area in parkswhich do not have walkways are deemed unusable for anykind of transportation, even in an emergency; (5) All edgesare assumed to be bidirectional. The intersection pointswere computed assuming that all roads, including flyoversand bridges, intersect all roads that they cross, irrespectiveof altitude difference. The subway stops were approximatedto the nearest node on the graph. The graph is non-planarbecause of the subway lines, which are mostly underground

110

in Manhattan. The locations of all major hospitals and someminor hospitals, in all 22 medical facilities, were also approx-imated to the nearest node on the graph.

3.1.2 Traffic ModelingAverage speed statistics that were available were inte-

grated into a simplistic traffic model. The on-site treatmentteams travel at a fixed speed initialized to a random valuebetween 7 and 10 miles per hour. Subways have a fixedspeed of 13 miles per hour. Each person has a maximumpossible speed initialized to a random value between 6 and9 miles per hour, consistent with average traffic speed inMidtown Manhattan. To account for congestion, effect ofill-health on mobility and other probabilistic effects, at eachtime instant, a person travels at an effective speed given by:

if(U(0,1) < 1.0-health)effective speed = 0.0;

elseeffective speed =U(health * maximum speed / 2.0, maximum speed);

where U(0, 1) is a real random number generated uniformlyin the range (0, 1). No congestion or road width is captured,so there is no enforced maximum number of people at a nodeor on an edge.

3.2 The People at Port AuthorityA “Person” is the most fundamental agent in our multi-

agent model, representing the class of individuals exposedto Sarin. However, by-standers and the general populationof Manhattan are assumed to play no role (not modeled);same is the case with people and organizations outside theisle of Manhattan.

3.2.1 Person’s ParametersBased on studies [6, 9]of factors influencing a person’s re-

sponse to a disaster scenario, the following attributes werechosen to be incorporated into our model: (1) State: headedto original destination or to a hospital; (2) Facts: cur-rent health level (Hl), currently being treated at a hospitalor not, current “amount” of medication / treatment, ac-cess to a long-distance communication device, probabilityof the communication device working when the person triesto use it (information update rate); (3) Knowledge: loca-tion and current capacities of known hospitals and on-sitetreatment units, time of last-update of this information, ta-bles of the LRTA∗ estimates for the known nodes, list of100 most recently visited nodes; (4) Personality: degree ofworry (Wl), level of obedience (Ol), perceived level of dis-tress (D = Wl×(1−Hl)). The obedience parameter Ol cap-tures the instruction-abiding trait of a person, and affectsthe decision to head to a hospital. The worry parameterWl represents the innate level of irrationality in the agent’sbehavior, and affects the following decisions: when to go toa hospital, when to get information from neighbors or viacell phone, or how to select the hospital.

3.2.2 Rules of BehaviorThe person’s initial goal is to reach the original destina-

tion (e.g., home or place of work) from the initial location(the Port Authority Bus Terminal). However, after expo-sure to Sarin, his/her health begins to deteriorate. At acertain health-level decided by environmental and person-

ality factors, the person changes the destination state to ahospital:

if(U(0,1) < Obedience) {if (health < unsafe health level)

Head to a hospital}else if (U(0,1) < distress level))

Head to a hospital}

where the unsafe health level is the suggested health levelwhen a person should head to a hospital.

Initially, each person agent knows only a random numberof hospitals and their absolute positions in the map (latitudeand longitude), but this knowledge gets updated during theevolution of a simulation using the different communicationchannels (described in Section 3.5):

if (heading to a hospital && U(0,1) < distress level) {if (U(0,1) < information update rate)

Get current hospital information via phone/radioelse

Talk to neighbors}

The choice of hospital is then made based on the list ofhospitals and on-site treatment facilities known, their cur-rent capacities, and personality and environmental factors:

if(U(0,1) < distress level) {Find nearest hospital

} else {Find nearest hospital in available mode

}

After being treated and cured at a medical facility, the per-son resumes moving towards his/her original destination.

3.2.3 LRTA∗ with Ignore-List for Route FindingThe Learning Real-Time (LRTA∗) algorithm, proposed

by Korf in 1990 [5], interleaves planning and execution in anon-line decision-making setting. In our model, the person-agent is modeled as maintaining an “ignore-list” of the last100 nodes he/she visited, and employs the following modifiedLRTA∗ algorithm:

1. Default: If all neighbors of the current node i are inthe ignore list, pick one randomly.

2. Else:

(a) Look-Ahead: Calculate f(j) = k(i, j) + h(j) foreach neighbor j of the current node i that is notin the ignore-list. Here, h(j) is the agent’s cur-rent estimate of the minimal time-cost requiredto reach the goal node from j, and k(i, j) is thelink time-cost from i to j, which depends on thetype of the link (road or subway) and its effectivespeed (subway or person speed).

(b) Update: Update the estimate of node i as fol-lows:

h(i) = max{h(i), minj∈Next(i)

f(j)}

(c) Action Selection Move towards the neighbor jthat has the minimum f(j) value.

111

As the planning time for each action executed by the agentis bounded (constant time), the LRTA∗ algorithm is knownto be usable as a control policy for autonomous agents, evenin an unknown or non-stationary environment.However, therational LRTA∗ algorithm was inappropriate in its directform for modeling persons trying to find the route to theiroriginal destination or hospital in an atmosphere of tensionand panic. Thus, the ignore-list was introduced to capture acommon aspect of panic behavior: people seldom return toa previously visited node when an unexplored node is avail-able. In other words, the only case when a person uses oldlearnt information is when they revisit a node they visitedover a hundred nodes ago. The algorithmic characteristics ofthis “ignore-list” heuristic are being investigated separately.

3.3 The Medical Facilities in ManhattanThe hospital agent is a stationary agent that is an ab-

straction of any medical facility that can play a role at thetime of a catastrophe. Twenty two major and minor hos-pitals have been included, and the number of hospital bedswas used as an indicator of the capacity (“resources”) of thehospital.

3.3.1 Hospital’s ParametersThe attributes of a hospital that are included in our model

are: (1) State: available, critical or full; (2) Facts: resourcelevel (representing both recoverable resources like doctors,nurses and beds, and irrecoverable resources like drugs andsaline), reliability of communication device (information up-date rate); (3) Knowledge: locations and current capacitiesof known hospitals; (4) Triage Behavior: health-levels be-low which a person is considered critical, non-critical or dis-chargeable.

3.3.2 Rules of BehaviorAs described in our Brazilian scenario model [7], the hos-

pital operates in three modes: “available”, “critical” and“full”. When a hospital’s resource level drops below the low

resource level ( 13

rdof initial resources), its mode changes

from available to critical. When a hospital’s resource level

drops below the very low resource level ( 110

thof initial re-

sources), its mode changes from critical to full. The hospitalmode directly influences the key decisions: whom to turnaway, whom to treat and how much resources to allocateto a person requiring treatment. The medical parlance forthis process is “triage”, and research is actively being con-ducted to evaluate different triage policies appropriate todifferent scenarios (for example, see the Simple Triage andRapid Treatment system [10]). The hospital’s behavior ateach time step is described by the following rules:

Treat all admitted patientsfor all persons inside the hospital{

if (health >= dischargeable health level)Discharge person

else if(person is waiting for admission) {if(hospital is in available mode)

Admit and treat the personelse if(hospital is in critical mode &&

health < critical health level)Admit and treat the person

}if (person is waiting &&

health < critical health level)Add to critical list

if (person is admitted &&health > non-critical health level)Add to non-critical list

}Discharge non-critical patients, admit critically ill

3.4 On-Site Treatment UnitsOn-site treatment is provided by Major Emergency Re-

sponse Vehicles (MERVs) which set up their units close tothe site of action. The HazMat Team consists of expertstrained in handling hazardous materials, who rescue peo-ple from the contaminated zone, collect samples for testing,and eventually decontaminate the area. In our model, wegroup HazMat and MERVs into one unit – “on-site treat-ment providers”. These small mobile hospitals are initiallyinactive and stationary at their hospital of affiliation. Whennotified of the attack, they move towards the catastrophesite. Their properties include: (1) Facts: starting location,time of dispatch; (2) Knowledge: locations and current ca-pacities of known hospitals; tables of the LRTA∗ estimatesfor the known nodes, list of 100 most recently visited nodes;(3) Behavior: exactly the same as a hospital in “critical”mode;

The model for which the statistics are reported in thispaper has 5 on-site treatment providers. In a real situation,the first responders to the emergency include the Police andFire department personnel. Ambulances arrive at the sceneand transport sick people to the hospitals. No ambulance-like services are currently part of the model. The role of thepolice in cordoning the area and crowd management is im-plicit in that on-lookers and by-standers do not complicatethe disaster management process in our model.

3.5 Communication ChannelsIn the model analyzed in this paper, only the information

about the hospital and on-site treatment provider locationsand capacities are communicated dynamically. The channelof communication used for on-site treatment provider acti-vation is not modeled; only the time of availability of the in-formation is controlled. The communication channels avail-able are: one-to-one between persons and any of the otherthree classes of agents adjacent to them, one-to-many fromthe hospital to all persons within its premises, and many-to-many from the hospitals to all other hospitals, persons andon-site treatment units with access to a public telephone,radio or a mobile communication device. The role of media,internet, misinformation and rumors are not modeled.

3.6 Sarin Gas Exposure

3.6.1 Time-course of Deterioration and RecoveryThe time-course variation of the health level (with and

without treatment) after the exposure is modeled using a 3-step probabilistic function depending on the person’s currenthealth level.

if (U(0,1) < health)health = health

+ U(0, treatment + maximum untreated recovery);else

worsening = (health > dangerous health level)?maximum worsening:((health > critical health level)?

maximum dangerous worsening:maximum critical worsening))

health = health - U(0,(1 - treatment)*worsening);

112

Table 1: Exposure level and health level rangesExposure level Health range People Exposed

High (lethal injuries) (0.0, 0.2] 5%Intermediate (severe injuries) (0.2, 0.5] 25%

Low (light injuries) (0.5, 0.8] 35%No symptoms (0.8, 1.0) 35%

The exact values used are dangerous health level = 0.5, crit-ical health level = 0.2, maximum worsening = 1.38 ∗ 10−4

per minute, maximum dangerous worsening = 4.16 ∗ 10−4

per minute and maximum critical worsening = 6.95 ∗ 10−4

per minute.

3.6.2 Level of ExposureBased on diffusion effects, air-currents, number of peo-

ple, temperature, time of day, rate of breathing and amountof time exposed to Sarin, the amount of Sarin inhaled bya person (“acquired dose”) at a certain distance from thesource can be estimated. Based on this dosage, a certainhealth response results (based on “dose-response curves” intoxicology). Unfortunately, it is impossible to estimate thenature, intensity and location of an attack (even within thePort Authority Bus Terminal). More importantly, there isno clear-cut data on the rate of health degradation afterexposure to a certain dosage. This is significant, as the ul-timate aim of the modeling is to see how the time takenby the on-site responder units to initiate treatment com-pares with the time taken by the Sarin poisoning to resultin death. Reasonable estimates for the rate of health deteri-oration were arrived at in consultation with toxicologists inthe CCPR team and based on related literature [10, 4]. Ta-ble 1 shows the four main classes of exposure that have beenmodeled, the corresponding ranges of initialization for thehealth level and the percentage of people initialized to thatcategory. These values reflect our attempt to capture thegeneral situation of previously documented events[8], whereonly a small fraction of the affected population suffered fa-tal injuries. One key assumption in our model is that thereis no secondary exposure, i.e., on-site treatment units andhospital staff are not affected by treating Sarin-exposed pa-tients.

3.6.3 Chances of SurvivalThe actual survival chances under optimistic and pes-

simistic conditions that result from the assumptions of ourmodel are depicted in Figure 2. People with fatal and severeinjuries can survive if they are treated on-site or if they aretransported to a nearby hospital. People with light injuriesand those showing no symptoms will always recover eventu-ally, but in this case, the damage to organs and the time torecover are the correct metrics of effectiveness of the emer-gency response. However, in this paper, we focus only onthe number of deaths. As the survival-chances curve shows,only people with health less than 0.5 can ever die. How-ever, all persons factor in, as they decide how informationpercolates and how resources are distributed.

4. ANALYSIS OF SIMULATIONSSince no well-defined approaches exist for choosing the

correct level of abstraction and identifying the essential pa-rameters for modeling a scenario, a significant portion ofagent-based modeling remains an art more then a science.

0

200

400

600

800

1000

0 0.2 0.4 0.6 0.8 1

Num

ber

of fa

talit

ies

Health level

Worst caseBest case

Figure 2: Sarin: Treatment and Survival Chances

The assumptions used in our model, made in consultationwith experts from the CCPR team and based on related lit-erature, were often made for want of accurate data or forsimplification of the analysis. It is reiterated that the simu-lations cannot by themselves serve as factual outcomes, andso, emergency response planners are expected to integratescientific expertise, field exercises and historical data withthese simulation results to make sound decisions in real sce-narios.

The model has been implemented in the Java version ofRePast 3.1[2], a popular and versatile toolkit for multi-agentmodeling. In the results described below, the following ad-ditional assumptions were made: (1) The simulation is per-formed only for the first 3000 minutes (= 2 days and 2hours). The assumption is that people who survive thefirst two days are not likely to die. Further, by this timeresources from the outside the island of Manhattan will be-come available and the scenario is beyond the scope of ourcurrent model; (2) Neither an on-site responder nor a hospi-tal can help a person if the person does not ask for treatment(“head to a hospital” mode); (3) None of the behavior pa-rameters change during a simulation, as learning behavioris supported only for the route finding algorithm. Unlessstated otherwise, all plots involve 1,000 people, 22 hospi-tals, and 5 on-site responder teams. Every point that isplotted is the average of 10 independent runs. All plotswithout responders start at a slightly different initial state(with identical stochastic properties).

4.1 People Behavior

4.1.1 Unsafe Health LevelA critical disaster management question is: When should

a person experiencing symptoms go to a hospital? Considerthe scenario when there are no on-site treatment units. InFigure 3, the influence of the health-level at which a persondecides to go to a hospital (called “unsafe health level”) onthe number of deaths is visualized. This plot suggests thatperson should decide to go to a hospital when his or herhealth approaches 0.2.

This unexpectedly low optimum value reflects a skewedhealth scale and can be explained thus. From Figure 2 weobserve that if the health level is greater than 0.1, almost95% of the people will recover fully with treatment, while ifthe health level is greater than 0.5, 100% of them will re-cover even without any treatment. When the unsafe health

113

20

30

40

50

60

70

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

Num

ber

of fa

talit

ies

Unsafe health level

No triage with first respondersWith triage with first responders

No triage no first-respondersWith triage no first-responders

Figure 3: Persons heading to a hospital with andwithout on-site treatment units (number of on-siteresponders = 5, on-site responder’s dischargeablehealth level = 0.5, hospital’s dischargeable healthlevel = 0.8, responder alert time = 15 minutes).

level is too low (< 0.2), people have been instructed to waitso much that their condition turns fatal. The second fac-tor affecting the optimum value for heading to a hospitalis the distribution of people across the different classes ofinjuries. As seen in Table 1, a cut-off of 0.2 ensures thatonly the people who experienced lethal injuries (50/1000)go to a hospital. The moment this cut-off if increased, tosay 0.5, crowding effects hamper emergency response as an-other 250 severely injured persons also rush to the hospitals.This situation is exacerbated by the fact that health levelgoverns mobility, and hence healthier people are expected toreach a hospital earlier than sicker people. Thus, when un-safe health level is high (> 0.2), people who do not requiremuch emergency treatment end up consuming a share of theavailable resources, which would have been better spent onthe sicker people already at the hospital or on persons whoare still on their way to the hospital. Clearly, the presenceof ambulances would alter the situation as the lethally in-jured persons would actually move faster than persons ofall other classes. The drop in death rate after 0.6 can beattributed to the fact that people with health level greaterthan 0.6 would have recovered by themselves ( see Fig. 2)on the way to the hospital, and hence may have not appliedany pressure on the hospital resources.

The number of deaths due to crowding is dramaticallymitigated if there are on-site treatment units, as seen inFigure 3. It is to be recalled that from the point of view of aperson, an on-site treatment unit is equivalent to a hospitalin “critical” mode. The number of deaths due to peopleheading to hospitals earlier than necessary is less, as most ofthese very sick people are now treated on-site, and hence, areno longer dependent on the resources of the hospitals. Whena person’s health level is greater than the unsafe health level,in addition to not heading to a hospital, the person refusestreatment even from an on-site treatment provider. Thoughthis assumption is unrealistic when the person’s health isless than 0.2 or so, it is plotted for completeness.

4.1.2 Worry and ObedienceTwo significant personality parameters that affect disaster-

time behavior of a person are the innate degree of worry and

30 40 50 60 70 80 90 100

0 0.2 0.4 0.6 0.8 1

0 0.2

0.4 0.6

0.8 1

20 30 40 50 60 70 80 90

100

Number of fatalities

Worry levelObedience level

Number of fatalities

Figure 4: Effect of people’s obedience and worrylevels (hospital’s dischargeable health level = 0.8).

0

50

100

150

200

250

300

350

400

0 200 400 600 800 1000

Num

ber

of fa

talit

ies

Hospital resource level

Figure 5: The effect of having more resources

obedience (see Sec. 3.2.2). These population parameters canbe controlled by education, awareness and training before anevent, and also by employing law enforcement officers dur-ing the emergency response. Obedient persons do not headto a hospital when their health level is above what is con-sidered unsafe, while disobedient persons will go based ontheir perceived level of distress. In order to understand theirinfluence on the global system behavior, a set of simulationswere performed by varying both Ol and Wl in the range [0, 1]and assuming that on-site responders are not active. Figure4 shows the results of their mutual interaction. By our defi-nition of obedience and worry, disobedient worrying personswill head to the nearest hospital too early, thus crowding themost critical resource. At the other extreme, obedient peo-ple who are not worried choose to go to a hospital only whenthey are really sick, and also distribute themselves betweenthe different hospitals; only when they become critically illdo they go to the nearest hospital irrespective of its mode.Disobedient people who are not worried do not worsen thesituation because they will still get hospital information andchoose to go to one, only when necessary (based on level ofill-health).

4.2 Hospital Behavior

4.2.1 Resource RequirementsThe meaning of the “resource” parameter is clarified in

Figure 5. The thought experiment that lead to this plot was:

114

30

35

40

45

50

55

60

65

70

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Num

ber

of fa

talit

ies

Dischargeable health level

No triageWith triage

Figure 6: Hospital’s patient-discharge behaviorwithout on-site treatment (Person’s unsafe healthlevel = 0.2).

when there is only one hospital, and the Sarin attack occursimmediately adjacent to it, how much resources are neces-sary to save the 1000 affected people? As the plot shows,if the hospital has resources > 100.0, then no more than 50deaths can result. A resource level > 200.0 can bring thenumber down between 40 or 20. The number of deaths isnever zero because the personality parameters make differ-ent people choose to head to the hospital at different times.

4.2.2 Optimal Dischargeable Health LevelThe hospital’s decision to discharge a patient is dictated

by its estimate of whether the patient can recover using justmedication, as opposed to requiring continuous monitoring.In our model, the hospital discharges persons whose healthlevel is greater than “dischargeable health level”. In Fig-ure 6, the relationship of this decision with the number ofdeaths is plotted, and is seen to follow the same pattern asthe “unsafe health level”. When the dischargeable healthlevel is too low, the person dies after being discharged pre-maturely. When it is too high, the person is given moremedical attention than necessary and effectively decreasesthe chances of survival of sicker persons.

It is not immediately clear why the death-rate drops whenthe dischargeable health level is greater than 0.6. One pos-sible explanation is that a person so discharged always re-covers fully, whereas a fraction of the people discharged ear-lier return for treatment, possibly to a different hospital.The peak near 0.0 of 50 deaths is less than the peak near0.6 of 65 deaths. This is because the hospital in reality isnot entirely refusing treatment to persons with health levelgreater than dischargeable health level. The person is givensome treatment, then discharged and then readmitted untilthe person’s health becomes greater than the unsafe healthlevel, at which point he/she “accepts” the hospital’s decisionto discharge him/her and resumes moving towards his/heroriginal destination. Also, unpredictable behaviors can re-sult when the linear ordering of the parameters (0 < criti-cal health level < non-critical health level < dischargeablehealth level < 1) is violated.

The behaviors with and without triage not being verydifferent may be related to the fact that hospitals broadcasttheir mode irrespective of whether they are enforcing triagepolicies or not. Persons use this information to choose the

15

20

25

30

35

40

45

0 5 10 15 20 25 30 0 20

40 60

80 100

120

15

20

25

30

35

40

45

Number of fatalities

Number of first respondersAlert time

Number of fatalities

Figure 7: Number of on-site responders versus theiralert time (on-site responder’s dischargeable healthlevel = 0.5, hospital’s dischargeable health level =0.8, person’s unsafe health level = 0.4).

hospital. Since we are counting only the number of deathsand since the very sick people go to the nearest hospitalirrespective of triage enforcement, only the difference in thebehavior of the hospital affects the result. However, in thecritical mode, the hospital admits all persons with healthlevel less than the critical health level (= 0.25). Thus thedifferences are minimal when the triage is enforced and thehospital is in the critical or available mode. The differencewould have been noticeable had the hospitals been smalleror had the number of people been more; then the hospitalswould have moved to “full” mode refusing admission to eventhe critically ill.

4.3 Role of On-Site Treatment ProvidersThe role of the on-site treatment providers is patent in the

parameter surface (Figure 7) of their alert time versus thenumber of fatalities. As expected, the plot also shows a near-linear dependence of the system on the alert time. However,beyond the 10 responders that seem to be required, the ef-fect on the improvement in the number of survivors is lessevident. Clearly, the bound on the number of dying peoplethat can actually be saved causes this surface flattening.

4.4 Significance of Communication

4.4.1 Getting Current Hospital InformationWe modeled the scenario where every person has a com-

munication device, and then controlled the rate of informa-tion update (which can capture the difficulty of access to acell-phone, taxi-phone or public phone booth, the congestednature of the communication network, the lack of responsefrom the callee, etc.). The impact of this parameter on thenumber of fatalities is plotted in Figure 8. As observed inthe Brazilian scenario analysis[7] also, the death rate de-clines when more people have complete hospital informa-tion. When everybody has access to the current informa-tion about all hospitals, healthier people, who would havesurvived the commute to a farther hospital, consume theresources of the nearby hospitals which they quickly reach.Critically ill people, whose average speed is much lower, areeffectively left with a choice only between critical or fullproximal hospitals and available distant hospitals – both ofwhich turn out to be fatal.

115

34

36

38

40

42

44

46

48

50

52

54

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Num

ber

of fa

talit

ies

Information update rate

Figure 8: Person’s ability to communicate (person’sunsafe health level = 0.2).

4.4.2 Activating the On-Site RespondersThe success of the on-site treatment responders is depen-

dent on how quickly they get alerted, as shown in Figure7. As a result of our parameter choice, we see that the netnumber of fatalities is stable (∼ 25), as long as the on-siteresponders arrive within 50 minutes. The fluctuations couldbe due to the fact that the persons are themselves movingand need to be able to locate the on-site responder.

5. DISCUSSIONSeveral important emergency response issues, such as when

to head to a hospital, when to discharge a person, number ofon-site treatment units necessary, the importance of publicawareness and law enforcement, the role of responder sizeand activation time, and the diffusion of information abouthospitals and capacities, were amenable to analysis by re-peated simulation. ABM shows tremendous potential as asimulation-based tool for aiding disaster management pol-icy refinement and evaluation, and also as a simulator fortraining and designing field exercises.

The “Sarin in Manhattan” model in itself can be extendedby addressing the assumptions discussed earlier. On thecomputational side, better knowledge and belief-state rep-resentation are necessary to simplify and generalize the com-munication mechanisms. Further, this will lead to simplerencoding of learning behavior; thus all parameters, includ-ing personality states, should be able to evolve with expe-rience. We modified the simple LRTA∗ algorithm to takeinto account the memory of recently visited nodes to ap-proximate real human panic behavior. This model needsto be refined, and more personality and learnt parametersneed to be factored in. Another aspect that is missing in ourmodel is the information about routes and location of sub-way stops. After modeling traffic congestion, the role of acentralized navigation system [12] in managing disaster-timetraffic and routing also warrants investigation. To improvethe ultimate utility of the tool, we need to devise a uni-form way of describing different catastrophic scenarios, withthe ability to validate over well-documented real instances.Further, a conventional AUML-based description of agentbehavior needs to be the input for the system. Some ofthe specific scenarios we hope to model in the near futureinclude food-poisoning, moving radioactive cloud, communi-cable diseases, natural disasters leading to resource damage

in addition to disease, and events requiring evacuation orquarantine. On the theoretical side, we would like to auto-mate the process of policy evaluation and comparison, andoptimal parameter value estimation. We are also investi-gating representations of plans so that multi-objective opti-mization via evolutionary algorithms can be used to designnew emergency response strategies. To address cultural andracial differences in response to catastrophes, game-theoreticbehavior modeling and analysis is being surveyed [1].

6. ADDITIONAL AUTHORSAdditional authors: Lewis Nelson (NYU School of Medicine,

email: lnelson@pol.net), Dianne Rekow (NYU College ofDentistry, email: edr1@nyu.edu), Marc Triola (NYU Schoolof Medicine, email: marc.triola@med.nyu.edu), Alan Shapiro(NYU School of Medicine, email: alan.shapiro@med.nyu.edu),Clare Coleman (NYU CCPR, email: colemc01@med.nyu.edu),Ofer Gill (NYU, email: gill@cs.nyu.edu) and Raoul-SamDaruwala (NYU, email: raoul@cs.nyu.edu).

7. REFERENCES[1] G. Boella and L. W. N. van der Torre. Enforceable

social laws. In AAMAS 2005, pages 682–689. ACM,2005.

[2] N. Collier, T. Howe, R. Najlis, M. North, and J. Vos.Recursive porous agent simulation toolkit, 2005.

[3] E. A. der Heide. The importance of evidence-baseddisaster planning. Annals of Emergency Medicine,47(1):34–49, 2006.

[4] C. L. Ernest. Clinical manifestations of sarin nerve gasexposure. JAMA, 290(5):659–662, 2003.

[5] R. Korf. Real-time heuristic search. ArtificialIntelligence, 42:189–211, 1990.

[6] R. Lasker. Redefining readiness: Terrorism planningthrough the eyes of the public, 2004.

[7] V. Mysore, O. Gill, R.-S. Daruwala, M. Antoniotti,V. Saraswat, and B. Mishra. Multi-agent modelingand analysis of the brazilian food-poisoning scenario.In The Agent Conference, 2005.

[8] T. Okumura, K. Suzuki, A. Fukuda, A. Kohama,N. Takasu, S. Ishimatsu, and S. Hinohara. The tokyosubway sarin attack: disaster management, part 1:Community emergency response, part 2: Hospitalresponse. Academic Emergency Medicine, 5(6):613–24,Jun 1998.

[9] PNAS. Adaptive Agents, Intelligence, and EmergentHuman Organization: Capturing Complexity throughAgent-Based Modeling, volume 99(3). May 2002.

[10] Y. Tokuda, M. Kikuchi, O. Takahashi, and G. H.Stein. Prehospital management of sarin nerve gasterrorism in urban settings: 10 years of progress afterthe tokyo subway sarin attack. Resuscitation,68:193–202, 2006.

[11] I. van Heerden and B. A. Hurricane pam exercise -july 2004, adcirc storm surge animations and relatedinformation, 2004.

[12] T. Yamashita, K. Izumi, K. Kurumatani, andH. Nakashima. Smooth traffic flow with a cooperativecar navigation system. In AAMAS 2005, pages478–485. ACM, 2005.

116

Wearable Computing meets Multiagent Systems: Areal-word interface for the RoboCupRescue simulation

platform

Alexander KleinerInstitut fur InformatikUniversitat Freiburg

79110 Freiburg, Germany

kleiner@informatik.uni-freiburg.de

Nils BehrensTechnologie-Zentrum

InformatikUniversitat Bremen

28359 Bremen, Germany

psi@tzi.de

Holger KennTechnologie-Zentrum

InformatikUniversitat Bremen

28359 Bremen, Germany

kenn@tzi.de

ABSTRACTOne big challenge in disaster response is to get an overviewover the degree of damage and to provide this information,together with optimized plans for rescue missions, back toteams in the field. Collapsing infrastructure, limited visibil-ity due to smoke and dust, and overloaded communicationlines make it nearly impossible for rescue teams to report thetotal situation consistently. This problem can only be solvedby efficiently integrating data of many observers into a singleconsistent view. A Global Positioning System (GPS) devicein conjunction with a communication device, and sensorsor simple input methods for reporting observations, offer arealistic chance to solve the data integration problem.

We propose preliminary results from a wearable computingdevice, acquiring disaster relevant data, such as locationsof victims and blockades, and show the data integrationinto the RoboCupRescue Simulation [8] platform, which isa benchmark for MAS within the RoboCup competitions.We show exemplarily how the data can consistently be in-tegrated and how rescue missions can be optimized by so-lutions developed on the RoboCupRescue simulation plat-form. The preliminary results indicate that nowadays wear-able computing technology combined with MAS technologycan serve as a powerful tool for Urban Search and Rescue(USAR).

KeywordsWearable Computing, GPS, Multi Agent Systems, MAS,USAR, GIS, RoboCupRescue

1. INTRODUCTIONOne big challenge in disaster response is to get an overviewover the degree of damage and to provide this information,together with optimized plans for rescue missions, back toteams in the field. Collapsing infrastructure, limited visibil-ity due to smoke and dust, and overloaded communicationlines make it nearly impossible for rescue teams to reportthe total situation consistently. Furthermore, they might beaffected psychologically or physically by the situation itselfand hence report unreliable information.

This problem can only be solved by efficiently integratingdata of many observers into a single consistent view. A

Global Positioning System (GPS) device in conjunction witha communication device, and sensors or simple input meth-ods for reporting observations, offer a realistic chance tosolve the data integration problem. Furthermore, an inte-grated world model of the disaster allows to apply solutionsfrom the rich set of AI methods developed by the Multi-Agent Systems (MAS) community.

We propose preliminary results from a wearable computingdevice, acquiring disaster relevant data, such as locationsof victims and blockades, and show the data integrationinto the RoboCupRescue Simulation [8] platform, which isa benchmark for MAS within the RoboCup competitions.Communication between wearable computing devices andthe server is carried out based on the open GPX protocol [21]for GPS data exchange, which has been extended for addi-tional information relevant to the rescue task. We show ex-emplarily how the data can consistently be integrated andhow rescue missions can be optimized by solutions devel-oped on the RoboCupRescue simulation platform. The pre-liminary results indicate that nowadays wearable computingtechnology combined with MAS technology can serve as apowerful tool for Urban Search and Rescue (USAR).

RoboCupRescue simulation aims at simulating large-scaledisasters and exploring new ways for the autonomous coor-dination of rescue teams [8] (see Figure 1). These goals leadto challenges like the coordination of heterogeneous teamswith more than 30 agents, the exploration of a large-scale en-vironment in order to localize victims, as well as the schedul-ing of time-critical rescue missions. Moreover, the simulatedenvironment is highly dynamic and only partially observableby a single agent. Agents have to plan and decide their ac-tions asynchronously in real-time. Core problems are pathplanning, coordinated fire fighting, and coordinated searchand rescue of victims. The solutions presented in this paperare based on the OpenSource agent software [1], which wasdeveloped by the ResQ Freiburg 2004 team [9], the winner ofRoboCup 2004. The advantage of interfacing RoboCupRes-cue simulation with wearable computing is twofold: First,data collected from a real interface allows to improve thedisaster simulation towards disaster reality. Second, agentsoftware developed within RoboCupRescue might be advan-tageous in real disasters, since it can be tested in many sim-

117

Figure 1: A 3D visualization of the RoboCupRescue

model for the City of Kobe, Japan.

ulated disaster situations and can also directly be comparedto other approaches.

Nourbakhsh and colleagues utilized the MAS Retsina formixing real-world and simulation-based testing in the con-text of Urban Search and Rescue [15]. Schurr and col-leagues [17] introduced the DEFACTO system, which en-ables agent-human cooperation and has been evaluated inthe fire-fighting domain with the RoboCupRescue simula-tion package. Liao and colleagues presented a system thatis capable of recognizing the mode of transportation, i.e., bybus or by car, and predicting common travel destinations,such as the office location or home location, from data sam-pled by a GPS device [12].

The remainder of this paper is structured as follows. Wepresent an interface between human rescue teams and therescue simulator in Section 2. In Section 3 we give some ex-amples how approaches taken from MAS can be utilized fordata integration and rescue mission optimization. In Sec-tion 4 we propose preliminary experiments from integratingdata into RoboCupRescue from a real device and concludein Section 5.

2. INTERFACING REAL RESCUE2.1 Requirement analysisIn wearable computing, one main goal is to build devicesthat support a user in the primary task with little or noobstruction. Apart from the usual challenges of wearablecomputing [20, 19], in the case of emergency response, thesituation of the responder is a stressful one. In order toachieve primary task support and user acceptance, specialattention has to be given to user interface design. For thisapplication, the user needs the possibility to enter informa-tion about perceptions and needs feedback from the sys-tem 1. Furthermore, the user needs to receive task-relatedinstructions from the command center.

The implementation has to cope with multiple unreliablecommunication systems such as existing cell phone net-works, special-purpose ad-hoc communication and existing

1Technically, this feedback is actually not required by theapplication, but we envision that it will improve user accep-tance.

emergency response communication systems. As the anal-ysis of the different properties of these communication sys-tems is beyond the scope of this article, we will thereforeabstract from them and assume an unreliable IP-based con-nectivity between the mobile device and a central commandpost. This assumption is motivated by the fact that bothinfrastructure-based mobile communication networks andcurrent ad-hoc communication systems can transport IP-based user traffic.

For mobile devices, a number of localization techniques areavailable today, for an overview see [6]. Although someinfrastructure-based communication networks are also ca-pable of providing localization information of their mobileterminals, we assume the presence of a GPS-based local-ization device. The rationale behind this is that the local-ization information provided by communication systems isnot very precise (e.g., sometimes limited to the identifica-tion of the current cell, which may span several square kilo-meters) and therefore not usable for our application. TheGPS system also has well-known problems in urban areasand in buildings. But based on additional techniques suchas the ones stated in [11], its reliability and accuracy canbe sufficiently improved. Particularly the coexistence of aGPS device with an Internet connection allows to utilizeInternet-based Differential GPS, which leads to a position-ing accuracy of decimeters [2].

The situation of the device and its user is also characterizedby harsh environmental conditions related to the emergencyresponse, such as fire, smoke, floods, wind, chemical spillingsetc. The device has to remain operable under such condi-tions, and moreover has to provide alternative means of in-put and output under conditions that affect human sensingand action abilities. As these requirements are quite com-plex, we decided to design and implement a preliminary testsystem and a final system. The components of the two sys-tems and their interconnections can be found in Figure 4.

2.2 A preliminary test systemIn order to analyze the properties of the communication andlocalization systems, a preliminary test system has been im-plemented, for which two requirements have been dropped,the design for harsh envionmental conditions and the abilityto use alternative input and output.

The communication and localization system is independentof the user requirements with the exception of the fact thatthe system has to be portable. Therefore we chose a mobileGPS receiver device and a GSM cell phone device as ourtest implementation platform. The GPS receiver uses thebluetooth [3] personal area network standard to connect tothe cell phone. The cell phone firmware includes a Java VMbased on the J2ME standard with JSR82 extensions, i.e.,a Java application running on the VM can present its userinterface on the phone but can also directly communicatewith bluetooth devices in the local vicinity and with Internethosts via the GSM networks GPRS standard.

The implementation of the test application is straightfor-ward: It regularly decodes the current geographic positionfrom the NMEA data stream provided by the GPS receiverand sends this information to the (a priori configured) server

118

IP address of the central command center. The utilizedprotocol between the cell phone and the command centeris based on the widely used GPX [21] standard for GPSlocations. Among other things, the protocol defines datastructures for tracks and waypoints. A track is a sequence oflocations with time stamps that has been visited with theGPS device. A waypoint describes a single location of inter-est, e.g., the peak of a mountain. We extended the protocolin order to augment waypoint descriptions with informationspecific to disaster situations. These extensions allow res-cue teams to report the waypoint-relative locations of roadblockades, building fires, and victims. Currently, the wear-able device automatically sends the user’s trajectory to thecommand center, whereas perceptions can manually be en-tered. A detailed description of the protocol extension canbe found in Appendix A.

2.3 Designing the full emergency response wear-able system

In order to fulfill the additional requirements for robustnessand user interface, the full system will be based on additionalhard- and software. The system uses a wearable CPU core,the so-called qbic belt-worn computer [4] (see Figure 3 (a)).It is based on a ARM CPU running the Linux operatingsystem, has a bluetooth interface, and can be extended viaUSB and RS232 interfaces. The wearable CPU core runsthe main application program. For localization, the samemobile GPS receiver as in the test system is used, but canbe replaced by a non-bluetooth serial device for increasedreliability. For communication, the system can use multi-ple communication channels whose already used GSM cellphone can be one of those 2.

As already stated, the design of the user interface is a crucialone for this application. Therefore, we envision a user inputdevice integrated in the clothing of the user, e.g., an arm-mounted textile keyboard [13] and a wireless link of the key-board to the belt computer. Such an interface has alreadybeen designed for other applications such as aircraft cabinoperation [14] (see Figure 2). Due to the harsh environmen-

Figure 2: A textile keyboard for aircraft cabin op-

eration.

tal conditions, we plan two independent output devices forinformation output and user feedback. A bluetooth head-set device provides audible feedback for user input, and atext-to-speech engine provides audible text output.

The second output device is a head-mounted display thatcan be integrated into existing emergency response gear such

2As we assumed IP-based connectivity, flexibleinfrastructure-independent transport mechanisms suchas MobileIP [16] can be used to improve reliability overmultiple independent and redundant communication links.

(a)

(b) (c)

Figure 3: The qbic belt-worn computer : (a) The belt

with CPU. (b) The head-mounted display. (c) Both

worn by the test person.

as firefighter helmets and masks (see Figure 3(b)). In appli-cations where headgear is not commonly used, the outputcan also be provided through a body-worn display device.

The application software driving the user interface is basedon the so-called WUI toolkit [22], which uses an abstract de-scription to define user interface semantics independent ofthe input and output devices used. The application code istherefore independent of the devices available in a particularinstance of an implementation, i.e., with or without head-mounted display. The WUI toolkit can also take contextinformation into account, such as the user’s current situa-tion, in order to decide on which device and in what formoutput and input are provided.

(a)

(b)

Figure 4: System diagrams: (a) test system based

on a GSM phone (b) full system design based on a

belt-worn wearable computer

3. MULTI AGENT SYSTEMS (MAS) FORURBAN SEARCH AND RESCUE (USAR)

119

3.1 Data integrationGenerally, we assume that if communication is possible andnew GPS fixes are available, the wearable device of a rescueteam continuously reports the team’s trajectory as a trackmessage to the command center. Additionally, the rescueteam might provide information for specific locations, as forexample, indicating the successful exploration of a building,the detection of a victim, and the detection of a blockedroad, by sending a waypoint message.

Based on an initial road map and on the information onroad blockage and the autonomously collected data on tra-jectories traveled by the agents, the current system buildsup a connectivity graph indicating the connectivity of loca-tions. The connectivity graph between a single location andall other locations is constructed by the Dijkstra algorithm.The connectivity between two neighboring locations, i.e., theweight of the corresponding edge in the graph, depends onthe true distance, the amount of blockage, the number ofcrossings, and the number of other agents known to travelon the same route. In the worst case, the graph can be cal-culated in O (m + nlog (n)), where n is the number of loca-tions and m the number of connections between them. Theknowledge of the connectivity between locations allows thesystem to recommend “safe” routes to rescue teams and tooptimize their target selection. The sequence in Figure 5(a)shows the continuous update of the connectivity graph for abuilding within the simulated City of Foligno. Note that thegraph has to be revised if new information on the connectiv-ity between two locations is available, e.g if a new blockagehas been detected or an old blockage has been removed.

The search for victims of many rescue teams can only becoordinated efficiently if the rescue teams share informationon the exploration. We assume that rescue teams reportwhen they have finished to explore a building and whenthey have found a victim, by transmitting the accordingmessage to the command center. The command center uti-lizes this information to distribute rescue teams efficientlyamong unexplored and reachable locations. The sequencein Figure 5(b) shows an agent’s increasing knowledge on theexploration status of the map over time. Victims (indicatedby green dots) and explored buildings (indicated by whitecolor) are jointly reported by all agents. Regions that aremarked by a yellow border indicate exploration targets rec-ommended by the command center to the agent.

3.2 Rescue sequence optimizationTime is a critical issue during a real rescue operation. Ifambulance teams arrive at an accident site, such as a caraccident on a highway, it is common practice to optimizethe rescue sequence heuristically, i.e., to estimate the chanceof survival for each victim and to rescue urgent cases ear-liest. During a large-scale disaster, such as an earthquake,the efficient distribution of rescue teams is even more im-portant since there are many more victims and usually aninsufficient number of rescue teams. Furthermore, the timeneeded for rescuing a group of victims might significantlyvary, depending on the collapsed building structures trap-ping the victims.

In RoboCupRescue, victims are simulated by the three vari-ables damage, health and buridness, expressing an individ-

(a) (b)

Figure 5: Online data integration of information re-

ported by simulated agents: (a) The connectivity

between the blue building and other locations in-

creases over time due to removed blockades. White

colored locations are unreachable, red colored loca-

tions are reachable. The brighter the red color, the

better the location is reachable. (b) The agent’s

information on the explored roads and buildings

(green roads are known to be passable, green and

white buildings are known as explored). Regions

marked with a yellow border are exploration targets

recommended by the command center.

ual’s damage due to fire or debris, the current health thatcontinuously decreases depending on damage, and the diffi-culty of rescuing the victim, respectively. The challenge hereis to predict an upper bound on the time necessary to res-cue a victim and a lower bound on the time the victim willsurvive. In the simulation environment these predictions arecarried out based on classifiers which were induced by ma-chine learning techniques from a large amount of simulationruns. The time for rescuing civilians is approximated by alinear regression based on the buridness of a civilian and thenumber of ambulance teams that are dispatched to the res-cue. Travel costs towards a target are directly taken fromthe connectivity graph. Travel costs between two reachabletargets are estimated by continuously averaging costs expe-rienced by the agents 3.

We assume that in a real scenario expert knowledge canbe acquired for giving rough estimates on these predictions,i.e., rescue teams estimate whether the removal of debrisneeds minutes or hours. Note that in a real disaster sit-uation the system can sample the approximate travel timebetween any two locations by analyzing the GPS trajectoriesreceived from rescue teams in the field. Moreover, the sys-

3Note that the consideration of specific travel costs betweentargets would make the problem unnecessarily complex.

120

tem can provide for different means of transport, e.g., car orby feet, the expected travel time between two locations. Thesuccessful recognition of the means of transport from GPStrajectories was already shown by Liao and colleagues [12].

30

35

40

45

50

55

60

65

70

0 1 2 3 4 5 6 7 8 9

# C

ivili

ans

KobeEasy KobeHard KobeMedium KobeVeryHard RandomMapFinal VCEasy VCFinal VCVeryHard

Greedy-HeuristicGenetic Algorithm

Figure 6: The number of civilian suvivors if applying

a greedy rescue strategy and a GA optimized rescue

strategy within simulated cities

If the time needed for rescuing civilians and the chance ofsurvival of civilians is roughly predictable, one can estimatethe overall number of survivors by summing up the necessarytime for each single rescue and by determining the overallnumber of survivors within the total time. For each rescuesequence S = 〈t1, t2, ..., tn〉 of n rescue targets, a utility U(S)that is equal to the number of civilians that are expected tosurvive is calculated. Unfortunately, an exhaustive searchover all n! possible rescue sequences is intractable. A goodheuristic solution is to sort the list of targets according tothe time necessary to reach and rescue them and to subse-quently rescue targets from the top of the list. However, asshown in Figure 6, this might lead to poor solutions. A bet-ter method could be the so-called Hungarian Method [10],which optimizes the costs for assigning n workers to m tasksin O

`

mn2´

. The method requires that the time needed un-til a task is finished does not influence the overall outcome.However, this is not the case for a rescue task, since a vic-tim will die if rescued too late. Hence, we decided to uti-lize a Genetic Algorithm [7] (GA) for the optimization ofsequences and to utilize it for continuously improving therescue sequence executed by the ambulance teams.

The GA is initialized with heuristic solutions, for example,solutions that greedily prefer targets that can be rescuedwithin a short time or urgent targets that have only littlechance of survival. The fitness function of solutions is setequal to the sequence utility U(S). In order to guaranteethat solutions in the genetic pool are at least as good as theheuristic solutions, the so-called elitism mechanism, whichforces the permanent existence of the best found solution inthe pool, has been used. Furthermore, we utilized a simpleone-point-crossover strategy, a uniform mutation probabilityof p ≈ 1/n, and a population size of 10. Within each minute,approximately 300, 000 solutions can be calculated on a 1.0GHz Pentium4 computer.

We tested the GA-based sequence optimization on different

city maps in the simulation and compared the result with agreedy strategy. As can be seen in Figure 6, in each of thetested environments, sequence optimization improved theperformance of the rescue team. One important propertyof our implementation is that it can be considered as ananytime algorithm: The method provides at least a solutionthat is as good as the greedy solution, but also a better one,depending on the given amount of time.

4. PRELIMINARY EXPERIMENTSThe system has preliminary been tested by successively in-tegrating data received from a test person. The test personequipped with the test device described in Section 2 walkedseveral tracks within a district of the City of Bremen (seeFigure 7). During the experiment, the mobile device con-tinuously transmitted the trajectory of the test person. Ad-ditionally, the test person reported victim found waypointsafter having visual contact with a victim. Note that vic-tim waypoints were selected arbitrarily, since fortunately novictims were found in Bremen.

In order to integrate the data into the rescue system, thereceived data, encoded by the extended GPX protocol thatrepresents location by latitude and longitude, has to be con-verted into a grid-based representation. We utilized the Uni-versal Transverse Mercator (UTM) [18] projection system,which provides a zone for any location on the surface of theEarth, whereas coordinates are described relatively to thiszone. By calibrating maps from the rescue system to thepoint of origin of the UTM coordinate system, locations fromthe GPS device can directly be mapped. In order to copewith erroneous data, we decided to simply ignore outliers,i.e. locations far from the track, that were detected based onassumptions made on the test person’s maximal velocity. Inthe next version of the system it is planned to detect outliersbased on the mahanalobis distance estimated by a KalmanFilter, likewise as dead reckoning methods used in the con-text of autonomous mobile robots. Figure 7(b) shows thesuccessive integration of the received data into the rescuesystem and Figure 7(a) displays the same data plotted byGoogleEarth. Note that GPX data can be directly processedby GoogleEarth without any conversion.

5. CONCLUSIONWe introduced the preliminary design of a wearable de-vice which can be utilized for USAR. Furthermore we havedemonstrated a system which is generally capable of inte-grating trajectories and observations from many of thesewearable devices into a consistent world model. As shown bythe results of the simulation, the consistent world model al-lows the system to coordinate exploration by directing teamsto globally unexplored regions as well as to optimize theirplans based on the sampled connectivity of roads, and tooptimize the sequence of rescuing victims. The applicationof this coordination also in real scenarios, i.e., to send theroad graph and mission commands back to the wearable de-vices of real rescue teams in the field, will be a part of futurework.

As we can see from our experiments, the accuracy of theGPS locations suffices for mapping trajectories on a givenroad graph. However, during a real disaster, a city’s infras-tructure might change completely, i.e., former roads might

121

(a) (b)

Figure 7: Successive integration of data reported by a test person equipped with a wearable device. (a) The

real trajectory and observations of victims plotted with GoogleEarth (victims are labeled with “civFound”).

(b) The same data integrated into the rescue system (green roads are known to be passable, white buildings

are known as explored, and green dots indicate observed victims).

122

be impassable or disappear at all, and people search for newconnections between places (e.g., off-road or even throughbuildings). Therefore, it is necessary that the system is ca-pable of learning new connections between places and tomodify the existing graph accordingly. Bruntrup and col-leagues already studied the problem of map generation fromGPS traces [5]. Our future work will particularly deal withthe problem of learning from multiple noisy routes. Wewill extend the existing rescue system with the capability ofadding new connections to the road graph and to augmentthese connections with the estimated travel time, sampledfrom the observed trajectories.

Furthermore we are investigating methods of visual odome-try for estimating the trajectories of humans walking withinbuildings, or more general, in situations where no GPS lo-calization is possible. We are confident that this odometrydata together with partial GPS localization will suffice tointegrate an accurate map of the disaster area, includingroutes leading through buildings and debris.

Finally, it would be interesting to compare the system withconventional methods that are used in emergency responsenowadays. This could be achieved by comparing the ef-ficiency of two groups of rescue teams exploring buildingswithin an unknown area, whereas one group is coordinatedby conventional radio communication and the other groupby our system via wearable devices.

6. REFERENCES[1] Resq freiburg 2004 source code. Available on:

http://gkiweb.informatik.uni-freiburg.de/

~rescue/sim04/source/resq.tgz. release September,2004.

[2] Satellitenpositionierungsdienst der deutschenlandesvermessung sapos. Available on:http://www.sapos.de/.

[3] The ieee standard 802.15.1 : Wireless personal areanetwork standard based on the bluetooth v1.1foundation specifications, 2002.

[4] O. Amft, M. Lauffer, S. Ossevoort, F. Macaluso,P. Lukowicz, and G. Troster. Design of the QBICwearable computing platform. In 15th InternationalConference on Application-Specific Systems,Architectures and Processors (ASAP ’04), Galveston,Texas, September 2004.

[5] R. Bruentrup, S. Edelkamp, S. Jabbar, and B. Scholz.Incremental map generation with gps traces. InInternational IEEE Conference on IntelligentTransportation Systems (ITSC), Vienna, Austria,2005.

[6] M. Hazas, J. Scott, and J. Krumm. Location-awarecomputing comes of age. IEEE Computer,37(2):95–97, February 2004.

[7] J. H. Holland. Adaption in Natural and ArtificialSystems. University of Michigan Press, 1975.

[8] H. Kitano, S. Tadokoro, I. Noda, H. Matsubara,T. Takahashi, A. Shinjou, and S. Shimada. RoboCup

Rescue: Search and rescue in large-scale disasters as adomain for autonomous agents research. In IEEEConf. on Man, Systems, and Cybernetics(SMC-99),1999.

[9] A. Kleiner, M. Brenner, T. Braeuer, C. Dornhege,M. Goebelbecker, M. Luber, J. Prediger, J. Stueckler,and B. Nebel. Successful search and rescue insimulated disaster areas. In In Proc. of theInternational RoboCup Symposium ’05, 2005.

[10] H. W. Kuhn. The hungarian method for theassignment problem. Naval Research LogisticsQuaterly, 2:83–97, 1955.

[11] Q. Ladetto, B. Merminod, P. Terrirt, and Y. Schutz.On foot navigation: When gps alone is not enough.Journal of Navigation, 53(02):279–285, Mai 2000.

[12] L. Liao, D. Fox, and H. A. Kautz. Learning andinferring transportation routines. In AAAI, pages348–353, 2004.

[13] U. Mohring, S. Gimpel, A. Neudeck, W. Scheibner,and D. Zschenderlein. Conductive, sensorial andluminiscent features in textile structures. In H. Kenn,U. Glotzbach, O. Herzog (eds.) : The Smart GloveWorkshop, TZI Report, 2005.

[14] T. Nicolai, T. Sindt, H. Kenn, and H. Witt. Casestudy of wearable computing for aircraft maintenance.In Otthein Herzog, Michael Lawo, Paul Lukowicz andJulian Randall (eds.), 2nd International Forum onApplied Wearable Computing (IFAWC), pages97–110,. VDE Verlag, March 2005.

[15] I. Nourbakhsh, K. Sycara, M. Koes, M. Yong,M. Lewis, and S. Burion. Human-robot teaming forsearch and rescue. IEEE Pervasive Computing: Mobileand Ubiquitous Systems, pages 72–78, January 2005.

[16] C. Perkins. Ip mobility support for ipv4. RFC, August2002.

[17] N. Schurr, J. Marecki, P. Scerri, J. P. Lewi, andM. Tambe. The defacto system: Coordinatinghuman-agent teams for the future of disaster response.Programming Multiagent Systems, 2005.

[18] J. P. Snyder. Map Projections - A Working Manual.U.S. Geological Survey Professional Paper 1395.United States Government Printing Office,Washington, D.C., 1987.

[19] T. Starner. The challenges of wearable computing:Part 1. IEEE Micro, 21(4):44–52, 2001.

[20] T. Starner. The challenges of wearable computing:Part 2. IEEE Micro, 21(4):54–67, 2001.

[21] TopoGrafix. Gpx - the gps exchange format. Availableon: http://www.topografix.com/gpx.asp. releaseAugust, 9th 2004.

[22] H. Witt, T. Nicolai, and H. Kenn. Designing awearable user interface for hands-free interaction indmaintenance applications. In PerCom 2006 - FourthAnnual IEEE International Conference on PervasiveComputer and Communication, 2006.

123

APPENDIXA. COMMUNICATION PROTOCOL<xsd:complexType name="RescueWaypoint">

<xsd:annotation><xsd:documentation>

This type describes an extension of GPX 1.1 waypoints.

Waypoints within the disaster area can be augmented

with additional information, such as observations of fires,

blockades and victims.

</xsd:documentation></xsd:annotation>

<xsd:sequence>

<xsd:element name="Agent"

type="RescueAgent_t" minOccurs="0" maxOccurs="1" />

<xsd:element name="Fire"

type="RescueFire_t" minOccurs="0" maxOccurs="unbounded" />

<xsd:element name="Blockade"

type="RescueBlockade_t" minOccurs="0" maxOccurs="unbounded" />

<xsd:element name="VictimSoundEvidence"

type="RescueVictimSoundEvidence_t" minOccurs="0" maxOccurs="unbounded" />

<xsd:element name="Victim"

type="RescueVictim_t" minOccurs="0" maxOccurs="unbounded" />

<xsd:element name="Exploration"

type="RescueExploration_t" minOccurs="0" maxOccurs="1" />

</xsd:sequence>

</xsd:complexType>

<xsd:complexType name="RescueVictim_t">

<xsd:annotation><xsd:documentation>

This type describes information on a victim

relatively to the waypoint.

</xsd:documentation></xsd:annotation>

<xsd:sequence>

<xsd:element name="VictimDescription"

type="xsd:string" "minOccurs="0" maxOccurs="1"/>

<xsd:element name="VictimSurvivalTime"

type="xsd:integer" "minOccurs="0" maxOccurs="1"/>

<xsd:element name="VictimRescueTime"

type="xsd:integer" "minOccurs="0" maxOccurs="1"/>

<xsd:element name="VictimProximity"

type="Meters_t" minOccurs="0" maxOccurs="1"/>

<xsd:element name="VictimBearing"

type="Degree_t" minOccurs="0" maxOccurs="1"/>

<xsd:element name="VictimDepth"

type="Meters_t" minOccurs="0" maxOccurs="1"/>

</xsd:sequence>

</xsd:complexType>

<xsd:complexType name="RescueFire_t">

<xsd:annotation><xsd:documentation>

This type describes the observation of fire

relatively to the waypoint.

</xsd:documentation></xsd:annotation>

<xsd:sequence>

<xsd:element name="FireDescription"

type="xsd:string" "minOccurs="0" maxOccurs="1"/>

<xsd:element name="FireProximity"

type="Meters_t" minOccurs="0" maxOccurs="1"/>

<xsd:element name="FireBearing"

type="Degree_t" minOccurs="0" maxOccurs="1"/>

</xsd:sequence>

</xsd:complexType>

<xsd:complexType name="RescueBlockage_t">

<xsd:annotation><xsd:documentation>

This type describes detected road blockages

relatively to the waypoint.

</xsd:documentation></xsd:annotation>

<xsd:sequence>

<xsd:element name="BlockageDescription"

type="xsd:string" "minOccurs="0" maxOccurs="1"/>

<xsd:element name="BlockageProximity"

type="Meters_t" minOccurs="0" maxOccurs="1"/>

<xsd:element name="BlockageBearing"

type="Degree_t" minOccurs="0" maxOccurs="1"/>

</xsd:sequence>

</xsd:complexType>

<xsd:complexType name="RescueVictimSoundEvidence_t">

<xsd:annotation><xsd:documentation>

This type describes evidence on hearing a victim

relatively to the waypoint.

</xsd:documentation></xsd:annotation>

<xsd:sequence>

<xsd:element name="VictimEvidenceRadius"

type="Meters_t" minOccurs="1" maxOccurs="1"/>

</xsd:sequence>

</xsd:complexType>

<xsd:complexType name="RescueExploration_t">

<xsd:annotation><xsd:documentation>

This type describes the area that has been exploration

around the waypoint.

</xsd:documentation></xsd:annotation>

<xsd:sequence>

<xsd:element name="ExploredRadius"

type="Meters_t" minOccurs="1" maxOccurs="1"/>

</xsd:sequence>

</xsd:complexType>

<xsd:complexType name="RescueAgent_t">

<xsd:annotation><xsd:documentation>

This type describes the observant agent.

</xsd:documentation></xsd:annotation>

<xsd:sequence>

<xsd:element name="AgentName"

type="xsd:string" "minOccurs="0" maxOccurs="1"/>

<xsd:element name="AgentTeam"

type="xsd:string" minOccurs="0" maxOccurs="1"/>

</xsd:sequence>

</xsd:complexType>

<xsd:simpleType name="Meters_t">

<xsd:annotation><xsd:documentation>

This type contains a distance value measured in meters.

</xsd:documentation></xsd:annotation>

<xsd:restriction base="xsd:integer"/>

</xsd:simpleType>

<xsd:simpleType name="Degree_t">

<xsd:annotation><xsd:documentation>

This type contains a bearing value measured in degree.

</xsd:documentation></xsd:annotation>

<xsd:restriction base="xsd:integer"/>

</xsd:simpleType>

124

Multi-Agent Simulation of Disaster Response

Daniel Massaguer, Vidhya Balasubramanian, Sharad Mehrotra, and Nalini Venkatasubramanian

Donald Bren School of Information and Computer ScienceUniversity of California, Irvine

Irvine, CA 92697, USA{dmassagu, vbalasub, sharad, nalini}@ics.uci.edu

ABSTRACTInformation Technology has the potential of improving thequality and the amount of information humans receive dur-ing emergency response. Testing this technology in realisticand flexible environments is a non-trivial task. DrillSimis an augmented reality simulation environment for testingIT solutions. It provides an environment where scientistsand developers can bring their IT solutions and test theireffectiveness on the context of disaster response. The ar-chitecture of DrillSim is based on a multi-agent simulation.The simulation of the disaster response activity is achievedby modeling each person involved as an agent. This finergranularity provides extensibility to the system since newscenarios can be defined by defining new agents. This paperpresents the architecture of DrillSim and explains in detailhow DrillSim deals with the edition and addition of agentroles.

Categories and Subject DescriptorsI.2.11 [Computing Methodologies]: Artificial Intelligence.Distributed Artificial Intelligence[Intelligent agents, Multi-agent systems]; H.1.2 [Information Systems]: Models andPrinciples. User/Machine Systems[Human information process-ing]; I.6.3 [Computing Methodologies]: Simulation andModeling Applications; I.6.4 [Computing Methodologies]:Simulation and Modeling. Model Validation and Analysis

General TermsDesign, Algorithms, Experimentation

KeywordsAgent-based simulation and modeling, applications of au-tonomous agents and multi-agent systems, artificial socialsystems

1. INTRODUCTION

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.AAMAS’06May 8–12 2006, Hakodate, Hokkaido, Japan.Copyright 2006 ACM 1-59593-303-4/06/0005 ...$5.00.

Efficacy of disaster response plays a key role in the con-sequences of a disaster. Responding in a timely and effec-tive manner can reduce deaths and injuries, contain or pre-vent secondary disasters, and reduce the resulting economiclosses and social disruption. Disaster response is basicallya human-centric operation where humans make decisions atvarious levels. Information technologies (IT) can help indisaster response since improving the information manage-ment during the disaster–collecting information, analyzingit, sharing it, and disseminating it to the right people at theright moment–will improve the response by helping humansmake more informed decisions.

While innovations in information technology are beingmade to enhance information management during a disas-ter [18], evaluating such research is not trivial since recreat-ing crisis scenarios is challenging. The two main approachesto recreate crisis scenarios are simulations [6, 9, 5, 8, 19] anddrills. Both approaches have their benefits and drawbacks,simulating a disaster entirely by software lacks realism; con-tinuously running drills is expensive.

In DrillSim [4], we propose to take the best of both ap-proaches. We have instrumented part of our campus withsensing and communication capabilities such that we inputdata from a real ongoing drill into a multi-agent simula-tion, and vice versa. This way, a simulated disaster re-sponse activity gains realism since it occurs within a realspace with input involving real people, sensors, communica-tion infrastructure, and communication devices. Simultane-ously, the limited drills are augmented with virtual agents,sensors, communications, and hazards enhancing the scopeof the response activity being conducted. This, along withthe modularity of DrillSim, enables a framework where theimpact of IT solutions on disaster response activities can bestudied.

DrillSim is an augmented reality micro-simulation envi-ronment in which every agent simulates a real person tak-ing part in the activity. Every agent learns about its envi-ronment and interacts with other agents (real and virtual).Agents execute autonomously and make their own decisionsabout future actions. In an actual real-world exercise, thedecisions each person takes during a disaster response de-pend on a set of physical and cognitive factors as well asthe role that person is playing. The modeling of DrillSimagent behavior considers these factors. Creating a scenariois now based on binding a small set of roles and physicaland cognitive profiles to the large number of agents. Oneof the key advantages of a micro-simulation using a multi-agent system is the ability to bring new roles anytime and

125

study their interaction with the old ones or even create acomplete different scenario.

The rest of the paper is organized as follows. Section 2compares our approach with related work. Section 3 presentsthe DrillSim environment. Section 4 describes the DrillSimagent model and Section 5 elaborates on how agent rolescan be instantiated and edited. In Section 6 we illustratethe use of DrillSim through experiments conducted in thecontext of an evacuation simulation. The paper concludeswith future work in Section 7.

2. RELATED WORKThe need for multi-agent models for emergency response

that incorporate human behavioral aspects has been real-ized [16, 17]; an example of one such recent effort is atthe Digital City Project at Kyoto University [19]. Othermulti-agent simulators for disaster response include the ef-forts within Robocup-Rescue Simulation Project [8]. Ourwork is similar in spirit to those suggestions and enhancesthese initial efforts significantly. First, agent models withinour simulator capture the sensing and communication capa-bilities of individual agents at very fine granularity. Thesemodels allow us to systematically model the information re-ceived by individual agents over time. Dynamic changes toinformation available to an agent results in behavior changes(at the agent) represented in our system using stochasticmodels based on neural nets. Second, our system consistsalso of a pervasive space [7] that captures a drill of the activ-ity in the real space. This pervasive space consists of a vari-ety of sensing, communication, and display technologies andis used to conduct and monitor emergency drills within ourcampus. This way, our simulations can replicate real drillscaptured within the pervasive space. That allows compar-ing the behavior of simulated humans with the behavior ofreal humans for validating and calibrating the simulated hu-man models. Furthermore, the DrillSim augmented realityenvironment also allows integrating a simulation with an on-going drill, enhancing the simulation with realism and aug-menting the drill with simulated actors, hazards, resources,and so on.

3. DRILLSIMDrillSim [4] is a testbed for studying the impact of In-

formation Technology (IT) in disaster response. DrillSimprovides a simulation environment where IT metrics (e.g.,delay, call blocking probability) of a given IT solution aretranslated to disaster metrics (e.g., time to evacuate, casual-ties). This way, different IT solutions for disaster responsesuch as [11, 21, 20, 12] can be systematically and consis-tently tested.

DrillSim is a plug-and-play system. It enables scientistsand developers to (i) test the impact of one solution at a timeand (ii) reconfigure the set of technologies used and evaluatethe overall efficacy of integrated solutions. In addition, thesystem is designed to allow plug-and-play operation of dif-ferent external simulators such as network, hazard, or trafficsimulators. This way, available network, hazard, traffic sim-ulators, etc developed by domain experts can be exploited inDrillSim. Within DrillSim, new agent behavior models canalso be plugged in and the impact of IT on agent behaviorcan be observed.

The software architecture of DrillSim is shown in Figure 1.

The core of DrillSim is the simulation engine. It is the princi-pal component that drives the activity and it is composed ofa multi-agent disaster response simulation. It consists of thesimulated geographic space, the current evacuation scenario(i.e. where people are and what they are doing), and thecurrent crisis as it unfolds. The engine keeps a log of everyevent, information exchange, and decision. Agents also keepan individual log, which is consistent with the global log.

The simulation engine interacts with the data manage-ment module, the input interfaces and external modules,VR/AR module, and output interfaces and visualization.The data management module manages the data exchangebetween components. It is responsible for handling a highrate of queries and updates coming from the agents, andlogging the events of the activity. An important aspect ofthis module is the representation of the spatial and tem-poral data so as to adequately support functioning of thesimulated activity.

The inputs to the simulation engine can be divided intoconfiguration inputs and interaction with external modules.Configuration inputs create a scenario by initialize parame-ters regarding space, resources, crisis, infrastructure, agentslocation, agent profiles, and agent roles. External modulescan be plugged to DrillSim so that crisis, communications,traffic, etc can be simulated in external simulators devel-oped by domain experts. Mediators translate the interfacesamong external simulators and DrillSim.

The VR/AR module is responsible for the Virtual Real-ity/Augmented reality integration. The activity takes placein a physical space instrumented with visualization, sensing,and communication infrastructure. This pervasive spaceincludes multi-tile displays, video and audio sensors, peo-ple counters, built-in RFID technology, powerline, Ether-net, and wireless communications [7]. This provides aninfrastructure for capturing the physical space and activi-ties unfolding during a drill. This information is then inputinto the simulation engine, augmenting the simulation withreal people, space, and also sensing and communication in-frastructure. The real world is also augmented with the sim-ulated world. This is achieved by projecting the simulatedworld into visualization devices (e.g, a PDA) and allowingusers to interact with the simulated world.

Several visualization interfaces are supported: multi-tiledisplay, workstation, PDA, and EvacPack. The multi-tiledisplay and workstation interfaces allow a user to observethe activity and control it (e.g., configure and start a simu-lation). The PDA and EvacPack allow a user to take partof the activity. Location-aware information is sent to thePDA and its, simplified, interface allows its user to see thesimulated scenario as well as interact with it. Evacpack is aportable computer composed of a Windows box with wire-less internet connection, a pair of MicroOptical SV-6 VRglasses [2], a wearable keyboard, a wireless mouse, head-phones, a microphone, and a webcam. With more resourcesthan the PDA, Evacpack also gets a location-aware view ofthe simulated scenario and allows its user to interact withthe simulated agents. A part from the visualization, theengine also outputs disaster response metrics regarding theactivity.

4. DRILLSIM AGENT MODELEach agent simulates a person and agents are the main

drivers of the response activity. We illustrate the agent

126

Figure 1: DrillSim architecture.

model in DrillSim through an evacuation activity followinga crisis (e.g. fire). Each agent has a subjective view of theworld it lives in. This view depends on the agent’s sensingcharacteristics (e.g., how far it can see). Based on this ob-served world and the agent’s cognitive characteristics, theagent takes decisions (e.g., exit the floor), which results inthe agent attempting to execute certain actions (e.g., walk-ing towards the floor exit). Every agent in DrillSim has thefollowing attributes: state, observed world, role, profile, andsocial ties with other agents. These attributes dictate howan agent behaves and we describe each attribute in moredetail below.

Agent AttributesState. The state of an agent comprises its current loca-

tion, health, and devices it carries. The state is also formedby the information the agent knows about the world, the de-cisions it has taken, and the plans it has generated to realizethese decisions.

Observed world. An agent’s observed world is what theagent knows about the world it lives in. It is composed ofthe information it knows a priori (e.g., a map of the build-ing) and the information it gains during its life. An agentgains information about the world it lives in via the sen-sors it has access to (e.g., its own eyes, its cellphone). Anagent’s observed world is represented as a matrix Obs anda message queue. The matrix contains the representation ofthe geographic space and the localization of agents, obsta-cles, and hazards. Each cell in the matrix corresponds toan observation of a small region in the real world–that is,the real world is geographically divided in equal sized cells.Each cell contains a tuple of the form:

Obsi,j =< time, obstacle, occupied, hazards > (1)

where time corresponds to the time the observation wasmade, obstacle is a value between 0 and 1 and representsthe difficulty an agent faces in traversing a cell, occupiedcontains a list of agents occupying that cell, and hazardcontains a list of hazards present in that cell. Each agentupdates its matrix based on their perceptual characteristics(specified in the agent’s profile).

The message queue contains messages received from otheragents. It is a finite and cyclic buffer queue–agents, as hu-mans, can only remember a finite number of messages. Theamount of messages an agent remembers is also specified

in its profile. Each message m contains the time m.timeit has been received, its source m.source, its destinationm.destination, receiving device m.device, and message con-tents m.msg.

Role. An agent’s role dictates the decisions an agentmakes. For instance, a student at school, on hearing afire alarm, may decide to evacuate the building or to finishthe paper he/she is working on. On the other hand, a firefighter’s decisions might involve enter the building on fire tolook for people or report to the fire lieutenant. Therefore,the role an agent plays in the activity is a key element whenmodeling the agent’s behavior. In fact, modeling the activ-ity as a multi-agent simulation provides the flexibility to beable to change the scenario being simulated by changing theagent roles. Section 5 gives more details in role managementin DrillSim.

Profile. An agent’s profile includes the agent’s perceptualand mobility characteristics, initial health, role, the com-munication and sensing devices carried, and some cognitivecharacteristics. The agent’s perceptual characteristics alongwith the devices carried determine the information the agentcan sense and communicate. The mobility characteristicsinclude the speed of movement of the agent.

An agent’s information abstraction is also influenced byother cognitive factors. To accommodate this, an agent’sprofile includes how often an agent takes a decision and thedefinition of an activation function s. The activation func-tion expresses how the agent perceives information. Thesimplest function would be s(x) = x, where for any objec-tive input x (e.g., temperature, risk), the agent perceivesit objectively. Currently, we are using a sigmoid function, which is a common activation function used in artificialneural networks [15].

Social ties. Agents represent people and, as such, peopledevelop social ties with each other. Social network analy-sis focuses on the relationships among social entities, andon the patterns and implications of these relationships [24].Currently, we are modeling two basic social networks. In ourcase, the social entities are agents and the relations capturehow much an agent trusts another agent or how much anagent would wait for another agent when evacuating.

The more an agent trusts another agent, the more the re-liability associated with the information received from thatagent. Agents also associate different degrees of trust to dif-ferent devices. To represent this social network, each agenthas a vector Rel where Rela+d contains the relevance theagent gives to a message from agent a received via device d.

The other social network is regarding the fact that, as alsoobserved in several evacuations in our building, people tendto evacuate in groups. To represent this social network, eachagent has a vector MovingInf where MovingInf(a) repre-sents how much an agent’s decision to evacuate is influencedby another agent a that has decided to evacuate.

Agent behaviorAgent behavior in DrillSim is motivated by well-studied

models of information processing in humans [22, 23]. Thesemodels are formed by four entities: Data, Information, Knowl-edge, and Wisdom. A data element is the codification of anobservation. An information element is an aggregation ofdata elements, and it conveys a single and meaningful mes-sage. A knowledge element is the union of pieces of infor-mation accumulated over a large period of time. A wisdomelement is new knowledge created by a person after hav-

127

Figure 2: Basic entity model of information process-ing.

Figure 3: DrillSim Agent Behavior process.

ing gained sufficient knowledge. There exists a function forabstracting wisdom from knowledge, knowledge from infor-mation, and information from data (fw, fI , fd). There alsoexists a function (fp) that codes observations into data (Fig-ure 2). The goal of each function is to gain a clearer percep-tion of a situation by improving the orderliness (i.e., loweringthe entropy); which enables further systematic analysis.

Agent behavior in our model is illustrated in Figure 3.We model agent behavior as a discrete process where agentsalternate between sleep and awake states. Agents wake upand take some action every t time units. For this purpose, anagent acquires awareness of the world around it (i.e. eventcoding), transforms the acquired data into information, andmakes decisions based on this information. Then, based onthe decisions, it (re)generates a set of action plans. Theseplans dictate the actions the agent attempts before going tosleep again. For example, hearing a fire alarm results in thedecision of exiting a floor, which results in a navigation planto attempt to go from the current location to an exit locationon the floor, which results in the agent trying to walk onestep following the navigation plan. Note that the time t isvariable and depends on each agent’s profile. Furthermore,when an agent wakes up, it may bypass some of the stepsfrom Figure 3. An agent acquires awareness every nc timeunits, transforms data into information every nd time units,makes decisions and plans every nI time units, and executesactions every na time units. The relationship between thesevariables is: t ≤ na ≤ nc ≤ nd ≤ nI (e.g., nI = nd = 2nc =4na = 4t). This bypassing allows us a finer description ofpersonalities and makes the system more scalable since theremight be thousands of agents in one simulation.

5. AGENT ROLE EDITINGIn DrillSim every agent simulates a real person. A sce-

nario is recreated by binding agents to a small set of prede-fined roles, instantiating each agent’s profile, and instanti-ating social networks among agents. For example, an evac-uation of an office building is recreated by creating as manyagents as people would work in the building and then bind-ing most agents to the evacuee role and the rest to other rolessuch as floor warden (person in charge of evacuating onefloor). Also, a profile distribution is instantiated for everyrole (e.g., every evacuee’s age is set), and the underlying so-cial networks present in an office building are instantiated.Factors such as the randomness involved in decision-making,

Figure 4: GUI for editing agent role.

the different initial situation of every agent, and the underly-ing social networks guarantee a certain degree of variabilityon the agents behavior.

DrillSim incorporates a few predefined roles. However,this is not sufficient to cope with all the scenarios that can beplayed out. Evacuating an office involves employees and firstresponders; evacuating a school involves students, teachers,and principals; and responding to a radioactive spill involvesa hazardous material (hazmat) team. To support this kindof extensibility in DrillSim, a user should be able to definenew roles as needed.

Figure 4 depicts the process of managing roles. A usercan edit existing roles or create a new role based on an-other role or from scratch. This process is done before run-ning the actual simulation and the new roles are stored ina role repository. When creating the initial scenario of asimulation, the user specifies how many agents are needed,the roles of each agent, and the social networks among theagents. For specifying a role, the user needs to specify in-formation regarding profile, information variables, decisionmaking, planning, and social networks.

Profile. For every role, the user indicates a profile distrib-ution associated with it. The profile specifies several factorsthat influence what people sense, what decisions they take,and how they perform actions. Some of these factors includetheir visual and hearing range, their personality (e.g. risktakers), their health, and their speed of walking. In addition,attributes of other agents such as health, age, and sex mayinfluence’s a person’s behavior. Defining the profile meansproviding a mean and a variance for each of the profile’s pa-rameters. The current prototype implementation supports asubset of these factors, i.e. visual acuity, personality, speedof walking. This subset will be revised and enhanced as andwhen they become relevant to the agent’s behavior.

Information variables. As depicted in Figure 3, theworld observed by an agent is abstracted into informationvariables, which are the input to the decision making mod-ule. Not all agents take the same decisions. For example, anevacuee might not decide to put on a floor warden’s vest andsafety helmet. Adding new roles involves sometimes addingnew decisions. Some information important for this deci-sion might have not been relevant for other roles. Hence,sometimes, a user might need to specify new informationvariables. Namely, the user has to specify how this new in-formation variables are named and how they are abstractedfrom observed world and state (e.g., health). The user spec-ifies the name of the new variable and the code that, basedon the agents observed world and state, abstracts the newinformation variables.

Decision making. An agent’s decision making is mod-

128

eled as a recurrent artificial neural network [15]. Briefly, thecore of the neural net describes the importance of each inputto the decision-making module (i.e. information variables,decisions already taken) and results in the probability of tak-ing each decision. Another part of the neural net deals with,given this probability, randomly deciding whether the deci-sions are taken. Given the same input, agents with differentroles may take different decisions. For example, on hearinga fire alarm, a floor warden will decide to put his/her vestand helmet on, whereas an evacuee may decide to exit thebuilding. When defining a new role, a user has to set theweights of the decision-making neural network.

Plan generation. Once an agent has decided to take adecision, it computes a plan of actions to perform such de-cision. For instance, when an agent decides to put the floorwarden’s vest and helmet, it has to plan a set of steps fromits current location to the vest and helmet. For each pos-sible decision, the user has to specify the code that returnsthe corresponding plan of actions.

Social networks. Some of the decisions depend also onthe underlying social networks. For example, recall that theimportance an agent gives to a message also depends on themessage source. Social networks are instantiated a posteri-ori, when defining a scenario. However, certain informationvariables depend on the social networks as well. Therefore,when defining a new role, a user needs to specify the depen-dencies with social networks.

6. CASE STUDY: EVACUATION SIMULA-TION IN DRILLSIM

This section exemplifies one of the advantages of usingmulti-agent simulation–adding new roles on-demand. Inparticular, the response activity being simulated is the evac-uation of our building floor and the different roles are basedon the emergency response plan of the University of Cali-fornia, Irvine (UCI) [3]. Based on this response plan andon the maps of our building floor in UCI, we ran a series ofsimulations on a DrillSim prototype. The rest of this sectiondescribes the different roles defined on the UCI emergencyresponse plan, presents an implementation of DrillSim, anddiscusses some experiments and their results.

Note that the results here reported are only for illustrationpurposes. The validity of them depends on the validationof the agent behavior. Validating the current roles and cal-ibrating them is part of our ongoing work.

6.1 ImplementationAn initial DrillSim prototype has been implemented in

Java. The multi-agent platform chose is JADE [10]. JADEseamlessly integrates with Java and provides a framework foreasy development and deployment of multi-agent systems.The prototype provides a GUI for controlling the simulationthat allows real humans to observe and interact with thedrill simulation. Figure 5 shows a snapshot of the GUI.In particular, it allows a user to start the simulation, pullthe fire alarm, input a hazard, get an arbitrary view of thesimulation in 2D or 3D, get a 3D view of what an agent isviewing, send messages to other agents, control an agent,and get statistics of the evacuation.

In addition to this GUI, this first prototype also includesan interface that allows creating and editing agents roles(Figure 6). In particular, it allows specifying the mean

Figure 5: Snapshot of the prototype.

Figure 6: GUI for editing agent role.

and variance for different profile parameters, the informa-tion variables along with the software modules to computethem, the weights of the decision-making neural net, thesocial network dependencies, and the software modules forplan generation. The roles are then stored in XML formatand loaded a posteriori from DrillSim.

6.2 Agent roles for evacuationThe experiments here realized with DrillSim are based on

evacuating a floor of a building in UCI. Roles are used torepresent both emergency personnel and average citizens ofthe public (visitors, employees). The emergency manage-ment plan for UCI defines the following three roles: zonecaptains, building coordinators, and floor wardens. Theseare regular UCI employees that take the roles of zone cap-tains, building coordinators, and floor wardens during theevent of an emergency.

In order to coordinate the evacuation of the campus or ashelter-in-place response, the campus has been divided into13 zones with one zone captain per zone. Zone captainswear a red vest and are responsible for a zone, requestingresources, and relaying information between Emergency Op-

129

Figure 7: Impact of the new role.

eration Center (EOC) and the zone personal. Zones are fur-ther subdivided in buildings. There is a building coordinatorfor each building. They wear yellow vests and are responsi-ble for evacuating their building, determining a head count,and reporting the building status. Floor Wardens (greenvest) are responsible for evacuating their floor, and they as-sist building coordinators as needed. This way, a floor war-den’s decisions involve wearing his/her green vest and safetyhelmet, going into rooms to ask people to evacuate, callingto report people that are not evacuating, doing nothing, andexiting the floor.

The other relevant role is the evacuees. Evacuees repre-sent average citizens of the public (visitors, employees) andthey are supposed to exit the building and gather at the as-sembly areas. However, even though people rarely panic indisaster situations, they do not always comply with warn-ings [1] and they do not always exit the building.

6.3 Experiment resultsWith the current prototype, we realized several experi-

ments to illustrate the addition of new roles–a key advan-tage of an agent-based simulation. These experiments alsoillustrate the capability of DrillSim as a testbed for emer-gency response. The results are depicted in Figure 7 andsummarized as follows.

In these experiments two roles have been considered: evac-uees and floor warden. We started our experiments withonly one role: an evacuee. Twenty-eight evacuee agentswere positioned in the floor and the fire alarm was triggered.All agents heard the fire alarm; but not all agents immedi-ately decided to evacuate. In fact, the evacuation progressedslowly and some agents never evacuated. The weights of thedecision-making neural net were set such that the presenceof a hazard (e.g., fire), hearing the fire alarm, and beingtold by other agents to evacuate were the most importantinformation variables that drive an agent to decide to exita floor. The computation of these variables is based on theagent’s observed world and on more subjective factors suchas the memory an agent has and the reliability it associatesto other agents and the fire alarm. The former was fixed forall agents. The latter was based on a social network ran-domly initialized. Planning involved computing steps fromthe agent current location to an exit. This was achieved byusing an algorithm based on A* [13, 14].

On the rest of the experiments, a new role was added: thefloor warden. In these experiments, we initialized the scene

with also twenty-eight agents. This time, one, three, or fiveof the agents were assigned a floor warden role. Figure 7shows the impact of the new role. Having one floor wardenimproves the evacuation, even though when the floor wardenleaves the floor there are still 3 evacuees that have not evacu-ated the floor. Using three floor wardens improves the evac-uation further. However, five floor wardens do not do betterthan three. For the new role, the weights of the decision-making neural net were set such that the same factors thatwere relevant for deciding to exit a floor are this time impor-tant for deciding to evacuate a floor instead. When an agentdecides to evacuate a floor, the first thing he/she does is goto his/her office to pick up the floor warden’s hat and vestand put them on. Afterwards it visits every room and asksagents to exit the building. Only when the floor warden hasvisited all rooms it decides to exit the floor. The reliabilitysocial network was similarly initialized as before. However,the importance that a floor warden would give to the firealarm and the importance evacuees give to the floor wardenwas high.

7. CONCLUSIONS AND FUTURE WORKProviding the right testbed for testing IT solutions in the

context of disaster response is crucial for improving our re-sponse to disasters such as the World Trade Center terroristattack. Traditionally, IT researchers would test their solu-tions in IT-oriented testbeds (e.g., a network simulator) thatwould evaluate their approaches based on IT metrics suchas delay, call blocking probability, packet lost probability,and quality of the service just to name a few. However,when testing an IT solution for improving the efficiency ofdisaster response, we need a testbed that translates theseIT metrics to disaster metrics such as evacuation time andcasualties. DrillSim is such a testbed that allows pluggingan IT solution to the simulation framework and obtainingdisaster metrics.

This paper presented DrillSim, focusing on the Multi-agent simulator component that simulates a disaster responseactivity. One of the key features of such a multi-agent basedsimulation where agents simulate humans is that it allowsthe editing of existing roles and the addition of new roles on-demand. This enhances DrillSim and makes it an extensibleframework where new scenarios can be created and executedon the fly. The methodology implemented in DrillSim formanaging agent roles was also described and demonstratedwith a series of experiments in the context of an evacuation.

Future workA very important point for every simulator is to be able

to discern to which extent it models reality. In our futurework, we plan to calibrate and validate our agent behaviormodels. We have instrumented part of our campus withvisualization, sensing, and communication infrastructure sothat an activity can be captured. With this infrastructure,we can contrast a simulation of an activity in the multi-agent simulator with a real drill of such an activity. Thisway, we can calibrate our agent behavior model and validateit. Moreover, this infrastructure also allows us to merge thereal drill with the simulation, achieving a very flexible andpowerful testbed.

Our objective in DrillSim is to be able to simulate campus-wide disaster response activities involving a large amount ofagents. With this goal in mind, scalability becomes anotherissue that needs to be tackled. A multi-agent simulation

130

provides us a natural way of distributing the computation,since agents can be seen as autonomous computation unitsfor partitioning computation. However, the high rate of dataqueries and updates that need to be resolved in real-timeposes still a challenge. Even worse, data cannot be sta-tically partitioned based on location; in activities such asevacuation, agents would initially be distributed across sev-eral areas but as we start the simulation, most agents wouldbe moving towards the same areas, overcrowding those areas(i.e. overloading those servers).

Apart from calibrating agent behavior and making it scal-able, some of the other issues that will be tackled to movefrom the current prototype to the next stable DrillSim ver-sion will also involve the interaction between real people andagents and generation of a role repository.

8. ACKNOWLEDGMENTSWe would like to thank the rest of the DrillSim team for

their dedication to the DrillSim project. This research hasbeen supported by the National Science Foundation underaward numbers 0331707 and 0331690.

9. REFERENCES[1] TriNet Studies & Planning Activities in Real-Time

Erthquake Early Warning. Task 2 - Lessons andGuidance from the Literature on Warning Responseand Warning Systems.

[2] MicroOptical-SV-6 PC Viewer specification.http://www.microopticalcorp.com/DOCS/SV-3-6.pdf,2003.

[3] Emergency Management Plan For the University ofCalifornia, Irvine.http://www.ehs.uci.edu/em/UCIEmergencyManagementPlan-rev5.htm, Jan2004.

[4] Drillsim: Multi-agent simulator for crisis response.http://www.ics.uci.edu/ projects/drillsim/, 2005.

[5] EGRESS. http://www.aeat-safety-and-risk.com/html/egress.html,2005.

[6] Myriad. http://www.crowddynamics.com, 2005.

[7] Responsphere. http://www.responsphere.org, 2005.

[8] Robocup-Rescue Simulation Project.http://www.rescuesystem.org/robocuprescue/, 2005.

[9] Simulex: Simulation of Occupant Evacuation.http://www.iesve.com, 2005.

[10] F. Bellifemine, A. Poggi, G. Rimassa, and P. Turci.”an object oriented framework to realize agentsystems”. In WOA 2000, May ”2000”.

[11] M. Deshpande. Rapid Information Dissemination.http://www.ics.uci.edu/ mayur/rapid.html, Aug 2005.

[12] A. Ghigi. Customized Dissemination in the Context ofEmergencies. Master’s thesis, Universita de Bologna,2005.

[13] P. E. Hart, N. J. Nilsson, and B. Raphael. A formalbasis for the heuristic determination of minimum costpaths. IEEE transactionss on Systems Science andCybernetics, pages 100–107, 1968.

[14] P. E. Hart, N. J. Nilsson, and B. Raphael. Correctionto ”a formal basis for the heuristic determination ofminimum cost paths”, 1972.

[15] S. Haykin. ”Neural Networks - A ComprehensiveFoundation”. Prentice Hall, 1999.

[16] S. Jain and C. R. McLean. An Integrating Frameworkfor Modeling and Simulation of Emergency Response.Simulation Journal: Transactions of the Society forModeling and Simulation International, 2003.

[17] S. Jain and C. R. McLean. An Architecture forModeling and Simulation of Emergency Response.Proceedings of the 2004 IIE Conference, 2004.

[18] S. Mehrotra, C. Butts, D. Kalashnikov,N. Venkatasubramanian, R. Rao, G. Chockalingam,R. Eguchi, B. Adams, and C. Huyck. Project rescue:Challenges in responding to the unexpected. SPIEJournal of Electronic Imaging, Displays, and MedicalImaging, (5304):179–192, 2004.

[19] Y. Murakami, K. Minami, T. Kawasoe, and T. Ishida.Multi-Agent Simulation for Crisis Management. KMN,2002.

[20] N. Schurr, J. Marecki, M. Tambe, P. Scerri,N. Kasinadhuni, and J. Lewis. The future of disasterresponse: Humans working with multiagent teamsusing defacto. In AAAI Spring Symposium on AITechnologies for Homeland Security, 2005.

[21] M. Tambe, E. Bowring, H. Jung, G. Kaminka,R. Maheswaran, J. Marecki, P. Modi, R. Nair,S.Okamoto, J. Pearce, P. Paruchuri, D. Pynadath,P. Scerri, N. Schurr, and P. Varakantham. Conflicts inteamwork: Hybrids to the rescue (keynotepresentation). In AAMAS’05, 2005.

[22] L. Thow-Yick. The basic entity model: a fundamentaltheoretical model of information and informationprocessing. Information Processing and Management,30(5):647–661, 1994.

[23] L. Thow-Yick. The basic entity model: a theoreticalmodel of information processing, decision making andinformation systems. Information Processing andManagement, 32(4):477–487, 1996.

[24] S. Wasserman and K. Faust. Social Network Analysis:Methods and applications. Cambridge UniversityPress, 1994.

131

Social Network Visualization as a Contact Tracing Tool

Magnus Boman�

The Swedish Institute ofComputer Science (SICS) and

Dept of Computer andSystems Sciences, Royal

Institute of TechnologyKista, Sweden

mab@sics.se

Asim GhaffarDept of Computer and

Systems SciencesThe Royal Institute of

TechnologyKista, Sweden

asim.ghaffar@gmail.com

Fredrik LiljerosDept of Sociology

Stockholm UniversityStockholm, Sweden

liljeros@sociology.su.se

ABSTRACT�������� ��� ������ ��� �� ������ � ��� ������������ ��� ����� ����� ���� ����� �� � ���� ��� ����� ��� �� ������� ������ ����� ��� �������������� ��������� �� ��� ��� ���� ��� �� ������ ������ ��� ��! �� �������� ��� �� �� ������� � � �� ����������� � � ���� ������ ���������� ����������� ���� �"#� ��� ��!�� �������� � $� ����� ����� � �� �������� ����� �� ������ �������� ��������� �� ����� ������ � %����� ����� � � �� ����� �� ��� � � �� � ������!� ������ ��� ��������� !�� � �� ��� ���� � ��� �� �����

Categories and Subject Descriptors$�&�' (�������� ���� ������)* ��������� �� +��������,����� �������� �� ���� � - .�" (�������� ���

����������)* /��� �� +����� ������� ,������� ��������� �� ��

General Terms+������

Keywords0����������� ���� ��� ��!� ������ ��������� +1�2� ����� ���������

1. INTRODUCTION3���������� �� ������� �� ��� ��������� � ����

������ ��� �� ��� �� � �������� ������� �� �� ���� ����� ���� 0���������� � � � ���� ������ ������ ��� ���������� �� ���������� �������� ����� �������� ������

�%�� ����� �� �� ������ �� ��� ����!���� 4���� ���0������� +������ ���40+�� �� ���������

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.AAMAS’06 May 8–12 2006, Hakodate, Hokkaido, Japan.Copyright 2006 ACM 1-59593-303-4/06/0005 ...55.00.

������ 6�� ��� ������� �� ��������� ������ � ��������� � ��� �+2�� �� �� ��������� ������ ����� �� ��������� 7*7 ��� ��� ���� ���� ��������� �� ���������� ����� ����� ������ ��� ������� �������� �� ���� ������40+� +2� ������� �� �������� � ���� ���� ��� �����8 ���� (9)�� �� � ��������� ����� � ��� ��:������������� ��� ������ ��� �� ������ � ��� ������������ ��� ����� ����� ���� ����� �� � ���� ��� ����� �;�� ������ ��! �������� ����� ������������ ����� �� +2� ������� ��� ����������� ��������� � ���� ��� ����������� ������� �� ����� �8�������� �������� �� ����������� ��� ���� ��� ������ ����� �<��� �=���� � ���� �� ��� ������� �� �� ������ ��� ��� �� ���������� �� �������� ��� ������ ��� ��! � � �������� ��� �� �� ������� � � �� ����������� � ������� ��� �� ���� ��� ������ ����� �� ������ ��

��� ����������� � ��� ������� ��� � �� ����� ��� ��� ����� � � ���������� �� :� �������� �� ��� ������ � ��� �� � �� ������ � �� ��� ��������� ���� ����� ����� ��� ������ ���������� ����������� ���� �"#���� ��! �� �������� � $� ����� ����� � �� �������� ������ �� ������ �������� ��������� �� ����� ������ �%�� ����� � �� ��������� ������ � �� �!� �� � ��� ������� �� ��� � � �� � ������!� ������ ��� ���������� !�� � �� ��� ���� � ��� �� ����� 6�� �� �� �� �������� � ��� ��� ���� ���� ��� ���������� ��� �������� ��������� ����� �������������� � ��� ������������ ����� �+1�2�� ������������ ����� � ������� ����� �������� ����� �� ��� !��� $� �� �� ������� ������������ �� ��� !�� �� ������� �� � !�� �� ��� �� � �� ��������� � ����� ����� �� �� ����� � ������� �� � � � ������ �� � ������ !�� �� ��� �� �� ��������� �+1�2 � ����� �� �� ����� ������ �� � ���� �� �� ���������� ��������� �� �� ��������� ��������� �������� ���������� � ������� �������� ��������� �� %�� � ��� � ��� ���� � ��� ���� �8��� ����� �� ��������� ������������ ��������� ��� ����� %���� �� � � �������� ��� ��� ����������� � ����� �� ������ ��� ���� ��+1�2� 6�� �� �� �������� ��������� �� �� ������ ����� � �� �� ������� �� �������� ������� ����� �� ���� ���� ������� �� ���� ��� �� � � � ������� %�� � �� �������� �� ������������ ���� �������� ���� ��� ������ ��� ��������� ��� ���!�� ��� ����� � �� ��� ��� ��!�$� ��� ����� � ��� ���� �� ��� >� � ��� ���� ��� � ��� �� � ����� ��<����� ����������� �� ������ ��� �����

132

���������� �� ��� �� ���� ��� ��� �� ���� �$� ��������� � ��� � ��� ��� �� ��� � �� ����

����� ��� ����� ������ �� �������� ����� � � ���� ������� ���������� �� ��� ��������� �� �� ������ ������ ��������� � ��� � ��� �� ���������� �� ��� �� ��� � ��������� %�� ��� ������ ����������� ���� ��� ��������� ��� ��� ������ �� �� ���� ��� �� ��������� ������� �� �� ���� ������� �� ��� ���� ���� �� ��������� ��� %�� ���� �� ������� ����:����� ������ ��� ��� ��������� �� ������� �� ������ �� ��� �� ��������� ��� ��� ������� ��� �8� � �������� ��� �� ��� ���������� ����� ������ � ���� ��?����� ������� ����������� �� ��>������� �� � ����� �����!�� ���� ������� � � �� ����������� � ��!�� ��� ���+1�2 �� ��� ��� ����!���� ������0������� ���� ����� ��������� � ��� � �?��

� ��* ��� ���� ��� �� ���� ��� ����� � �� ��� ��������� � ���- �� � � ����� �� ��� �<��� ��� ������ ��� ��� ���� ��� � ������ ���� ���� ���� (@)�� %���� �� �� ���� ��� ���������� ���� ���� ("� 7)�� ��� ����������� ������� ��� �� ������������� ��������� � ��� ���� ���� ����� ��� ���� �� ���� ����������� � %�� � ���� ����� ���� � � ��������� �� ������ �� �� �������� ���������� �� ����� $� �������� ��� � ����� ���� ������� ������� ��� ������ ��� �� � �� ���� �� �� ����������� � ����� ������ ;�� ������� � � �� � ���� ��������� ������ �������>� A� �������� ������� ��� A%B (7C) ��� ���� � �� �� ������ ���������� �� ��������� ������� � �� ��� ������ ��������� �� ������������ �������� ������� ��� $���A� (D) ��� ��� �� ��������� ��� ���� � � �� :�8�������� =� ����� ��� ������������ ��� ������� ������������ �� ���� ������ ���� ��! �� �������� � 6�� �8����� $���A� � ���� E#� F��� ��������� >��� (')� ��� ��! �� �������� ��� ��?��������� �� ���� ��� �� ��!� � ��� G?�! (E)� ��� ��� ����� G?�! � ����������� ��� ��� ��� ��! ���� � � ��� ��������� ��� �������� ���� ������ � G?�!� � � ��������� �� ������ ������ 1����� ��!� � ��� ����� � (H)� �� ���� �� �� ����� ��� ��! �%�� �� � ���� ��� ��� ��� ��� ����� �� ������� �������� ���� ��! �� � ����� ������ �������� (&)��� ���� ���� ����� ����� ������� �� ��� �� ���������������� �

2. DATAG����� �� �� � ������ ��� ��� ��������� � ��� ��

����!���� ������� %�� ��������� � ��� ��� � � �� ������ �� ���� �� ��������� ������ � 0�� �� ���� ��� � � �� ��� ������� ��������� � �� ��� ������� ����������� ��� ����� �� � %�� �� � �������� � ��� �������� 2 �� �� ����� � ��� �� ������ ���� ���� ��� �� �� ������� ����� ��� �� ������ �� ����� � �� %���� � �������� �� ��� �� � ��� ����� ��� � ��� ������� � � � ��� ��� ������ ���� ��� ��� �� �=� ����� �� �� � �� ��������� ��� ������ �� ����������� ������ ��� ��� ��� ��������� � ����;�� � �� ������� ��� ���� �� ��� ��� #�� �� ��� ���

������� �� ��� ��� ��� �� ����� ��� �� � �� ��� �� ����� ��������� ������� %��������� ����� �!� �� �� ��� ��� ������� � �� ��� ��� ���� ������� ������������ ��� ��� ��� �� ����! ��� ��� � � �� ��� �������������� �� ������ � ������ �� �����>���� �� ������ �� ��� ��� ��� ���� �� ���� �� ����>�� ���� ���

���� � �������� ��� ����� ��� �� �� � $� +1�2� ������ �� �� � ��� ����� ������� �� �� ����� ��� �� ��� ���� ��� ������ ������� �� ����$� ����� ��� � �� ����������� ����>����� �������� ���

����� �� ��� ����� �� �������

7� A� ����� ������ ��� ��� ��� ��� ��! �� ����������

E� #� �������� ���� �� ����� �� �������

"� #� �������� ������ �� �������

'� +!� ���� ��� ��� � ������� �� ���������

9� #� �������� ��� �� � � ����� �� � ���������

&� +� 8� ��������� �� ��� ��������� � �� �8�� ��� �� >��� �� ��� +1�2 �� �� �� ��� ��������� ��� �������� ������� >��� �

D� 4��� �������������� ��� � �� �� ��� ����� ���� ��+1�2 � �� �� ��� ����� �� �>������

H� G������ � � � � �� �� ������ !�� ��������� �� ������� ���� �� ��� �

@� %�� ���� ����� �� �8������� ��� �������� �� ��� ����� � �� ����� �G������ '� 97E+� 12+� ���� ��� �������� � ����

7C� %���� ����� �� �� ���� ����� �� �� ?��!� ��������� ��� ������� ���� �� �� �� ����� 9CC ��� �

3. DESIGN AND IMPLEMENTATION%����� ��� � �� ��� ����������� � �� �� ���� ���

�!�� �� ������� ���� ��� � � ���� � 4����� ������ � "#���� ��! �� �������� ������������ ����� �� ��� ����� � � ����� �� ��� +1�2� ����>� ������ ����� ��� ��� �� ��� ��� ����� $� ��� � � �� +1�2� ������ ��� � �� ����� �� �� �� ���� ����� ������ ��� �������� �� ��� ��������� %�� ��������� ��� ����� �� ����������� �� �� ���� �� ��� �� ���� � ���� �� ����� ������ ���� ��� ���� ��� ���� %�� ����� � ��� ���� ��� ��� ������ ���������� ������ ��� �� �� �������� ��� � ���I ����� ������� �� "#� �� ����� ������������ �� ��������� ��������� �� ���� �������� ���� �������� ������ ��� � �� � �� ���� ����������� �� � ��� ���������� %�� ���� ��� � � � ����� ��� * ����������� � ��� ��!��� �������� ����� �JA0� ��� +1�2� ����>� ��������� �+1�22���� �� ���������JA0 ��� #�����K�@� � �� ������ %�� ������� ������ ���������� ����� ����� � �� ��� �� � � ��� �� �<��� ��� �� ���� ���� �� ��� �� ��� ���� ������ �� � -������ ��� ��� � ������ �� �� 1������� � ��� �� ������ �������� ��� ��� �� JA0� %���� ��� ��� ����� ����������� ��� �������� �������� � ������ ����� �� ��������� ��� ECCC ����� ��� �L �� �� ���� ��� ���� ��� �� ����� 8� � ��� ��

8� �������� �������� ���� ��M��� ������������� 2������������� � �� �� ����� ��� ������� ����� ��� K� N�� O�8� ���� !����� ��� ���� >8��� ���>� � � �������� ���� � ������� ��� ����� ��� ���� ������������� �� �� ���� �� �� ���� %�� JA0 �� ������� �� !�� ���� ���� ��� ��� ��� �� � �� ����� ��� �� ������� �� ���� �� �� ����� 8� � �� �!� ��� ������ �� ��� �� ����

133

����� �� ������ ���� ��� ��� ������� ������

���� ���� !��� ��� "��"�� ����� ���� #� �����

�� ��� $��"�� ����� ���� %�� ����� �� � & ����

�$�'

��� ��� �� ������ �� ��� ������ 3������ ��� ��� ���� ��� ��� � �� �� �� �������� ��� ��� ��� ��� � ���� ��� �������� ��� �� � ��� ��������� ������ 0�� �� ��� ��<����� ���� �� ����� � %�� � � � ��������� �� �� � �� �� �� ����� ���������� ������ ������ ��� � ��� ������� ������� ���� %�� JA0 ��� ���� �� ����� �� ��� ��� ��� ������!� �������� � ���?��� �� "# ��� � ����� ���!�� �� � ��� �������������� �� ��� JA0� 2���� ����� � ���������� ��� JA0 ��� � ���������� ����>� ����������� �������� ���� �� ��� ��� ��� >�� ���� � �� ����� $� ��� JA0 ������� � ��� � ��� ���� ��� ��� �� � � ���� �� ����� ��� ������ ����� +1�22���6���� 7 �� ���� �� ���� ��� 8�8� � ����� ��

$������� �� ��8� � ����� �� ;���#����� %�� � � ����� �� � ���� �� ������ ����� ��� �� � �������� ���� �� �������� ��� �� ���� �������� 2� ��� ��� ��6���� 7� ����� �� ���� ����� ����� ��������� �� ����������� � �� � ��� ����� � �� ��� ��� ��������� ����� ������ �� ��� >��� ����� ��� �� ����� ������ ������������ � 6�� �8����� ��� ��� ��� ��� �8���� ;�������� � ���� ���� ��� ���! - ��� ��� ��� ��� ��������� ���� �� ���� �� �� �� �� ���� �� �������� �� � ���� �� ��� �������� ������ � ;�� ����� ����� ������ ����� ��� ��� � �������� ��� ��� ��������� ��������� �� ��� �� �� �� �������� ����� � %���� � � ���� ����� ��!* ��� ���� �� ��� �8����� ����� +������������� �� ����� ����� ���� � ����� ����� ��������� ����� �������� ����� ��� ��������� � ���� �� ���� ����������� ������ ;�� �� ��� � � ���� ������� �� ��� ���� ����� ��! ��� ��������� ���� � �� ���� �� ���� ����� �� ������ ����������

4. CONCLUSIONS�� ��� ��� ����� ������� ��� ��! �� �������� ����

��� �� ������ �������� ���� � 2� �� � � ����� ��� ����� ����������� � ��� ��! �� �������� ������ +�������� ��������� �� �� ����� �� ���� �� ���������������� � ��� ��������� � +1�2� ����>� ������������ ��

��������� �� ��� ��������� ��� ��������� � �� � ����������� ����������� ����������� �������� ��� � ������ �� ����� ����������� � � 2� �������� �� ������ ���� ����� ������ ��� ���� ������ �� ���! ����� ���� �%�� ������� ��� ���� �!�� �� ������ ��� ��� ��� ������� �!�� �� ���������� ����� %� � � �� ��� �� �������� ��� ������ ������ ��� �� =� ����� ������� �� � �� � �� �������� � ��������� $� � �������� ��� ������� ��� � ���� �� ���!�� ��� �������� ������������������* ��� � ! �� �������� ��� ������� �� ������� ��������� �

5. ACKNOWLEDGMENTS%�� ����� ���� ��!� �� ���! ��� � ��� � 0�������

+������ 2���� ��� ����� �������� ������� $� ��������� /��?��� �!�� ���� ������ ���� ��� 0������ ��� ���� J0�%MG2%=6$J#01 ���?��� #N�;J0% C7E@77�

6. ADDITIONAL AUTHORS2�������� ����� * +�!�� ������� �#�������� �� +���

��� 0���������� �� F�� ��� ��� � B����� ! $� ������������* ��������� ��������������

7. REFERENCES(7) 2� 2���� � 0� 3������ 0� +������ � 3� 6���� ��

;� +��� �� L � �� ��� ���� ����� ������� ���� ����� ������������� ���������� �� �� ��������������� � ���� ��P��� �P������� "D�&�*D&HQDD'� 7@@"�

(E) A� F���? �� 2� +���� G?�!,����� ��� ������ ��! ��� � � ������ � E7�E�*'DQ9D� 7@@H�

(") �� F���� 0� 4����� 3� F����F�� ��� �� 0� /����#���������� �� ���� �� ������� ���������� � ��� ��� ���������� �� ������� � ��� ���� � �� �� ������ ��������� � $� ���� ���� ����� �� &@&QDCC� 7@@@�

(') L� F���� � G� B��� � .� 1�� A� ���������� ��#� ����� 08�������� ���� ��� �� �������� �������� ��� ��! � ���������� ������ � 77*D9Q7C&� 7@@@�

(9) /� F��� �� � +������8, ���� ��� �� �������8������ ������ �������� ����� ��� �����8 �������$� ���� �� ��� ����� ������ ��������� �� H'QH@� .���� ECC9�

(&) .� +� 0� ���� �� 1� /� 28����� ��!�" ���#����������� $������ ������ %��� �� &��� '�� +$%G�� � 7@@&� F���!�� $� ���������

(D) .��#� 6�!���� %�� $���A� ����!��� $� ���� �((( ������������� �� ����)���� �� 7&DQ7D'� $000G�� � ECC'�

(H) .� =���� �� B� 3��� �� .� 2� /���� ����� �,����!�� ��� ���������� ���������� �� ��������� $����� ���� �� 'E7Q'"C� �$43=$� ECC9�

(@) 6� /��?��� � G� =����� �� .� 4�� ��!�� %�� ��������� ��! �� ������ �� ������ �������� � ���� $�!!!���*�����"+�� +,�����-�+././.0.� +� ECC9�

(7C) �� .� ���������� B� +� +����� �� �� 0� /���� ���%�� �� �� �� ������������� �� � ��?����������������!�� ��� "� ����� �� �� ��������� $� �����((( �� ����)���� �� @"Q7CC� $000 3�������������� G�� � 7@@&�

134

Section 4

Agent-Based Architectures and

Position Papers

135

Protocol Description and Platform in Massively MultiagentSimulation

Yuu NakajimaKyoto University, Department

of Social Informaticsnkjm@kuis.kyoto-u.ac.jp

Hironori ShiinaKyoto University, Department

of Social Informaticsshiina@kuis.kyoto-u.ac.jp

Shohei YamaneKyoto University, Department

of Social Informaticsyamane@kuis.kyoto-

u.ac.jp

Hirofumi YamakiKyoto University, Department

of Social Informaticsyamaki@i.kyoto-u.ac.jp

Toru IshidaKyoto University, Department

of Social Informaticsishida@i.kyoto-u.ac.jp

ABSTRACTThe spread of the ubiquitous computing environment en-ables the realization of social systems with large scale. Mul-tiagent simulations are applied to test such large systemsthat inevitably include humans as its parts. To develop suchsimulation systems for large-scale social systems, it is neces-sary for experts of application domains and of computationsystems to cooperate with each other. In such simulations,there are variety of situations that each agent faces. Also,the scalability is one of the primary requisites to reproducephenomena in a city where hundreds of thousands of peo-ple live. As a solution to these problems, we introduce anarchitecture for multiagent simulation platforms where theexecution of simulation scenario and the implementation ofagents are explicitly separated. This paper also gives theevaluation through the implementation.

1. INTRODUCTIONAlong with the spread and improvement of mobile phones,

environment for ubiquitous computing is becoming popular.As for the conventional information service, a user uses ser-vice with a terminal fixed in the room. However, in ubiqui-tous environment, each user has his/her own portable deviceand use services via network at various places such as out-of-doors. Because each person has his/her own device suchas a mobile phone, it is possible to show each user differentinformation. In addition, GPS (Global Positioning System)and RFID (Radio Frequency Identification) tags enable de-

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.ATDMMay 8 2006, Hakodate, Hokkaido, Japan.Copyright 2006 ACM 1-59593-303-4/06/0005 ...$5.00.

vices to get information of the location and the situationof a user. In such an environment, it is possible to provideservices based on properties, the purpose, the location andthe context of each user. Navigation service in public spaceis one of such services[4].

It is necessary to grasp the location and the situation ofa user for navigation. It is able to get situations of certaincrowd with surveillance cameras. However, places wheresurveillance cameras can be placed are limited, and camerasmay not get enough information. In cases such as city-scalerefuge navigation, it is necessary to grasp the situation of acrowd over a wide area. Mobile phones equipped with GPSand RFID tags are useful for this purpose. It is possible todecide how to navigate people by grasping the situation ofa crowd.

On the other hand, it is necessary to send personalizedinformation to each user. It is difficult to instruct accord-ing to individual situations by conventional methods suchas announcement for the whole with loudspeaker. Trans-mission to devices which each person has such as mobilephones realizes to send necessary information to each userindividually.

Thus, the environment to grasp individual situations andto send personalized information is becoming available.

To know the behavior of such social systems that in-clude humans, it is desirable to perform proving experi-ments. However, it is often hard to perform such experi-ments that include a large number of people in a large areawith the size of a city. Instead, it has been proposed toanalyze such systems by multiagent simulations where eachperson is modeled as an agent. Previous works include anapproach to design agent protocols to perform simulationswhere evacuation guidance has been used as a testing exam-ple[7]. There has been some research on large-scale multi-agent platforms [5], for example, MACE3J [1], MadKit [2]and Robocup Rescue [6].

We aim to realize a large-scale multiagent simulation en-vironment that is applied to the analysis of social systems

136

Figure 1: Both Protocol and Internal Model are Im-plemented in Each Agent

that support people in a city, with a number up to million.In this paper, we describe an architecture to control millionagents by describing protocols. We also give an implemen-tation for the architecture and evaluate it.

The following problems are addressed in this paper.

i. The separation of protocol design and agent im-plementationTo build a system to simulate large-scale social sys-tems, agent protocols that reflect human behaviorsshould be described by experts on human systems do-main such as traffic management or disaster preven-tion, and the implementation of agents should be doneby experts on information systems. Thus, it is desir-able for a development environment to separate the de-scription of protocols and the internal implementationof agents. In our architecture, the protocol processingsystems and the agent systems are independent fromeach other.

ii. Dynamic protocol switchingIn simulations of large-scale social systems, each agentfaces a variety of situations. A single protocol descrip-tion to deal with all such situations may become largeand complex. Instead, our architecture allows exper-imenters to dynamically switch protocol descriptionsgiven to agents corresponding to the changing situa-tions.

iii. ScalabilityMost of existing protocol processing systems and agentsystems are not designed with the management of alarge number of agents in mind. To perform simula-tions for large-scale social systems, simulation systemshave to control a large number of agents that modelhuman behaviors. We achieve the scalability by apply-ing large-scale agent server which is recently developedand works on event driven object models.

In below, we propose an architecture for large scale multi-agent simulation platform that copes with these three prob-lems. In Sections 3 and 4, we describe a platform that con-sists of scenario description language system Q and large-scale agent server Caribbean. In Sections 5 and 6, we de-scribe the evaluation of the platform and an application ex-ample of evacuation guide.

2. ARCHITECTURE

Figure 2: External Protocol Interpreter ControlsAgent System

Figure 3: Protocol Interpreter on Agent SystemControls Agent

There are two possible types for the mechanism to controlagents by giving designed protocols. One of them is the oneshown in Figure 1, where protocol description and agent in-ternal model are implemented together into an agent. Theother is shown in Figure 2, where an external protocol pro-cessing system controls agent internal agent internal model.

In the approach shown in Figure 1, the developer of agentsystem implements an agent by integrating protocol descrip-tion, which is given in an inexecutable language such asAgentUML[9], and agent internal model. In this methodwhere both protocol description and agent internal modelare implemented in a single agent, the agent implementerhas to absorb the knowledge of domain experts first, andthen reflects their ideas to agent implementation, which isnot efficient. Also, it is hard to switch the protocol accordingto the changing situations while performing a simulation.

In contrast, the approach shown in Figure 2, protocol de-scription is given in an executable protocol description lan-guage, and an external protocol interpreter interprets it andcontrols the agent internal model. In this approach, domainexperts can directly design protocols without consideringthe internal implementation of agents. Thus, domain ex-perts and agent implementers can independently develop amultiagent simulation system.

In this research, we propose an architecture shown in Fig-ure 3 that extends the one given Figure 2 by implementingboth protocol interpreters and agent internal models on alarge-scale agent server to achieve scalability. A large-scale

137

agent server can manage hundred-thousands of agents bykeeping agents as objects and by allocating threads to thoseobjects appropriately. As an example of such large-scaleagent servers, we describe Caribbean[10] in the followingsection.

Since protocol description and agent development are sep-arated in this approach as in Figure 2, protocol designers canchange protocols without knowing the detail of agent imple-mentation. The protocol interpreter requests the executionof sensing and actions in the protocol given to agents andreceives the result, which enables the dynamic switching ofprotocols given to agents.

3. FUNDAMENTAL TECHNOLOGIESWe have combined a scenario description language Q[3]

and a large-scale agent server Caribbean to build a platformfor large-scale multiagent simulations. Below, we describethe two technologies.

3.1 Scenario Description Language QQ1 is an interaction design language that describes how

an agent should behave and interact with its environmentincluding humans and other agents. For details see [3]. Inmodeling human actions, it has been shown that the Q ap-proach, describing the interaction protocol as a scenario, ismore effective than alternative agent description methodsthat simply describe the appearance of a human being [8].

The features of the Q language are summarized as follows.

• Cues and ActionsAn event that triggers interaction is called a cue. Cuesare used to request agents to observe their environ-ment. A cue has no impact on the external world.Cues keep waiting for the event specified until the ob-servation is completed successfully. Actions, on theother hand, are used to request agents to change theirenvironment. Cue descriptions begin with “?” whileaction descriptions begin with “!”.

• ScenariosGuarded commands are introduced for the case whereinwe need to observe multiple cues in parallel. A guardedcommand combines cues and actions. After one of thecues becomes true, the corresponding action is per-formed. A scenario is used for describing state tran-sitions, where each state is defined as a guarded com-mand.

• Agents and AvatarsAgents, avatars and a crowd of agents can be defined.An agent is defined by a scenario that specifies whatthe agent is to do. Avatars are controlled by humansso they do not need any scenario. However, avatarscan have scenarios if it is necessary to constrain theirbehavior.

In addition, a tool called Interaction Pattern Card (IPC)is introduced into Q to support scenario descriptions. Evencomputer novices can easily describe scenarios using thistool.

1Q is available from http://www.lab7.kuis.kyoto-u.ac.jp/Q/index_e.htm

Figure 4: Overview of Caribbean/Q

3.2 Caribbean Agent ServerCaribbean2 is a large-scale agent server implemented in

Java language.Caribbean manages agents as objects. There are two

types of objects in Caribbean, service objects and eventdriven objects. Objects in Caribbean communicate eachother using Caribbean messaging facility. Service objectscan be run at any time and are used for implementing suchmodules as databases with common information which arefrequently accessed. In contrast, event driven objects runsonly when they receive messages from other objects. Caribbeanscheduler allocates threads to event driven objects based onmessages. Usual modules in a system on Caribbean are im-plemented as this type of objects.

If threads are allocated to all the objects to run them con-currently, only up to one thousand objects can be run. In-stead, Caribbean enables executing objects of large numberby adequately selecting event driven objects to be allocatedthreads to.

Caribbean limits the number of objects in the memoryand controls the consumption of the memory, by swappingobjects between the memory the auxiliary store. When thenumber of objects on memory exceeds a limit, Caribbeanmoves objects that are not processing messages to the aux-iliary store. When objects in the auxiliary store receivemessages from other objects, Caribbean swaps them intothe memory to process the messages. By performing theseswapping efficiently, Caribbean manages a large number ofagents that cannot be stored in the system memory at once.

4. IMPLEMENTATION

4.1 Structure of Caribbean/QBy applying the proposed architecture, we build a scal-

able simulation environment that realizes the separation ofprotocol design and agent development and the dynamic

2Caribbean is available form http://www.alphaworks.ibm.com/tech/caribbean

138

switching of scenarios. We developed a large-scale multia-gent simulation platform, Caribbean/Q, by combining sce-nario description language Q and large-scale agent serverCaribbean based of the proposed architecture. Figure 4 de-picts the outline of the system.

A Q scenario describes an interaction protocol betweenan agent and the outer world. An example protocol is givenin Figure 5 as a state transition description. This protocolis of an agent that guides evacuees in disasters. A part ofthis state transition can be described in Q as shown in thedotted box in Figure 6.

The conventional processor of Q language, which is imple-mented in Scheme, cannot control enough agents to realizemassive navigation. Therefore, it is necessary to develop thenew processor of Q language which is available on the agentserver Caribbean.

Q language is an extension of Scheme, and a Q scenariohas been interpreted by the processor implemented in Scheme.In order to execute Q scenarios on Caribbean, which is im-plemented in Java, the approach is translating Q scenariosinto data structure of Java. This approach gets it easy tohandle scenarios on the agent server, and realized quick ex-ecution of scenarios. The translator which translates a Qscenario into a syntax tree object in Java is implementedin Scheme. This translation can be realized by parsing aQ scenario because syntax of Scheme, which is Q’s motherlanguage, is similar to data structure.

In Caribbean/Q, the Q translator takes a Q scenario asinput, and converts it to a syntax tree that is read by thestate machine object in Caribbean. The state machine exe-cute the converted syntax tree stepwise, by which the pro-tocol given in Q is executed. The scalability of Caribbeanis thus exploited by importing the Q processing system asevent driven object in Caribbean.

4.2 Execution of ScenarioSince the conventional processor of Q language, which is

implemented in Scheme, allocates one thread to one scenariointerpretation continuously, the number of controlled agentsis limited. Therefore it is impossible to control agents onan agent server, which are much more than threads, withthis processor. The proposed method in this research isto utilize event-driven mechanism of the agent server forscenario processing. This method realizes control of manyagents on the agent server with scenarios.

Both protocol interpreter and agent internal models areimplemented as event driven objects in Caribbean. Eachagent internal model object has one corresponding state ma-chine object. When switching scenarios, a new state ma-chine object that correspond to the new scenario is gener-ated and is allocated to the agent.

When the request for the execution of a scenario is givento a state machine object, message exchanges begin betweenthe object and the corresponding agent internal model ob-ject. First, the state machine object sends a request as aCaribbean message for the execution of cues or actions to theagent internal model object as a Caribbean message. Then,the agent internal model object execute the indicated cuesor actions against the environment, and sends a Caribbeanmessage to notify the state machine object of the result. Fi-nally, the state machine object receives the result reads thesyntax tree, converted by Q translator, and makes a transi-tion to the next state. By iterating this process, the given

Figure 5: Example of a Guide Agent Protocol

Figure 6: Q Scenario is Translated to the Corre-sponding Syntax Tree Using Q Translator

scenario is executed.Note that, during the execution of the scenario, the state

machine object only repeats sending request messages forthe execution of cues and actions and receiving result mes-sages. Agent internal model objects have only to processthese messages and the implementation inside the objects isentirely up to the developer.

Because of the separation of agent internal model objectsand state machine objects, the dynamic switching of pro-tocols become easy. Thus, experimenters can dynamicallyallocate appropriate protocols to agents according to thechanging situation in a simulation.

5. EVALUATIONIn this section, the performance of Caribbean/Q system

is evaluated. We compare the performance of the originalCaribbean system (Figure 7 (a)) and that of the Caribbean/Qsystem (Figure 7 (c)) to evaluate the trade off between thetwo merits of Caribbean/Q (the separation of protocol de-scription and agent development, and the dynamic switch-ing of protocols) and system performance. Also, by com-paring Caribbean/Q (Figure 7(c)) and an implementationwhere the original Q system is externally attached to con-trol Caribbean (Figure 7(b)), we validate the improvement

139

Figure 7: Configuration of the System for Evaluation: (a)Caribbean, (b)External Q Interpreter,(c)Caribbean/Q

in scalability. The computer used in the following experi-ment has Xeon 3.06GHz dual processors and 4GB memory,which is enough to keep all the Caribbean objects on mem-ory.

To test the performance that Caribbean/Q allocates sce-narios to agents, the following simple scenarios 3 with simplecues and actions 4 are used.¶ ³

(defcue ?receive)

(defaction !send)

(defscenario scenario ()

(scene1

((?receive) (!send) (go scene1))))

µ ´In this experiment, action counters are used to confirm

that all the agents execute an action before they go to thenext states, in order to guarantee that each agent executesthe uniform number of cues and actions and to avoid situa-tions where only a small set of agents run.

The chart in Figure 8 shows the relationship between thenumber of agents and the processing time for the agents toexecute 1,000,000 actions.

From Figure 8, the performance of Caribbean/Q is ap-proximately 1/4 of that of original Caribbean. This is be-cause one action of an agent in original Caribbean corre-sponds to one Caribbean message and that in Caribbean/Qcorresponds to four messages; the request for the observa-tion of a cue its result, the request for the execution of an

3In complex scenarios, the number of states and the numberof parallel observed cues increases. The increase in the num-ber of states does not affect the throughput, since a statetransition corresponds to a single edge in the syntax tree.The increase in the number of parallel observed cues doesnot affect the performance either, since it only increases thenumber of patterns that shows the names of cues returnedfrom agent internal model objects.4Complex cues and actions are not suitable to evaluate theperformance of Caribbean/Q to manage scenarios. Here,cue ?receive is observed right away, and action !send onlywrites “SEND” to the environment.

Figure 8: Evaluation Result of Platform

action, and its result. 5

The original Caribbean system requires that the data andthe functions of an agent are implemented to a single eventdriven object. In contrast, the implementation of an agentin Caribbean/Q is divided into two objects, a state machineobject and an agent internal model object, to separate pro-tocol description and agent internal model and to switchprotocols dynamically. This demonstrates that there is atrade-off between the two merits in developing multiagentsimulations and the performance.

As shown in Figure 8, the management of more than thou-sand agents failed in the implementation where the origi-nal Q interpreter is just attached externally to the originalCaribbean system as shown in Figure 7(b). In contrast,

5In this example, the ratio of the overhead to the executiontime of cues and actions is estimated relatively large, be-cause simple cues and actions are used. In real applications,cues and actions are more complex, and thus the ratio willbe smaller.

140

Figure 9: Screenshot of Large-Scale EvacuationNavigation System

Caribbean/Q successfully managed 1,000,000 agents. Theincrease in the number of agents does not affect the timeto process an action, which means the time to process thewhole system is proportional only to the cues and the actionsexecuted.

6. APPLICATION EXAMPLEIn this section, we describe a sample application of Carib-

bean/Q. We built a large-scale evacuation guidance system,which assumes a wide area disaster, as an example of so-cial systems, and performed a simulation by implementingagents that correspond to evacuees using Caribbean/Q.

In a guidance system which uses ubiquitous informationinfrastructure on a city, the system can acquire informationof each individual user in real time. However, quantity ofthe information becomes enormous. There occurs a prob-lem that a human who control system cannot handle all theinformation. Our approach is that a human gives rough nav-igation to agents and the agents give precise navigation toeach person. We aim at realizing a mega scale navigationsystem using GPS-capable cellular phones.

In the guidance system, the controller guides the evac-uees, each of whom has a cellular phone equipped with GPSreceiver, by using the control interface shown in Figure 9.The controller gives summary instructions to guiding agentsdisplayed on the map, and each guiding agent gives the pre-cise instructions to the corresponding evacuee. Figure 10depicts the structure of the simulation system. Importantmodules are described as below.

• Control interfaceThe controller instructs the guiding agents the direc-tion to evacuate through the control interface. In theinterface, the map of a wide area is displayed so thatthe controller view the current locations of evacuees.The controller can also assign evacuation sites, setplaces of shelters, and record the information aboutdangers such as fires.

Figure 10: Simulation of Large-Scale EvacuationNavigation System

On control interface, the distribution of people in thereal space is reproduced on the virtual space with hu-man figures based on positions of people acquired withsensors. The state of the virtual space is displayedon the monitor of the control center, so that the con-troller can grasp how people is moving in the real worldwidely through the bird-eye view of the virtual space.In addition, the controller can instruct particular peo-ple by handling human figures on the screen. The sys-tem notifies the people of the instructions using theirregistered phone numbers or e-mail addresses. Due tothis interface, it is possible to grasp situations of allpeople with global view and provide local navigationwith consideration of global coordination.

• Guiding agentsGuiding agents guides evacuees in the disaster area.An guiding agent instructs the corresponding evacuee.

These functions are implemented as functions of guid-ing agents.

– Receiving location information from a user’s GPSmobile phone, an agent sends a surrounding mapaccording to the user’s location. On this map,locations of places with damage such as fires, alocation of a shelter to evacuate to, and a direc-tion to head toward are described. The user sendshis/her location and gets a new map when he/sheneeds.

– An agent is instructed on a direction of evacu-ation by the control center. The agent retrievesshelters around the user, and selects a destinationaccording to the ordered direction and distancebetween the user and each shelter. If the destina-tion is changed by instructions, the agent notifiesthe user.

– If there exists a person who needs rescue, his/herplace is given to neighbor evacuees.

• Evacuee agentsIn the simulation systems, evacuee agents act for hu-man evacuees. The behavior of the evacuee agent isgiven as the following scenario. Actions and cues in

141

Table 1: Actions of Evacuee AgentAction Name Description!changeDirection Change the direction to head toward.!move Move along a road segment.!avoidDamage Select a next road intersection avoiding damage.!approachShelter Select a road intersection close to a shelter.!followDirection Select a road intersection following a given direction.!randomSelect Select a road intersection randomly.!finishEvacuation Finish evacuation.

Table 2: Cues of Evacuee AgentCue Name Description?notifiedStart Observe a message which triggers a step.?instructed Observe a direction instruction message.?dangerous Check if the current direction is approaching damage.?backShelter Check if the current direction is heading far away from a shelter.?finishMove Check if a move distance has amounted to a target value.?straggle Check if a current direction is against a given direction.?endEdge Check if an agent has reached a road intersection.?nearDamage Check if damage is near.?nearShelter Check if a shelter is near.?directed Check if an agent is instructed on a direction.?arriveShelter Check if an agent arrives at a shelter.

the scenario are defined as shown in Table 1 and Table2.¶ ³

(defscenario evacuation ()(wait ((?notifiedStart) (go evacuate))

((?instructed) (go instructed)))(instructed ((?straggle) (!changeDirection) (go wait))

(otherwise (go wait)))(evacuate ((?dangerous) (!changeDirection) (go move))

((?backShelter) (!changeDirection) (go move))(otherwise (go move)))

(move ((?arriveShelter) (!finishEvacuation))((?finishMove) (!finishMove) (go wait))((?endEdge) (go select))(otherwise (!move) (go move)))

(select ((?nearDamage) (!avoidDamage) (go move))((?nearShelter) (!approachShelter) (go move))((?directed) (!followDirection) (go move))(otherwise (!randomSelect) (go move))))

µ ´• Database of Environment

This system gets geographical information of the dis-aster area in virtual space from a database holding nu-merical value maps (1/25000) issued by the Geograph-ical Survey Institute. Evacuation navigations and dis-aster situations as entered through the control inter-face are recorded in this database a regular intervals.

In this prototype, evacuee agents are given a simple uni-form scenario. In future works, more complex situation issimulated by giving more variety of scenarios. Such scenar-ios will include ones that reflect social roles, such as firemenand police, individual contexts, such as injury, and so on.

7. CONCLUSIONIn this paper, we have proposed an architecture for large-

scale multiagent simulation platform. We implemented asystem that based on this architecture, evaluated it, andgave a sample application.

The problems we tackled in this work is as follows.

i. Separation of protocol design and agent devel-opmentThe architecture realizes the separation of protocol de-sign and agent development, which enables the expertsof different domains to cooperatively and efficiently de-velop large-scale multiagent simulation system.

ii. Dynamic switching of protocolsBy separating protocol processing system and agentinternal models, experimenters can easily switch pro-tocols according to the changing situations while run-ning the simulation.

iii. ScalabilityBy implementing both protocol processing system andagent internal models in a large-scale agent server,scalability of the system is improved.

The result of experiments shows that the Caribbean/Qsystem successfully manages simulations with 1,000,000 agents.However, to perform simulations more effectively, the speed-ing up is still necessary. To achieve it, technologies to dis-tribute a simulation among multiple computers and to per-form parallel is necessary. Besides the issue, we plan tostudy visualization methods of large-scale multiagent simu-lation and analysis methods.

AcknowledgmentWe would like to thank Mr. Gaku Yamamoto and Mr.Hideki Tai at IBM Japan Tokyo Research Laboratory, andMr. Akinari Yamamoto at Mathematical Systems Inc., fortheir various help. This work was supported by a Grant-in-Aid for Scientific Research (A)(15200012, 2003-2005) fromJapan Society for the Promotion of Science (JSPS).

142

8. REFERENCES[1] L. Gasser and K. Kakugawa. Mace3j: Fast flexible

distributed simulation of large, large-grain multi-agentsystems. In The First International Joint Conferenceon Autonomous Agents & Multiagent Systems(AAMAS-02), Bologna, 2002. ACM.

[2] O. Gutknecht and J. Ferber. The madkit agentplatform architecture. In Agents Workshop onInfrastructure for Multi-Agent Systems, pages 48–55,2000.

[3] T. Ishida. Q: A scenario description language forinteractive agents. IEEE Computer, 35(11):42–47,2002.

[4] T. Ishida. Society-centered design for sociallyembedded multiagent systems. In CooperativeInformation Agents VIII, 8th International Workshop(CIA-04), pages 16–29, 2004.

[5] T. Ishida, L. Gasser, and H. Nakashima, editors.Massively Multi-Agent Systems I. LNAI, 3446.Springer-Verlag, 2005.

[6] H. Kitano and et al. Robocup rescue: Search andrescue in large-scale disasters as a domain forautonomous agents research. In SMC, Dec. 1999.

[7] Y. Murakami, T. Ishida, T. Kawasoe, andR. Hishiyama. Scenario description for multi-agentsimulation. In Proceedings of the second internationaljoint conference on Autonomous agents and multiagentsystems (AAMAS-03), pages 369–376, 2003.

[8] Y. Murakami, Y. Sugimoto, and T. Ishida. Modelinghuman behavior for virtual training systems. In theProceedings of the Twentieth National Conference onArtificial Intelligence (AAAI-05), 2005.

[9] J. Odell, H. V. D. Parunak, and B. Bauer.Representing agent interaction protocols in UML. InAOSE, pages 121–140, 2000.

[10] G. Yamamoto and H. Tai. Performance evaluation ofan agent server capable of hosting large numbers ofagents. In AGENTS-01, pages 363–369, New York,NY, USA, 2001. ACM Press.

143

D-AESOP: A Situation-Aware BDI Agent System for Disaster Situation Management

J. Buford, G. Jakobson Altusys Corp.

Princeton, NJ 08542, USA +1 609 651 4500

{buford, jakobson}@altusystems.com

L. Lewis Southern New Hampshire U. Manchester, NH 03106, USA

+1 603 878 4876 l.lewis@snhu.edu

N. Parameswaran, P. Ray U. New South Wales

Sydney NSW 2052, Australia +61 2 9385 5890

{paramesh,p.ray}@cse.unsw.edu.au

ABSTRACT Natural and human-made disasters create unparalleled challenges to Disaster Situation Management (DSM). One of the major weaknesses of the current DSM solutions is the lack of a comprehensive understanding of the overall disaster operational situation, and very often making decisions based on a single event. Such weakness is clearly exhibited by the solutions based on the widely used Belief-Desire-Intention (BDI) models for building the Muiti-Agent Systems (MAS). In this work we describe D-AESOP (Distributed Assistance with Events, Situations, and Operations) situation management architecture to address the requirements of disaster relief operations. In particular, we extend the existing BDI model with the capability of situation awareness. We describe how the key functions of event correlation, situation recognition, and situation assessment could be implemented in MAS architecture suitable to the characteristics of large-scale disaster recovery. We present the details of a Situation-Aware BDI agent and the distributed service architecture of the D-AESOP platform.

Categories and Subject Descriptors

I.2.11 [Distributed Artificial Intelligence]: Intelligent Agents

General Terms Algorithms, Management, Design

Keywords Disaster relief operations, situation management, multi-agent systems, BDI agent model, FIPA

1. INTRODUCTION The tsunami generated by the Indian Ocean earthquake in December 2004 took an enormous toll of life and destruction, making it one of the deadliest disasters in modern history. Disasters create unparalleled challenges to the response, relief and recovery operations. Preparing, mitigating, responding to and recovery from natural, industrial and terrorist-caused disasters is a national priority for many countries, particularly in the area of building Disaster Situation Management (DSM) systems. DSM is defined as the effective organization, direction and utilization of counter-disaster resources, and comprises a broad set of activities of managing operations, people and organizations. Implementation of those activities involves information management, decision-making, problem solving, project and program planning, resource management, and monitoring and coordination. DSM is a complex multi-dimensional process involving a large number of inter-operating entities (teams of humans and systems) and is affected by various social, medical, geographical, psychological, political, and technological factors. From the information technology viewpoint the DSM processes can be hindered by lack of adequate, comprehensive and timely information; the presence of conflicting goals, policies, and priorities; lack of effective coordination between different rescue operations augmented with the inability of many units to act autonomously. The lessons learned from major natural and man-made disasters demonstrate the acute need for innovative, robust, and effective solutions to cope with the scale, unpredictability, and severity of various disaster situations. While significant research and engineering results have been demonstrated on the lower sensor-motor level of DSM systems, the high-level modeling of the behavior of these systems, including modeling of the world, recognition and prediction of emerging situations, cognitive fusion of events, and inter-component collaboration, is still far from solution. This paper discusses the Multi-Agent Systems (MAS) approach of building DSM, focusing mainly on post-incident operations related to modeling, understanding, and reasoning about the disaster situations. Previously we described a case study of urban transportation threat monitoring using situation management [2]. In the post-incident recovery phase, a situation manager uses the situation models and event data captured during the preventive and deployment phases to help operations staff characterize the scope of incident, deploy evacuation

144

resources, communicate to the public, control and contain the perimeter, manage recovery and cleanup, and collect forensics. MAS approach [16] has proven to be an effective solution for modeling of and reasoning about complex distributed and highly dynamic unpredictable situations, including the ones happening in modern battlefields, homeland security, and disaster management applications. There are several characteristics of

agent system behaviour, which make agents appropriate models for DSM applications, namely (a) Autonomous behaviour – capability to act independently in

a persistent manner defined either by its own agenda or by an agenda posted by a higher or peer-level entity, an another agent or a supervisor;

(b) Rational behaviour – a goal-directed capability to sense the world, reason about the world, solve problems, and, consequently, to affect the world;

(c) Social behaviour – an agent’s recognition of its role within a community of other agents exhibited by its capabilities to communicate, cooperate, share data and goals, as well communicate with humans.

(d) Spatial and temporal behaviour – agents are situated in an environment, either the physical or the virtual one; they move in space and time with the capabilities of sensing both of these dimensions of the World.

There are several other features of agents like the abilities of learning, self-organization, and resource management, which, although important ones, are out of the scope of this paper. The need to model the intelligent acts of perception of the operational environment, goal-directed behavior, and reasoning about the environment, prompted the MAS community to use

the well-known conceptual architecture of Belief-Desire-Intention (BDI) agents [1][16]. Since its inspection, BDI model has experienced several functional advancements and software implementations, however recent attention to large-scale disaster relief operations, homeland security tasks, and management of asymmetric network-centric battlespaces have revealed the weakness of the current BDI model, namely the weakness to cope with the fast moving, unpredictable and complex

operational situations. The major reason for this weakness in the BDI model is two-fold: (a) a relatively simple reactive paradigm “Event-Plan”(EP paradigm), where plans are invoked by a single event, and (b) lack of synergy between the reactive plan invocation and plan deliberation processes. In this work we propose to extend the BDI model with the capability of situation awareness, particularly, we propose “Event Situation Plan” paradigm (ESP paradigm), where plans are invoked not as a response to a single event, but are generated based on recognized dynamic situations. This dynamic situation recognition process is carried out by two synergistic processes: reactive event correlation process and deliberative analogy-based plan reasoning process. We can refer to several recent approaches and experimental systems which use the elements of situation awareness, and which experiment with the tasks of disaster management and medical emergency situation management, including using the methods of high-level data fusion and situation analysis for crisis and disaster operations management [9][11], knowledge-driven evacuation operation management [15], and urban transportation threat monitoring [2]. In this paper we describe how the proposed ESP paradigm for the BDI agent is mapped to FIPA [5] compliant D-AESOP system, a distributed multi-agent situation management platform being developed by Altusys.

Disaster Relief Operations

Medical Relief Operational Space

Mobile First Aid Evacuation Hospital Operations

High-LevelGoals, Policies

Constraints

SituationalEvents

Geographic& Weather Information

Disaster SituationAssessment

Damage CasualtiesMedical SuppliesRoads

CommunicationInter-situational RelationsSide Effects (Epidemic,

Weather, Panic, Law & Order)

CorrelationMeta Data

AdditionalAdjusted Data

Requests

Real-TimeOperationsFeedback

DisasterSituation

Model

SituationRefinement

Request

Human IntelligenceBuilding damage

CasualtiesSupplies needed

Reports fromPolice, Emergency Units,

AuthoritiesEyewitness accounts

Signal IntelligenceEmbedded Sensors

Satellite imagesAerial Images (UAV, planes, etc)Distributed, chemical, biological

Video, etc. sensors andSensor networks

Disaster Data Collection

Plans/Actions/Decisions

ProgressReports

Relief OperationsDecision SupportFirst aid delivery

Mobile ambulatoriesHospital selection

Hospital operationsTransportation selection

Dispatch of medical teamsRouting

Supplies planningBackup scenarios

InformationCorrelationTemporal

SpatialStructuralMedical

Environmental

OperationsImplementation

SchedulingCoordinationMonitoring

Relief Operations Plans & ProceduresRegulartory and Legal Requirements

Organizational jurisdictionsOrganizational capabilities

Maps, roads, medical facilities

Disaster Relief Operations

Medical Relief Operational Space

Mobile First Aid Evacuation Hospital Operations

High-LevelGoals, Policies

Constraints

SituationalEvents

Geographic& Weather Information

Disaster SituationAssessment

Damage CasualtiesMedical SuppliesRoads

CommunicationInter-situational RelationsSide Effects (Epidemic,

Weather, Panic, Law & Order)

CorrelationMeta Data

AdditionalAdjusted Data

Requests

Real-TimeOperationsFeedback

DisasterSituation

Model

SituationRefinement

Request

Human IntelligenceBuilding damage

CasualtiesSupplies needed

Reports fromPolice, Emergency Units,

AuthoritiesEyewitness accounts

Signal IntelligenceEmbedded Sensors

Satellite imagesAerial Images (UAV, planes, etc)Distributed, chemical, biological

Video, etc. sensors andSensor networks

Disaster Data Collection

Plans/Actions/Decisions

ProgressReports

Relief OperationsDecision SupportFirst aid delivery

Mobile ambulatoriesHospital selection

Hospital operationsTransportation selection

Dispatch of medical teamsRouting

Supplies planningBackup scenarios

InformationCorrelationTemporal

SpatialStructuralMedical

Environmental

OperationsImplementation

SchedulingCoordinationMonitoring

Relief Operations Plans & ProceduresRegulartory and Legal Requirements

Organizational jurisdictionsOrganizational capabilities

Maps, roads, medical facilities

Figure 1 Closed-Loop Post-Disaster Medical Relief Operations Management using DSM

145

The rest of the paper is organized as follows. The next section describes an overall model for DSM applied to medical relief operations. We then decompose the DSM model into a multi-agent system. The following two sections describe the agent model using the Belief-Desire-Intention paradigm and a medical relief ontology for the BDI agents respectively. We then describe a realization of the MAS architecture in the D-AESOP situation management platform.

2. MEDICAL RELIEF OPERATIONS

2.1 General Scenario From the overall picture of post-incident DSM we focus in this paper on the medical relief operations. Medical relief operations are a critical element of DSM. They provide treatment and support to those injured due to the disaster or whose previously sustained conditions or vulnerability (e.g., elderly or displaced patients) place them in medical jeopardy due to the disaster. The major medical relief operations include (a) Overall planning of the medical recovery efforts, and

coordination of medical, rescue, supply and other teams;

(b) Dispatching, scheduling, and routing of mobile emergency vehicles;

(c) Field mobile ambulatory aid;

(d) Evacuation of victims;

(e) Emergency hospital operations coordination, and

(f) Logistics support for medical supplies and equipment.

Medical relief operations are characterized by a significant distribution of data across teams of people, systems, information sources, and environments, and the ongoing data collection and changing state makes the overall picture very dynamic. Further, there is a strong benefit to the overall effort if different teams can share relevant information. For example, in order to perform effective provisioning of field medical services, the mobile ambulatory teams need to develop a common understanding of the medical situation on the ground, share road and access information, and coordinate medical relief and evacuation operations.

2.2 Disaster Management Model: From Situation Perception to Actions The DSM (Figure 1) constructs a real-time constantly refreshed Situation Model from which relief operations can be planned and updated. The Situation Model contains a knowledge-level view of the disaster from the medical relief perspective, using an ontology specifically designed for that domain. The model is created and updated by a constant flow of events and reports collected from the operational space. These events include both human intelligence and signal intelligence. Because of the large amount of raw data being collected, the event stream needs to be processed and correlated to produce “situational events”, i.e., events at the domain level. This reduction and inference step is performed by an information correlation stage. We describe later in the paper how the information correlation function and the situation assessment function can be distributed in a Multi-Agent System (MAS) architecture.

Integrated with the real-time Situation Model are decision support systems (DSS) for medical relief operations. The DSS rely on the Situation Model and operations staff oversight to manage the scheduling, dispatching, routing, deployment, coordination and reporting tasks. A chain of distributed communication and control systems leads from the DSS to the medical personnel in the field to direct the execution of these tasks. Medical relief organizations have the responsibility and expertise to prepare for disaster recovery in many different scenarios, which includes defining goals and policies, enforcing legal and regulatory requirements, and specifying deployment plans. These goals, policies, requirements and plans are incorporated into the DSM knowledge base and are used by the situation awareness function and DSS to ensure plans, actions, and priorities are formed consistently. The situation assessment function is assumed to use techniques for reasoning with incomplete information characteristic of this type of environment. These techniques permit incomplete and possibly inconsistent situations with different probabilities and event support to be maintained and changed as new information arrives.

2.3 Feedback to Refine the Disaster Situation Model The DSM (Figure 1) adapts to requests and feedback from medical relief personnel and DSS, as indicated in the reverse path. These requests can lead to refinement of the Situation Model, meta-level guidance to the information correlation function, and a focus on specific sensor data.

3. SITUATION-AWARE BDI AGENT SYSTEMS APPROACH TO DSM

3.1 Basic Principles of the Approach We see situation management as a closed-loop process, where primary information is sensed and collected from the managed operations space (the World), then analyzed, aggregated, and correlated in order to provide all required inputs for the situation recognition process. During the next step the reasoning processes are performed to select predefined plans or automatically generate them from the specifications embedded in the situations. It is assumed that all the mentioned steps are performed by agents of the MAS. Finally the actions are performed to affect the World. As the World gets affected, new information about the World is sensed and the proccess is repeated. Having such iterative control loop cycle is an important element of our approach. One of the important aspects of using MAS for DSM is that the concept of an agent takes two embodiments: the physical embodiment, e.g., the mobile emerency vehicles, and virtual embodiment of software agents. Concequently, the DSM environment creates an interesting subtask, namely mapping the physical agents (vehicles, robots, human teams, etc.) into the abstract framework of MAS. This task involves several enginering considerations, including energy consumption, relative autonomy of physical agents, information sharing, security, etc.

146

The natural structure of the DSM operations prompts the MAS organization, where distributed agent teams (communities) having peer-to-peer decentralized internal communication among the agents, are controlled externally by a higher-level control agent.

As was mentioned in the Introduction, one of the major contributions of this paper is the definition of the Event-Situation-Plan (ESP) paradigm, which drives invocation of a plan in BDI model not directly by an event, but via the situation recognition process. In our approach, we see two synergistic processes (Figure 2), the Reactive Situation Recognition Process enabled by Event Correlation (EC) and Deliberative Plan Reasoning Process driven by Case-Based Reasoning (CBR). Both processes work in a loop, where the primary situations recognized by EC might be refined and combined by the CBR and EC might get context-sensitive meta-situations in order to proceed with the event correlation process. In case of incomplete information, EC might pass requests (queries) to event collection procedures for additional information. One can see (Figure 2) a local loop in the Deliberative Plan reasoning Process, where sub-plans of a plan can trigger an iterative plan deliberation process. The EC and CBR processes will be discussed later in this section.

Figure 2 Reactive Situation Recognition and Deliberative

Plan Reasoning Processes

3.2 Abstract BDI Agent Architrcture The Belief-Desire-Intension (BDI) model was conceived as a relatively simple rational model of human cognition [1]. It operates with three main mental attitudes: beliefs, desires and intentions, assuming that human cognitive behaviour is motivated by achieving desires (goals) via intentions providing the truthfulness of the beliefs.

As applied to agents, the BDI model got concrete interpretation and first order logic based formalization in [13]. Among many BDI agent models, the dMARS formalism serves as a well-recognized reference model for BDI agents [4]. Since we use the dMARS framework as a starting point to our approach on situation-aware BDI asgents, we will informally sketch the basic notions of dMARS. A BDI agent is built upon the notions of beliefs, desires, events, plans and intensions.

Beliefs are the knowledge about the World that the agent posesses and believes to be true. Beliefs could be specifications of the World entities, their attributes, relations between entities, and states of the entities, relations. In many cases, the agent’s beliefs include the knowledge about other agents as well as models of itself. Desires are agent’s motivations for actions.

Two kinds of activities are associated with the desires: (a) to achieve a desire, or (b) prove a desire. In the first case, by applying a sequence of actions the agent wants to reach a state of the world, where the corresponding desire formula becomes true, while in the second case, the agent wants to prove that the world is or isn’t in a particular state by proving that the corresponding belief formula is true or not. Often desires are called goals or tasks.

Plans are operational specifications for an agent to act. An agent’s plan is invoked by a trigger event (acquisition of a new belief, removal of a belief, receipt of a message, acquisition of a new goal). When invoking a plan, an agent tests whether the plan invocation pre-conditions are met, and tests run-time conditions during the plan execution. The actions in the plan are organized into an action control structure, which in dMARS is a tree-like action flow. Actions could be external ones, essentially procedure calls or method invocations; or internal ones of adding and removing of beliefs. Abstract plans are stored in the agent’s plan library. During agent operations certain abstract plans are selected from the library and instantiated depending on variable bindings, substitutions and unifications.

An agent’s intention is understood as a sequence of instantiated plans that an agent is commited to execute. Always while responding to a triggering external event, an agent is invoking a plan from the plan library, instantiating it and pushing into a newly created stack of intentions. Contrary to that, when an agent reponds to an internal triggering event, i.e., an event created by an internal action of some previous plan instance, then the new plan instance is pushed onto the stack of the previous plan that caused the invocation of the new plan instance. An abstract architecture of BDI agent is presented on Figure 3

Figure 3. Abstract BDI Architecture (Motivated by dMAS

Specification [4]) Situation-Aware BDI Agent

147

3.3 Situation-Aware BDI Agent: Abstract Architecture In this section we discuss how the basic principles of our approach, discussed in the Section 3.1 are mapped in the abstract architecture of the situation-aware BDI agent. The current BDI models have a simple plan invocation model, where either the plan is triggered by a single event or by a single goal. Preference between these two invocation methods leads to event or goal-directed planning of actions. While the single goal directed planning usually satisfies the application’s needs, single event directed planning does not. In the majority of cases of disaster operations planning, battlefield management, and security applications, decisions are made not on the basis of a single event, but rather correlating multiple events into a complex event and mapping it to a situation happening in the operational space. The central piece of our approach to extending the capabilities of the BDI agent is the introduction of situation awareness. According to the proposed approach, a plan will be invoked by a situation, rather than by a single event (Figure 4).

The flow of multiple external events received by the BDI agent and the events generated by the agent itself, while executing the plans, are correlated into compound high-level events called synthetic events. The real-time event correlation process [6] takes into account temporal, causal, spatial, and other domain-specific relations between the multiple events as well constraints existing between the information sources producing the events.

The introduction of the paradigm of plan invocation by a situation has specific importance to the disaster situation management domain, since plans can now take account the patterns of multiple events.

The event correlation process could be an iterative multi-stage process, where some synthetic events could be used for building more complex synthetic events.

Figure 4. Situation Aware BDI Agent

The synthetic events serve as a basis for recognizing situations taking place in the world. They are used in the triggering patterns of abstract situations while invoking them from the Situation Library. The abstract situations are instantiated and are combined into an overall situational model of the world.

The situations contain either references to the plans that will be invoked by triggering conditions specified in the situations, or contain specifications for reasoning and generating plans. The

steps of plan instantiation and execution are similar to those performed in the dMAS BDI model.

3.4 Event Correlation Process in BDI Agents Event correlation is considered to be one of the key technologies in recognizing complex multi-source events. We are using event correlation as a primary tool leading to situation recognition. As shown later, the importance of the event correlation process influenced us to introduce a special type of event correlation agent.

The task of event correlation can be defined as a conceptual interpretation procedure in the sense that a new meaning is assigned to a set of events that happen within a predefined time interval [7]. The conceptual interpretation procedure could stretch from a trivial task of event filtering to perception of complex situational patterns occurring in the World. The process of building correlations from other correlations allows the formation of a complex fabric of multiple inter-connected correlation processes, suitable for the paradigm of integrated distributed cognition and collective behavior that is proposed here. Intermixing between different correlation connections creates a flexible and scalable environment for complex situation modeling and awareness solutions.

The temporal model-based event correlation technology used in this work has been developed and implemented for managing complex telecommunication networks. More about the details of the technology could be found [6, 7].

3.5 Case-Based Reasoning Process in BDI Agents Our approach to an agent plan deliberation process is to use a specific model of reasoning called case-based reasoning (CBR), where a case is a template for some generic situation [8] [11]. The formation of the library of standard case templates for representing the typical generic situations allows (a) construction of specific DSM models by selecting the appropriate case templates, (b) modifying and instantiating the selected cases with concrete parameter values, and (c) combining the instantiated cases into overall case representation of the situation. Further, the CBR approach enables learning from experience and adapting more-or-less standard situations to accommodate the nuances of current situations.

4. SITUATION AWARENESS IN D-AESOP

4.1 Modeling Structural and Dynamic Features of Situations Understanding the situations happening in dynamic systems requires modeling of main human cognitive processes, i.e., perception, memory, problem solving and learning [2]. These tasks should be undertaken in a dynamic environment, where events, situations and actions should follow the structural, spatio-temporal, and conceptual relations and constraints of the domain. Modeling situations has been in the research focus of several scientific disciplines, including operations research,

148

ergonomics, psychology, and artificial intelligence (AI). Most notably, John McCarthy and Patrick Hayes introduced the notion of Situation [10], where situations were considered as snapshots of the world at some time instant, while a strict formal theory was proposed in [12].

Informally, we will describe situations as aggregated states of the entities and the relations between the entities observed at some particular discrete time moment or time interval. The key prerequisite for successful situation management is the existence of an adequate situational model of the real situation to be managed. Many application areas deal with large and complex systems containing thousands of inter-dependent entities. While dealing with situations and situation modeling in D-AESOP, there are three important aspects:

(a) Structural aspects of situations: collections of entities forming the situations, the relations between the situations, construction of situations from components, and organization of situations at the conceptual level into situation ontologies

(b) Dynamic aspects of situations: how situation entities and relations change in time, how transitions happen between situations, how temporal relations between the events effect situation transitions, and

(c) Representational aspects of situations: how to describe situations and their dynamic behavior, how to represent situations to humans in an understandable and efficient way, and how to program situations.

4.2. Situation-Driven Plan Invocation As mentioned in the Section 3.3, plans are invoked by situations. D-AESOP identifies several alternative solutions here, including direct plan invocation by an action (method) embedded in the situation, conditional plan selection, and automatic plan generation by a reasoning procedure. In the following example we describe a situation recognition process and direct invocation of a plan by an embedded action An emergency situation could be recognized using event correlation rules (The rule is described in a language similar to CLIPS [3]). Suppose an event of type A was issued at time t1 from a dispatched medical emergency vehicle (MEV) ?mev1, but during the following 10-minute interval an expected event of type B was not issued from another MEV ?mev2. The events to be correlated, then, are A and not-B. Note that not-B is treated formally as an event. An additional constraint is that MEV ?mev1 and ?mev2 belong to a team. This constraint is expressed by a grouping object GROUP with identified group type and parameters. The time constraint between events A and not-B is implemented using a temporal relation AFTER.

CorrelationRuleName: EXPECTED-EVENT-RULE Conditions: MSG: EVENT-TYPE-A ?msg1 TIME ?t1 VEHICLE: VEHICLE-TYPE-MEV ?mev1 Not MSG: EVENT-TYPE-B ?msg2 TIME ?t2 VEHICLE: VEHICLE-TYPE-MEV ?mev2 GROUP: GROUP-TYPE-MEV ?mev1 ?mev2 AFTER:?t1 ?t2 600 Actions:

AssertSituation: LOST-MEV-CONTACT-SITUATION VEHICLE1 ?mev1 VEHICLE2 ?mev2 EVENT1 ?msg1 EVENT2 ?msg2

If the conditions of the rule EXPECTED-EVENT-RULE are true, then the situation LOST-MEV-GROUP-CONTACT-SITUATION is asserted into the event correlation /situation recognition process memory.

Below is given a relatively simple association between a situation and plan, where the situation LOST-MEV-GROUP-CONTACT-SITUATION has an embedded action (method), which invokes plan SEND-EMERGENCY-HELICOPTER. SituationName LOST-MEV-CONTACT-SITUATION

SituationClass MEV-SITUATION Parameters VEHICLE1 VEHICLE2 EVENT1 EVENT2 ……… Actions

PLAN SEND-EMERGENCY-HELICOPTER

5. DISTRIBUTED AESOP PLATFORM FOR IMPLEMENTING DSM SYSTEM

5.1 Instantiation of the Abstract Agent Model into MAS The abstract architecture of the situation aware BDI agent describes the conceptual level of the processes occurring in the BDI agents. Here is a mapping of those abstract features into concrete functions of the agents in MAS. In our approach the abstract features of the BDI agents are mapped into the following categories of agents: (a) Agents-Specialists (event correlation, situation awareness, and plan generation agents, (b) Perception and Information Access Agents, (c) Interface Agents, and (d) Belief System Management Agents. An important task of the system design is representation of the physical DSM agents (vehicles, teams of humans, hospitals, etc) in the MAS framework, i.e., mapping from the physical agent level onto MAS agent level. We are not going to discuss this issue in detail, since it is out of the scope of this paper.

5.2 Agent and Agent Platform Interoperability A distributed agent architecture is highly suitable for the disaster recovery environment because it is inherently adaptive to the topology and capability of the collection of agent nodes which are distributed according to field operations, medical centers, and control centers. However such environments might bring together multiple dissimilar agent platforms as shown in Fig. 5(a). Rather than a single agent platfrom across all systems, a more likely scenario is hetereogeneous agent platforms that have been developed for different facets of disaster relief. In order for heterogeneous agent platforms to interoperate in such a

149

dynamic environment, there must be agreement on message transport, communication language, and ontology.

Figure 5. (a) Multiple heterogeneous agent platforms in disaster recovery (b) abstract FIPA AP architecture [5]

FIPA (Foundation for Intelligent Agents) [5] provides interoperability between agent platforms and a directory mechanism by which agent platforms can discover other agent services (Fig. 5 (b)). Important features of FIPA specifications include 1) a generic message transport by which FIPA-compliant agent platforms (AP) connect, 2) ability for nomadic agents to adapt to changing network conditions using monitor and control agents, 3) a formal agent communication language based on a set of defined communicative acts and a method by which agents can establish a common ontology for communcation. In additional, FIPA defines an experimental specification for an ontology service by which agents can share an ontology and translate between different ontologies. Note that FIPA does not currently specify ontologies for application domains. The D-AESOP system described next is intended to be implemented using FIPA compliant agent platforms to support this interoperability.

5.3 AESOP: Distributed Service Architecture Based MAS Implementation The foundation for implementation of the DSM system is distributed AESOP (Assistance with Events, Situations, and

Operations) service architecture (see Fig. 6). D-AESOP identifies several classes of agents as discussed in the previous section, with specific customizations, which reflect the idiosyncrasies of the DSM domain. These agent classes are: Disaster Information Access Agents, Relief Teams Communication/Interface Agents, DSM Agents-Specialists and DSM Belief Management Agents. Each agent is an embodiment of a service within D-AESOP. The use of standard services with well-defined functionality and standard inter-component communication protocols allows the building of open, scalable, and customizable systems. The encapsulation of the idiosyncrasies of components and the use of functions of addition, replication, and replacement of services provides an effective environment for developing multi-paradigm, fault-tolerant, and high-performance systems. The Disaster Information Access Agents, Relief Teams Communication/Interface Agents and DSM Agents-Specialists are inter-connected via fast event transfer channel, while the agents-specialist are getting the required knowledge and data from the DSM Belief Management Agents via online data and knowledge transfer channel. D-AESOP uses Core System Services such as Naming, Directory, Time, Subscription, and Logging services, which are used as the major services to build the DSM services. Different instances of the services can be used as long as they satisfy overall functional and semantic constraints. For performance or functional reasons, multiple processes of the same service could be launched. For example, a hierarchy of Event Correlation Services could be created. This hierarchy could be used to implement a multilevel event correlation paradigm, e.g., to implement local and global correlation functions.

6. CONCLUSION In this paper we described an MAS approach to DSM. The central part of our approach is the introduction of the concept and model of situation awareness into the environment of BDI agent based MAS. The DSM is very demanding and challenging domain from the viewpoint of IT solutions, and is complicated by several social, political, organizational and other non-IT aspects. From the research described in this paper but also from the results of many other research and development projects, it is obvious that despite the achieved results, many issues of comprehensive, effective and secure DSM need yet to be solved, including advancement of the MAS models discussed in this paper. We refer to some of them: optimal mapping from the physical infrastructure of DSM agents (vehicles, robots, human teams, etc. into the abstract framework of MAS; advancement of the agent capabilities to recognize complex situations reflecting temporal, causal, spatial, and other domain specific relations; exploration of MAS with self-adaptation, learning, and situation prediction capabilities; and deeper understanding the rules, policies, and behavioral constraints among the agents.

150

REFERENCES [1] Bratman, M. Intension, Plans, and Practical Reason.

Harvard University Press, 1987. [2] Buford, J., Jakobson,G., and Lewis, L. Case Study of

Urban Transportation Threat Monitoring Using the AESOP Situation Manager™. 2005 IEEE Technologies for Homeland Security Conference, Boston, MA., 2005.

[3] CLIPS 6.2. http://www.ghg.net/clips/CLIPS.html [4] d'Inverno, M., Luck, M., Georgeff, M., Kinny, D., and

Wooldridge, M. The dMARS Architechure: A Specification of the Distributed Multi-Agent Reasoning System, in Journal of Autonomous Agents and Multi-Agent Systems, 9(1-2):5-53, 2004.

[5] FIPA. FIPA Abstract Architecture Specification. SC00001L, Dec. 2003.

[6] Jakobson, G. Buford, J., and Lewis, L. Towards an Architecture for Reasoning About Complex Event-Based Dynamic Situations, Intl Workshop on Distributed Event based Systems DEBS’04, Edinburgh, UK, 2004.

[7] Jakobson, G., Weissman, M. Real-Time Telecommunication Network Management: Extending Event Correlation with Temporal Constraints. Integrated Network Management IV, IEEE Press, 1995.

[8] Lewis, L. Managing Computer Networks: A Case-Based Reasoning Approach. Artech House, Norwood, MA, 1995.

[9] Llinas, J. Information Fusion for natural and Man-made Disasters. in Proc of the 5th Intl Conf on Information Fusion, Sunnyvale CA, 2002, 570-574.

[10] McCarthy, J. and Hayes, P. Some philosophical problems from the standpoint of artificial intelligence. In Donald Michie, editor, Machine Intelligence 4, American Elsevier, New York, NY, 1969.

[11] Pavón, J., Corchado E. and Castillo L. F. Development of CBR-BDI Agents: A Tourist Guide Application. in 7th European Conf on Case-based Reasoning, Funk P. andGonzález Calero P. A. (Eds.) Lecture Notes in Computer Science, Lecture Notes in Artificial Intelligence (LNAI 3155), Springer Verlag., 2004, 547-555.

[12] Pirri, F. and Reiter, R. Some contributions to the situation calculus. J. ACM, 46(3): 325-364, 1999.

[13] Rao, A and Georgeff, M. BDI Agents: From Theory to Practice. In Proc of the First Intl Conf on Multiagent Systems (ICMAS’95), 1995.

[14] Scott, P. and Rogova, G. Crisis Management in a Data Fusion Synthetic Task Environment, 7th Conf on Multisource Information Fusion, Stockholm, 2004.

[15] Smirnov, A. et al. KSNET-Approach Application to Knowledge-Driven Evacuation Operation Management, First IEEE Workshop on Situation Management (SIMA 2005), Atlantic City, NJ, Oct. 2005.

[16] Wooldridge, M. An Introduction to Multi-Agent Systems. John Wiley and Sons, 2002.

Core AESOP System Services (Naming, Directory, Time, Property, Subscription, Logging, Scripting, etc.)

AESOP Java Platform (J2EE)

Data & KnowledgeTransfer Channel

SituationAwareness

Agents

EventCorrelation

Agents

VehicleRoutingAgents

ReliefPlanningAgents

DSM Agents Specialists

DatabaseManagement

Agents

OntologyManagement

Agents

PlansManagement

Agents

RulesManagement

Agents

DSM BeliefManagement

Agents

Fast EventsTransfer Channel

Relief TeamsCommunication/Interface Agents

SensorManagement

Agents

ReportsManagement

Agents

DisasterInformation

Access Agents

HumanInterfaceAgents

Vehicle/TeamCommunication

Agents

EventNotification

Service

KnowledgeAcquisition

Service

Core AESOP System Services (Naming, Directory, Time, Property, Subscription, Logging, Scripting, etc.)

AESOP Java Platform (J2EE)

Data & KnowledgeTransfer Channel

SituationAwareness

Agents

EventCorrelation

Agents

VehicleRoutingAgents

ReliefPlanningAgents

DSM Agents Specialists

DatabaseManagement

Agents

OntologyManagement

Agents

PlansManagement

Agents

RulesManagement

Agents

DSM BeliefManagement

Agents

Fast EventsTransfer Channel

Relief TeamsCommunication/Interface Agents

SensorManagement

Agents

ReportsManagement

Agents

DisasterInformation

Access Agents

HumanInterfaceAgents

Vehicle/TeamCommunication

Agents

EventNotification

Service

KnowledgeAcquisition

Service

Figure 6 Distributed AESOP Platform for DSM

151

Role of Multiagent System on Minimalist Infrastructure for Service Provisioning in Ad-Hoc Networks for Emergencies

Juan R. Velasco

Miguel A. López-Carmona Marifeli Sedano Mercedes Garijo

David Larrabeiti María Calderón

Departamento de Automática Universidad de Alcalá

Edificio Politécnico – Crtra N-II, Km. 31,600 – 28871 Alcalá de Henares

(Spain) {juanra|miguellop}@aut.uah.es

Departamento de Ingeniería de Sistemas Telematicos

Universidad Politécnica de Madrid ETSI Telecomunicación – Ciudad Universitaria, s/n – 28040 Madrid

(Spain) {marifeli|mga}@dit.upm.es

Departamento de Ingeniería Telemática

Universidad Carlos III de Madrid Escuela Politécnica Superior – Av. Universidad, 30 – 28911 Leganés

(Spain) {dlarra,maria}@it.uc3m.es

ABSTRACT In this position paper, the agent technology used in the IMPROVISA project to deploy and operate emergency networks is presented. The paper begins by describing the main goals and the approach of IMPROVISA. Then we make a brief overview of the advantages of using agent technology for the fast deployment of ad-hoc networks in emergency situations.

Categories and Subject Descriptors C.3 [Special-Purpose And Application-Based Systems] - Real-Time and Embedded Systems

General Terms Performance, Security, Human Factors.

Keywords Catastrophes; multi-agent systems; semantic ad-hoc networks; intelligent routing; multilayer-multipath video transmission.

1. INTRODUCTION IMPROVISA Project (from the Spanish translation of “Minimalist Infrastructure for Service Provisioning in Ad-hoc networks”) addresses the issue of real service provisioning in scenarios lacking a fixed communications infrastructure, where the cooperation of humans and electronic devices (computers, sensors/actors, robots, intelligent nodes, etc) is paramount. As an example of this sort of scenario, emergency management in natural catastrophes will be used. Besides fixed infrastructure such as cellular 3G networks, for mobility reasons we shall also exclude satellite communications from the target scenario –although it may be available in a subset of nodes–. This assumption is specially valid for in-door rescue squadrons, communication with personal devices, or in dense forest zones.

This target scenario introduces a number of challenges at all layers of the communication stack. Physical and link layers are still under study and rely mainly on the usage of technologies such as OFDM (Orthogonal Frequency Division Multiplexing), phased-array antennas, and FEC (Forward Error Correction) techniques. Technological solutions to the routing problem can be found in the field of ad-hoc networking. Mobile Ad-hoc Networks (MANETs) [5] are made up of a set of heterogeneous, autonomous and self-organising mobile nodes interconnected through wireless technologies. Up to the date, most of the research in MANETs has focused on the design of scalable routing protocols. However, very few complete prototype platforms have shown the effectiveness of the ad-hoc approach for emergency support.

This project is focused on the development of real concrete application-oriented architectures, by the synergic integration of technologies covering practical ad-hoc networking, security frameworks, improved multimedia delivery, service-oriented computing and intelligent agent platforms that enable the deployment of context-aware networked information systems and decision support tools in the target scenario. This position paper focuses on the role of the multiagent system into the main architecture.

Figure 1 Networks for disaster management

Volunteer

Firemen

Firemen

HOSPITAL

Ambulance

Ambulance

HospitalPolice

Civil services

Emergency

112

Firemen

Watch

Ad-hoc Network Data lines

Police Headquarters

Field

Team

Team

Chief

Tower

152

Figure 1 shows how different groups of professional and volunteer workers act together to solve a natural catastrophe. Computers (in different shapes) are everywhere: every worker has their own PDA; cars, ambulances or helicopters have specific computer and communication systems; and central points, like hospital, army or civil care units have their main servers. The goal of the project is to develop a general architecture that may be used over any situation where conventional communications are not allowed: apart from disasters, during terrorist attacks GSM communications are disabled to prevent GSM-based bomb activation.

2. AD-HOC NETWORKS AND MULTI-AGENT SYSTEMS

Agents may be used into three main aspects:

One of the main problems on ad-hoc networks is how to route information across the network ([9] and [2]). In our scenario, different groups may be far one from the other, so routing problem has to deal with separate ad-hoc networks. We plan to work over intelligent routing and the use of an upper level intelligent system to use other mobile systems, like cars or helicopters that move over the area to “transport” data among the separate ad-hoc networks. In this case, security is one of the most important aspects. [4] and [13] propose some good starting points for our security research, both on trust and intrusion detection.

On most of these scenarios, video transmission [11][12] is a must (remote medical support, dynamic maps displaying the location of potential risks, resources, other mobile units, fire advance, wind direction, remote control of robots, etc.). Ad-hoc networks provide new challenges for multilayer-multipath video distribution due to the typically asymmetric transmission conditions of the radio channels and the possibility of exploiting path multiplicity in search of increased performance (video quality and resilience).

Communication support is needed, but not enough to create a useful ad-hoc network. In order to develop an efficient application level, an expressive language and a service discovery protocol are needed. These kinds of solutions are available for fixed networks. This project will provide an agent-based intelligent level to support ad-hoc semantic web. [10] presents a first approach to adapt conventional semantic web languages to ad-hoc networks, that may be useful as starting point.

Analyzing the main goals of the project, agent technology appears to be the best option to support the system infrastructure. While the first idea may be use of a well known agent platform, like JADE [8] (that researchers have used for several projects, fruitfully), that complies with FIPA standards [3] and has a version suitable for small devices, like PDA’s (JADE-LEAP), ad-hoc networks have special aspects that make us believe that FIPA standard architecture, that is implemented by JADE, is not directly useful. There is a proposal (still on an initial stage) to adapt FIPA agents to ad-hoc networks [1]. While it seems to be a valid proposal from a theoretical point of view, there are some practical issues that make difficult its implementation. We propose the design and implementation of a specific architectural extension to FIPA/FADE to enhance the scope of application of this sort of agents to the proposed scenario. Main project researchers have long experience on

agent platform [7] and methodology design [6]; previous research project results will be taken into account.

3. CONCLUSIONS Multi-agent systems may be used to provide an intelligent layer for an ad-hoc network over computer systems. IMPROVISA project will use agent technology on three main aspects: ad-hoc network routing, multilayer-multipath video transmission and semantic ad-hoc networks. The project plans to deliver an integrated demonstration platform to assess the real applicability and added-value of the ad-hoc approach.

4. ACKNOWLEDGMENTS This work is being funded by Spanish Education Ministry under project IMPROVISA (TSI2005-07384-C03)

5. REFERENCES [1] Berger, M. and Watkze, M.: AdHoc Proposal – Reviewed

Draft for FIPA 25. Response to 1st AdHoc Call for Technology, May 2002. <http://www.fipa.org/docs/input/f-in-00064>.

[2] Clausen, T. and Jacquet, P.: Optimized Link State Routing Protocol . IETF RFC 3626, 2003.

[3] FIPA: FIPA Homepage. <http://www.fipa.org>. [4] Hu, Y.-C., Johnson, D. B. and Perrig, A.: SEAD: secure

efficient distance vector routing for mobile wireless ad hoc networks. 4th IEEE Workshop on Mobile Computing Systems and Applications (WM-CSA’02), 2002a, pp 3-13.

[5] IETF, MANET Working Group: Mobile Ad-hoc Networks. <http://www.ietf.org/html.charters/manet-charter.html> .

[6] Iglesias, C. A., Garijo, M., González, J.C., and Velasco, J. R. 1998. Analysis and Design of Multiagent Systems Using MAS-Common KADS. 4th international Workshop on intelligent Agents Iv, Agent theories, Architectures, and Languages. LNCS, vol. 1365, pp 313-327.

[7] Iglesias, C.A., Gonzalez, J.C., and Velasco, J.R. MIX: A General Purpose Multiagent Architecture. Intelligent Agents II. Second International Workshop on Agent Theories, Architectures, and Languages, LNAI 1037. August 1995

[8] JADE: JADE Homepage. <http://jade.cselt.it>. [9] Johnson, D.B., Maltz, D.A. and Hu, Y.-C.: The Dynamic

Source Routing Protocol for Mobile Ad-Hoc Networks (DSR). Internet Draft: draft-ietf-manet-dsr-09.txt, IETF MANET Working Group,15 April 2003.

[10] König-Ries, B. and Klein, M.: First AKT Workshop on Semantic Web Services. 2004,. <http://www.ipd.uka.de/ DIANE/docs/AKT-SWS2004-Position.pdf>.

[11] Liang, Y.J., Steinbach, E.G. and Girod, B.: Real-Time Voice Communication over the Internet using Packet Path Diversity. Proc. ACM Multimedia, Sept 2001, pp. 777-792.

[12] Miu, A. et al.: Low-latency Wireless Video over 802.11 Networks using Path diversity. IEEE ICME, 2003, 441-444.

[13] Ramanujan, R., Ahamad, A., Bonney, J., Hagelstrom, R. and Thurber, K.: Techniques for intrusión-resistant ad hoc routing algorithms (TIARA). IEEE Military Communications Conference (MILCOM´02), 2002, vol 2, pp.890-894

153

Agent-Based Simulation in Disaster Management Márton Iványi

AITIA International Inc 1039 Budapest

Czetz János utca 48-50, Hungary +36 1 453 8080

mivanyi@aitia.ai

László Gulyás AITIA International Inc

1039 Budapest Czetz János utca 48-50, Hungary

+36 1 453 8080 lgulyas@aitia.ai

Richárd Szabó Dept. of Software Technology and

Methodology

1117 Pázmány Péter sétány 1/c, Budapest, Hungary

richard_szabo@invitel.hu

ABSTRACT This paper outlines the possible uses of agent-based and agent-based participatory simulation in various aspects of disaster management and emergency response. While the ideas discussed here build on the capabilities of an existing toolset developed by the authors and on their expertise in agent-based simulation, the paper is mainly a statement of the authors’ position with respect to the applicability of agent-based simulation in the subject field.

Categories and Subject Descriptors I.6, J.4., J.7, K.3, K.4

General Terms

Design, Experimentation, Human Factors

Keywords Agent-Based & Participatory Simulation, Optimization, Training

1. INTRODUCTION A series of recent unfortunate events draw attention to the paramount importance of the ability to organize and manage rescue efforts effectively and efficiently. This paper intends to provide a quick overview of the ways agent-based and agent-based participatory simulation may contribute to achieving this goal.

Agent-based modeling is a new branch of computer simulation, especially suited for the modeling of complex social systems. Its main tenet is to model the individual, together with its imperfections (e.g., limited cognitive or computational abilities), its idiosyncrasies, and personal interactions. Thus, the approach builds the model from ‘ the bottom-up’ , focusing mostly on micro rules and seeking the understanding of the emergence of macro behavior. [1][2][3][5][6] Participatory simulation is a methodology building on the synergy of human actors and artificial agents, excelling in the training and decision-making support domains. [7] In such simulations some agents are controlled by users, while others are directed by programmed rules. The scenarios outlined below are based on the capabilities

of the Multi-Agent Simulation Suite (MASS), developed at the authors’ organization. [8] MASS is a solution candidate for modeling and simulation of complex social systems. It provides the means for rapid development and efficient execution of agent-based computational models. The aim of the Multi-Agent Simulation Suite project is to create a general, web-enabled environment for versatile multi-agent based simulations. The suite consists of reusable core components that can be combined to form the base of both multi-agent and participatory multi-agent simulations. The project also aims at providing a comfortable modeling environment for rapid simulation development. To this end, the suite offers a high-level programming language dedicated to agent-based simulations, and a development environment with a number of interactive functions that help experimentation with and the finalization of the model

2. SIMULATION IN DISASTER MANAGEMENT In our view the methodology of agent-based simulation may help in preparing for the task of disaster management or emergency response. There are two major areas of this applicability: experimentation (optimization) and training.

Figure 1 Fire evacuation simulation in MASS

2.1 Experimentation and Optimization Simulations may be used to experiment with and to optimize the installation of structures and allocation of various resources. For example, the consequences of design- or installation-time decisions or evacuation rules and procedures can be studied and evaluated in cases of hypothetical floods, school fires, stadium stampedes. [4] This way, agent-based simulation can help decision makers in reaching more safe and more solid decisions. Obviously, we can never prepare for the unpredictable, and human behavior, especially under the stress of an emergency situation is typically hard to predict. Yet, the benefit of agent-based simulation is that it can provide dependable patterns of collective behavior, even if the actions of the individual are hard or impossible to predict exactly. In our view, this is a major

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. AAMAS’06, May 8-12, 2006, Hakodate, Hokkaido, Japan. Copyright 2006 ACM 1-59593-303-4/06/0005...$5.00.

154

contribution of the agent approach. Figure 1 shows a screenshot from a fire evacuation simulation in MASS. The little dots are simulated individuals (agents) trying to escape from an office building under fire. (The bird eye view map depicts the layout of the authors’ workplace.) The dark lines represent office walls, which slow down, but cannot prevent the spread of fire. The main exit is located near the bottom-right corner of the main hallway.

2.2 Training Applications In disaster management both in internal and public education/training applications could be of great use. Agent-based simulation may also help in developing highly usable, cost-effective training applications. Such trainings can take the form of computer ‘games’ that simulate real life situations with both real and artificial players. Here the trainee can take control of one or other simulated individual. This area is especially suited for participatory simulation and the application may also be derived from experimentation-type simulations of the previous section. For example, Figure 2 shows a screen capture from the participatory version of the simulation on Figure 1.

Figure 2 A participatory simulation derived from the simulation on Figure 1.

There are several uses participatory training applications can be put to. They can train professional emergency support personnel in the emergency response team (where trainees may take the role of i.e., a top, middle or ground level decision makers), the voluntary fireman squad or workers of a building with a high risk. Such solutions are a cost-effective and easy means to deepen the trainees’ knowledge of rare situations and help developing the proper natural responses in them. Figure 3 shows a screenshot from a demonstrational educational emergency management application. The upper-left panel of the screen shows the map of Hungary with the main transport routes and icons for events requiring emergency response. The bottom-left panel lists related (simulated) news items, while the upper-right panel summarizes the status of various emergency response vehicles (firefighter and rescue helicopters, ambulances, etc.) The bottom-right panel is for detailed information on the selected items (pictured as downloading information on the selected item from the upper-right panel).

In addition to the previously described internal training use participatory training simulations may also be helpful in external training. External trainings are a sort of public relations operations, helping to explain the difficulty and communicate the difficulties and challenges of emergency response or disaster management to the greater, general public. This aspect is gaining extreme importance due to inherent conflicts between the drastically increased government attention to disaster management and the need for public control over government spending,

especially as the actual, deployed emergency management applications are very sensitive with respect to security.

Figure 3 Educational Emergency Management Software (in

Hungarian). The map involved is that of Hungary.

3. CONCLUSIONS This paper discussed the applicability of agent-based simulation and that of its extension participatory agent-based simulation to disaster management. Our position is these methods could be very helpful in preparing for the task of disaster management or emergency response. In particular, we identified two application areas: experimentation (optimization) and training. We also mentioned a special possible use of the latter in public relations, i.e., in explaining and communicating to the greater public the immense difficulty and importance of disaster management efforts. The ideas discussed in this paper build on the capabilities of MASS, an existing toolset developed by the authors, as demonstrated by screen captures from existing simulation applications.

4. ACKNOWLEDGMENTS The partial support of the GVOP-3.2.2-2004.07-005/3.0 (ELTE Informatics Cooperative Research and Education Center) grant of the Hungarian Government is gratefully acknowledged.

5. REFERENCES [1] Bankes, S. C.: "Agent-based modeling: A revolution?" Proceedings

of the National Academy of Sciences of the USA, Vol. 99:3, pp. 7199–7200, 2002.

[2] Brassel, K.-H., Möhring, M., Schumacher, E. and Troitzsch, K. G.: "Can Agents Cover All the World?" Simulating Social Phenomena (Eds. Conte, R., et al.), Springer-Verlag, pp. 55-72. 1997.

[3] Conte, R.: "Agent-based modeling for understanding social intelligence", Proceedings of the National Academy of Sciences of the USA, Vol. 99:3, pp. 7189-7190, 2002.

[4] Farkas, I., Helbing, D. and Vicsek, T.: "Human waves in stadiums", Physica A, Vol. 330, pp. 18-24, 2003.

[5] Gilbert, N. and Terna, P.: "How to build and use agent-based models in social science", Mind & Society, Vol. 1, pp. 55-72, 2000.

[6] Gilbert, N. and Troitzsch, K. G.: Simulation for the social scientist, Open University Press, Buckingham, UK, p. 273. 1999.

[7] Gulyás, L., Adamcsek, B. and Kiss, Á.: "An Early Agent-Based Stock Market: Replication and Participation", Rendiconti Per Gli Studi Economici Quantitativi, Volume unico, pp. 47-71, 2004.

[8] Gulyás, L., Bartha, S.: “FABLES: A Functional Agent-Based Language for Simulations” , In Proceedings of The Agent 2005 Conference on: Generative Social Processes, Models, and Mechanisms, Argonne National Laboratory, Chicago, IL, USA, October 2005.

155

Agent Based Simulation combined with Real-Time Remote Surveillance for Disaster Response Management

Dean Yergens, Tom Noseworthy Centre for Health and Policy Studies

University of Calgary Calgary, Alberta, Canada

{dyergens,tnosewor}@ucalgary.ca

Douglas Hamilton Wyle Life Sciences

NASA Johnson Space Center Houston, Texas, USA

dhamilton@wylehou.com

Jörg Denzinger Department of Computer Science

University of Calgary Calgary, Alberta, Canada

denzinge@cpsc.ucalgary.ca

ABSTRACT In this position paper, we describe the convergence of two disaster management systems. The first system, known as GuSERS (Global Surveillance and Emergency Response System), is a communication-based system that uses low-bandwidth satellite two-way pagers combined with a web-based geographical information system. GuSERS facilitates surveillance of remote areas, where a telecommunications and electrical infrastructure may not exist or may be unreliable. The second system is an agent-based simulation package known as IDESS (Infectious Disease Epidemic Simulation System), which develops infectious disease models from existing data. These two systems operating in tandem have the potential to gather real-time information from geographically isolated areas, and use that information to simulate response strategies. This system could direct appropriate and timely action during an infectious disease outbreak and/or a humanitarian crisis in a developing country.

Keywords Multi Agent System, Simulation, Disaster Management, Health Informatics, Epidemic, Geographical Information System, Humanitarian Response, GuSERS, IDESS, Public Health Surveillance.

1. INTRODUCTION Developing countries are known to have poor communication networks. This often involves unreliable telecommunication and electrical infrastructure, as well as poor transportation networks. Real-time communication is further complicated by the geographical nature of many regions in developing countries such as mountainous terrain and jungle-like environments which severely impact timely communication.

Environmental factors, such as seasonal weather patterns, also affect communication networks, as in the case of heavy rains that routinely wash out roads, bridges and telecommunication infrastructures, isolating many communities for weeks. The growth and adoption of cellular based networks in developing countries is encouraging; however, these cellular networks often only cover densely populated urban areas, leaving rural communities without this valuable service. Communication networks have traditionally had a major impact on the surveillance and reporting of infectious diseases, and other forms of humanitarian crisis, such as refugee movement caused by political uncertainly and environmental disasters. Developing countries are not the only regions affected by environmental

factors. Ground-based communication networks can be defeated in any areas of the world, as hurricane Katrina recently demonstrated.

Communication of events is only one element in disaster management. Another “core” element is the ability to appropriately respond to such events. This involves the ability to forecast (simulate) how an epidemic or humanitarian crisis is affecting a geographical region.

Two systems have been developed by us since January 2000 that address these issues. The first is a system known as GuSERS [1]. GuSERS takes the approach of utilizing low-bandwidth satellite two-way pagers to send information to a web-based geographical information system. The second system is an agent-based simulation package known as IDESS [2]. IDESS rapidly generates simulations of infectious disease outbreaks through the use of existing widely available data in order to develop timely response strategies.

An IDESS simulation model could be improved in terms of prediction accuracy by adapting it to use real-time multi-agent information. By converting GuSERS, respectively its nodes, into agents, and having them and their information act as a primary data source for IDESS, models could be generated that focus on how a disease is actually spreading or how a disaster is affecting a certain area. Best methods of intervention can then be deployed.

These two systems are described in more detail below.

2. GLOBAL SURVEILLANCE AND EMERGENCY RESPONSE SYSTEM (GuSERS) 2.1 Introduction The bi-directional satellite messaging service used by GuSERS is a cost-effective, reliable means of communicating with remote healthcare clinics and facilities. It requires no ground-base infrastructure, so it is not affected by environmental conditions or political instabilities. Moreover, it does not incur large investment of capital and operating costs seen with telemedicine, which requires high-bandwidth technology.

2.2 Methods Low-bandwidth satellite paging services, Global Positioning Systems (GPS), portable solar power systems, and Internet based Geographical Information Systems (GIS) were all integrated into a prototype system called the Global Surveillance and Emergency Response System (GuSERS). GuSERS has been tested and

156

validated in remote areas of the world by simulating disease outbreaks. The simulated disease incidence and geographic distribution information was reported to disease control centers in several locations worldwide.

2.3 Results Initial testing of the GuSERS system demonstrated effective bidirectional communication between the GuSERS remote solar powered station and the disease control centers. The ability to access the GuSERS information through any standard web-browser was also validated. A practical demonstration of using bi-directional satellite messaging service during an emergency situation took place during Hurricane Rita. Co-authors (DY and DH) maintained communication even though cellular networks were overwhelmed with telecommunication traffic. Clearly there is an advantage to using telecommunication infrastructure that is low-bandwidth and not ground-based.

3. INFECTIOUS DISEASE EPIDEMIC SIMULATION SYSTEM (IDESS) 3.1 Introduction IDESS (Infectious Disease Epidemic Simulation System) is a system that combines discrete event simulation, Geographical Information Systems (GIS) and Automated Programming in order to develop simulation models of infectious disease outbreaks in order to study how an event may spread from one physical location to another in a regional/national environment.

We present a scenario that investigates the effect of an infectious disease outbreak occurring in Sub Saharan and its spread of infection from town to town. IDESS was created to be used by epidemiologists and disaster management professionals to quickly develop a simulation model out of existing GIS data that represents a geographical layout of how towns and cities are connected. The result can be used as a framework for simulating infectious disease outbreaks, which can then be used to understand its possible effects and determine operational and/or logistical ways to respond including containment strategies. This approach is in contrast with other infectious disease models that are more sophisticated, however, may take longer to develop and not be readily deployed in a manner of hours for any geographical region [3][4].

3.2 Methodology The IDESS system parses existing GIS data, collecting information about the various towns and cities in a specific region based upon the input GIS dataset. This information is placed into a relational database that stores the geographical location (latitude and longitude) and the name of the town or city. Extracting road information from GIS data presented a unique challenge, as typically the town GIS data is represented by points, while the road is represented by a vector, which does not actually connect towns to a specific road. We addressed this issue by automatically building town/city networks through physical proximity of a road and any towns. This proximity was defined by the user by stating how many kilometers a road could be from a town to actually assume the connection. Additional parameters

such as infection rate, mortality rate and HIV prevalence were also included in the model.

3.3 Conclusion IDESS shows promise in the ability to quickly develop agent based simulation models that can be used to predict the spread of contagious diseases or the movement of a population for any geographical environment where GIS data exists. Future research involves integrating the IDESS system with a weather module.

4. CONNECTING GuSERS AND IDESS Combining GuSERS and IDESS is an obvious next step. The IDESS system will be integrating nodes and information from the GuSERS system to provide agent-based simulations on an on-going basis. Data in the GuSERS system will supplement existing GIS data in building the IDESS simulation model. IDESS simulations can then also be used for training purposes by feeding back into GuSERS simulated events and testing the responses by the GuSERS nodes.

5. CONCLUSION Multi Agent System based simulation provides a very efficient environment for disaster management, due to the ability to handle and manage multiple forms of data that may be arising from the field. The formation of a real-time agent based management system combined with the ability to provide simulation capabilities allows the evaluation of various response strategies and thus provides the response managers with decision support. Using a multi-agent system approach in developing such systems provides the usual software engineering advantages, but also mirrors the distributed nature of emergency situations.

6. ACKNOWLEDGMENTS Our thanks to the Centre for Health and Policy Studies University of Calgary, the Alberta Research Council for Stage 1 funding of the Global Surveillance and Emergency Response System (GuSERS) and the following significant contributors John Ray, Julie Hiner, Deirdre Hennessy and Christopher Doig.

7. REFERENCES [1] Yergens, D.W., et al. Application of Low-Bandwidth

Satellite Technology for Public Health Surveillance in Developing Countries. International Conference on Emerging Infectious Diseases, (Feb. 2004), 795.

[2] Yergens, D.W., et al. Epidemic and Humanitarian Crisis Simulation System. 12th Canadian Conference on International Health (Nov. 2005) Poster.

[3] Barrett, C, Eubank, S, Smith, J. If Smallpox Strikes Portland. Scientific American, (March 2005), 54-61.

[4] Ferguson, N, et al. Strategies for containing an emerging influenza pandemic in Southeast Asia. Nature, 437, 8 (September 2005), 209-214.

157

The ALADDIN Project: Agent Technology To The Rescue∗

Nicholas R. Jennings, Sarvapali D. Ramchurn,

Mair Allen-Williams, Rajdeep Dash, Partha Dutta, Alex Rogers, Ioannis VetsikasSchool of Electronics and Computer Science,

University of Southampton, Southampton, SO17 1BJ, UK.{nrj,sdr,mhaw05r,rkd,psa,acr,iv}@ecs.soton.ac.uk

ABSTRACTALADDIN1 is a five year project that has just started andwhich aims to develop novel techniques, architectures, andmechanisms for multi-agent systems in uncertain and dy-namic environments. The chosen application is that of dis-aster management. The project is divided into a number ofthemes that consider different aspects of the interaction be-tween autonomous agents and study architectures to buildplatforms to support such interactions. In so doing, this re-search aims to contribute to building more robust and re-silient multi-agent systems for future applications in disastermanagement and other similar domains.

1. INTRODUCTIONThis paper outlines the research we will be performing in theALADDIN project. This project aims to develop techniques,methods and architectures for modelling, designing and build-ing decentralised systems that can bring together informationfrom a variety of heterogeneous sources in order to take in-formed actions. To do this, the project needs to take a to-tal system view on information and knowledge fusion and toconsider the feedback between sensing, decision-making andacting in such systems (as argued in section 1). Moreover,it must be able to achieve these objectives in environmentsin which: control is distributed; uncertainty, ambiguity, im-precision and bias are endemic; multiple stakeholders withdifferent aims and objectives are present; and resources arelimited and continually vary during the system’s operation.

To achieve these ambitious aims, we view such systems asbeing composed of autonomous, reactive and proactive agents[3] that can sense, act and interact in order to achieve indi-vidual and collective aims (see section 2). To be effective insuch challenging environments, the agents need to be ableto make the best use of available information, be flexibleand agile in their decision making, cognisant of the fact thatthere are other agents, and adaptive to their changing envi-

∗ALADDIN stands for “Autonomous Learning Agents forDistributed and Decentralised Information Networks”.1http://www.aladdinproject.org

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.Copyright 200X ACM X-XXXXX-XX-X/XX/XX ...$5.00.

ronment. Thus we need to bring together work in a number oftraditionally distinct fields such as information fusion, infer-ence, decision-making and machine learning. Moreover, suchagents will invariably need to interact to manage their inter-dependencies. Such interactions will also need to be highlyflexible because of the many environmental uncertainties andchanges. Again this requires the synergistic combination ofdistinct fields including multi-agent systems, game theory,mechanism design and mathematical modelling of collectivebehaviour.

Finally, to provide a focus for this integrated view, the ideasand technologies developed within the research programmewill be exercised within the exemplar domain of disaster re-covery. This domain has been chosen since it requires timelydecision making and actions in the highly uncertain and dy-namic situations highlighted earlier, because it is an impor-tant domain in itself, because it is demanding both from afunctional and an integrated system point of view. The de-velopment of decentralised data and information systems thatcan operate effectively in highly uncertain and dynamic envi-ronments is a major research challenge for computer scientistsand a key requirement for many industrial and commercial or-ganisations. Moreover, as ever more information sources be-come available (through the Web, intranets, and the like) thenetwork enabled capability of obtaining and fusing the rightinformation when making decisions and taking actions is be-coming increasingly pressing. This problem is exacerbated bythe fact that these systems are inherently open [1] and needto respond in an agile fashion to unpredictable events. Open-ness, in this context, primarily means that the various agentsare owned by a variety of different stakeholders (with theirown aims and objectives) and that the set of agents presentin the system at any one time varies unpredictably. This, inturn, necessitates a decentralised approach and means thatthe uncertainty, ambiguity, imprecision and biases that areinherent in the problem are further accentuated. Agility isimportant because it is often impossible to determine a prioriexactly what events need to be dealt with, what resources areavailable, and what actions can be taken.

2. RESEARCH THEMESThe project is divided into four main research themes deal-ing with individual agents, multiple agent scenarios, decen-tralised architectures, and applications. We describe each ofthese in more detail in the following sections.

2.1 Individual AgentsThis research theme is concerned with techniques and meth-ods for designing and developing the individual agents that

158

form the basic building blocks of the distributed data andinformation system. This is a significant challenge for twomain reasons. First, we need to take a holistic view of theindividual agent. Thus, each individual must:

• fuse information obtained from its environment in orderto form a coherent view of its world that is consistentwith other agents;

• make inferences over this world view to predict futureevents;

• plan and act on its conclusions in order to achieve itsobjectives given these predictions.

These activities need to be performed on a continuous basisbecause of the many feedback loops that exist between them.Second, each actor must operate in this closed loop fashion inan environment in which: there are significant degrees of un-certainty, resource variability, and dynamism and there aremultiple other actors operating under a decentralised con-trol regime. To be effective in such contexts, a number ofimportant research challenges need to be addressed. Specif-ically, uncertainty, including temporal variability and non-stationarity, will be apparent at a number of different levelswithin the system. Thus apart from the inherent variabilityin the real world systems physical communication topology,components may fail or be damaged and our model of uncer-tainty will have to cope with this.

2.2 Multiple AgentsThis research theme is primarily concerned with the way inwhich the various autonomous agents within the system in-teract with one another in order to achieve their individualand collective aims. It covers three main types of activity:

• how the interactions of the autonomous agents can bestructured such that the overall system exhibits certainsorts of desirable properties;

• the sorts of methods that such agents can use to coor-dinate their problem solving when the system is opera-tional;

• how the interactions of such agents can be modelled andsimulated in order to determine the macroscopic behav-iour of the overall system based on the microscopic be-haviour of the participants.

To tackle the above activities, a number of techniques will beused to analyse and shape the interactions between multipleagents in order to achieve the overall system-wide proper-ties. To this end, while mathematical models of collectivebehaviour will generally be developed, in cases where agentsare only motivated to achieve their own selfish goals, gametheory and mechanism design will be used [2].

2.3 Decentralised System ArchitecturesThis research theme is concerned with the study and devel-opment of decentralised system architectures that can sup-port the individual and multiple agents in their sensing, de-cision making and acting in the challenging environments wehave previously characterised. The defining characteristic ofsuch systems is that they do not rely on a centralised co-ordinator or controller. This is motivated both by the in-herent structure of the domain/application and a numberof perceived system or operational benefits (including fault-tolerance, modularity, scalability, and system flexibility - as

detailed in section 1). Now in contrast to their centralisedcounterparts, decentralised data and information systems of-fer many advantages. In a centralised system, data is com-municated to a designated agent where it is fused and, sub-sequently, decisions are made. The results of the fusion ordecision process are then communicated back to the otheragents in the system. However this leaves the system open tomany vulnerabilities since the central agent is a single pointof failure. Further, such systems place large demands on com-munications and this limits the size of the system that canbe developed. Given this context, the key research activitiesinvolved in this area are:

• to determine the range of issues and variables that willgovern the possible architectures and determine howthese options can be compared and contrasted;

• to evaluate these options to determine their relativemerits in varying circumstances.

2.4 Applications: Disaster ManagementTo ensure the specific methods and techniques developed inthe research fit together to give a coherent whole, the projectwill develop a number of software demonstrations. These willbe in the broad area of disaster management (for the afore-mentioned reasons). In order to develop demonstrators thatcan be used either for testing the resilience of mechanisms de-veloped in the other themes or simply to demonstrate theireffectiveness, the main activities of this theme will focus on:

• devising a model of disaster scenarios which clearly cap-tures most, if not all, of the important variables thatneed to be monitored.

• benchmarking the technologies developed in the projectagainst other existing mechanisms used by emergencyresponse services in real-life disaster scenarios.

3. CONCLUSIONTo enable emergency responders to make informed choices isan important challenge for computer science researchers. Itrequires command and control mechanisms that can be flex-ible in the presence of uncertainty and can respond quicklyas new information becomes available. This applies at thestrategic, operational and tactical levels of the problem. Toachieve this, the ALADDIN project will undertake research inthe areas of multi-agent systems, learning and decision mak-ing under uncertainty, will develop architectures that bringtogether such functionality, and will then apply them to thearea of disaster management.

AcknowledgementsThe ALADDIN project is funded by a BAE Systems/EPSRCstrategic partnership. Participating academic institutions inthe ALADDIN project include Imperial College London, Uni-versity of Bristol, Oxford University.

4. REFERENCES[1] Open information systems semantics for distributed artificial

intelligence. Artificial Intelligence.

[2] R. K. Dash, D. C. Parkes, and N. R. Jennings. Computationalmechanism design: A call to arms. IEEE Intelligent Systems,18(6):40–47, 2003.

[3] N. R. Jennings. An agent-based approach for building complexsoftware systems. Communications. of the ACM, 44(4):35–41,2001.