Swick Assistance: Dedicated Virtual Assistants for Professionals, Entrepreneurs & Small Teams
Integrating Intelligent Assistants into Human Teams
-
Upload
david-copeland -
Category
Documents
-
view
31 -
download
1
description
Transcript of Integrating Intelligent Assistants into Human Teams
![Page 1: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/1.jpg)
Integrating Intelligent Assistants into Human Teams
Katia Sycara
The Robotics Institute
Carnegie Mellon University
Pittsburgh, PA 15213
(412) 268-8225
www.cs.cmu.edu/~softagents
Michael Lewis
School of Information Sciences
University of Pittsburgh
Pittsburgh, PA 15260
(412) 624-9426
www.pitt.edu/~cmlewis
![Page 2: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/2.jpg)
Team Members CMU
Liren Chen
Somesh Jha
Rande Shern
Dajun Zeng
Keith Decker
Anadeep Pannu
Vandana Verma
Prasad Chalasani
Kostya Domashnev
Onn Shehory
![Page 3: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/3.jpg)
Team Members U. of Pittsburgh
Michael Lewis (PI)
Terry Lenox
Emily Roth
![Page 4: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/4.jpg)
Talk Outline
• Goals
• Potential Impact for the Navy
• Approach
• Research Issues
• Progress
• Plan for Next Year
![Page 5: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/5.jpg)
Overall Research Goal Increase the effectiveness of joint Command and Control Teams
through the incorporation of Agent Technology in environments that are:
• distributed
• time stressed
• uncertain
• open (information sources, communication links and agents dynamically appear and disappear)
Team members are distributed in terms of:
• time and space
• expertise
![Page 6: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/6.jpg)
Impacts for Navy
• Reduce time for a C2 team to arrive at a decision
• Allow C2 teams to consider a broader range of alternatives
• Enable C2 teams to flexibly manage contingencies (replan, repair)
• Reduce time for a C2 team to form a shared model of the situation
• Reduce individual and team errors
• Support team cohesion and team work skills
• Increase overall team performance
![Page 7: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/7.jpg)
Transition Opportunities
• Maritime Crisis planning
• Target identification training
• Air campaign planning
• Strike planning
• Aircraft maintenance
![Page 8: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/8.jpg)
Overall Approach
• develop an adaptive, self-organizing collection of Intelligent Agents (the RETSINA infrastructure) that interact with the humans and each other.
– integrate multimedia information management and decision support
– anticipate and satisfy human information processing and problem solving needs
– perform real-time synchronization of human actions
– notify about significant changes in the environment
– adapt to user, task and situation
• develop model libraries of individual and team tasks
• develop verifiable useful human-agent interaction techniques
![Page 9: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/9.jpg)
Overall Research Issues
• Agents and Agent Interactions
• Human Agent Interaction
• Information Filtering and Integration
![Page 10: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/10.jpg)
Overall Research Issues: Agents and Agent Interactions
• interleaving planning, replanning, execution monitoring and information gathering in a multiagent setting
• single agent architecture and self-awareness
• agent coordination scheme
• finding appropriate agents
• agent interoperability
• agent-to-agent task delegation protocols
• learning through agent interactions
![Page 11: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/11.jpg)
Overall Research Issues: Human Agent Interaction
• agent-based team aiding
• functional allocation between humans and agents (insert agents into military simulations and perform controlled experiments with human subjects to assess utility)
• human-agent trust
• development of task models (graphical task editor)
• user-guided instantiation of agents (agent editor)
![Page 12: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/12.jpg)
Insert TeamAiding.ppt
![Page 13: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/13.jpg)
Overall Research Issues: Information Filtering and Integration
• learning and tracking multiple interests of users
• increase relevance of retrieved information (refinement key words, relevance feedback, summary of most important information in documents)
• detecting ``interesting'' patterns from multiple data sources
• information integration and conflict resolution
![Page 14: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/14.jpg)
Retsina Functional Organization
![Page 15: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/15.jpg)
Characteristics of RETSINA Agents
• Agents act autonomously to accomplish objectives
– Goal-directed
– Taskable
– Running unassisted for long periods
– Proactive & Reactive
![Page 16: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/16.jpg)
Characteristics of RETSINA Agents (Contd.)
• Agents engage in peer-to-peer interactions
– Agents are taskable, i.e. users or other agents can delegate tasks to them, user acceptability and trust an important issue
– Can interact as cooperative teams or self-interested individuals
– Interaction protocols
– Coordination Strategies
– Negotiation Protocols
• Agents adapt to their environment, user, task and each other
– Adapt both at the individual level and at the societal level
– Employ Alternate Methods
– Learn from (and about) users and each other
![Page 17: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/17.jpg)
Progress
• RETSINA system infrastructure development
– Java implementation
• RETSINA agent architecture
– increased planning sophistication in individual agents
• Middle agents
• Agent interaction protocols
![Page 18: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/18.jpg)
Middle Agent Types
PreferencesInitially Known By
Provider Only Provider +Middle Agent
Provider + Middle +Requester
Requester Only (Broadcaster) “Front-Agent” Matchmaker
Requester +Middle Agent
Anonymizer Broker Recommender
Requestor +Middle + Provider
Blackboard Introducer/Bodyguard
Arbitrator
Service Parameters Initially Known By
Service providers have capabilities and service parametersService requesters have service request and preferences
![Page 19: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/19.jpg)
Retsina Agent Architecture
![Page 20: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/20.jpg)
RETSINA Planning Mechanisms
• hierarchical task network-based formalism
• library of task reduction schemas
– alternative task reductions
– contingent plans, loops
• incremental task reduction, interleaved with execution
– information gathered during execution directs future planning
• resource and temporal constraints
![Page 21: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/21.jpg)
A task Structure (Advertisement Task Structure)
![Page 22: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/22.jpg)
Progress (Contd.)
• Agent interoperability
– language for capability advertisement (Aardvark)
– agent name server and distributed matchmakingª
• Human Agent Interaction
– Task Editor
– Agent Editor
– Human Agent Trust
– Team TANDEM experiments
________________________
ª www.cs.cmu.edu/~softagents/retsina/ans
![Page 23: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/23.jpg)
Insert Aardvark.ppt:language for capability advertisement
![Page 24: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/24.jpg)
Insert Interact.ppt: Agent Editor
![Page 25: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/25.jpg)
Progress (Contd.)
• Applications
– Information filtering: Webmateª, DVINA
– Agents in team aiding: ModSAF, multiagent air patrol, agent-aided aircraft maintenance
___________________________
ª www.cs.cmu.edu/~softagents/webmate
This application is done in collaboration with the CMU wearable computer project.
![Page 26: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/26.jpg)
ModSAF Vision
![Page 27: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/27.jpg)
![Page 28: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/28.jpg)
![Page 29: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/29.jpg)
Insert AirMain.ppt:Aircraft Maintenance Task
![Page 30: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/30.jpg)
Overview of the WebMate System
• Use the multiple TF-IDF vectors to keep track of user interests in different domains which are automatically learned
• Use the trigger pair model to automatically extract relevant words for refining search
• The user can provide multiple pages as relevance guidance for information search
![Page 31: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/31.jpg)
Insert WebMate.ppt(more detailed description)
![Page 32: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/32.jpg)
Insert WebMateDemo.ppt(detailed description of WebMate demo)
![Page 33: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/33.jpg)
Overview of Informedia
• One of the six Digital Libraries Initiative projects funded by the NSF, DARPA, NASA and others in collaboration with WQED
• A multimedia library that will consist of over one thousand hours of digital video, audio, images, text and other related materials
• Uses combined speech, language and image understanding technology to transcribe, segment and index the linear video.
![Page 34: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/34.jpg)
Plans for Next Year
• Continue enhancing the functionality of individual agents (e.g., more sophisticated planning mechanisms)
• Improve the robustness of the RETSINA infrastructure
• Finish the implementation of the agent advertisement language (Aardvark)
• Refine agent task delegation framework, particularly contingent task delegation
• Investigate situation-dependent agent coordination strategies
• Investigate information- and action-based conflict resolution
• Expand the ModSAF team-aiding scenarios by introducing agents of additional types and functionalities
![Page 35: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/35.jpg)
Plans for Next Year (Contd.)
• Develop explicit agent tasking mechanisms
• Identify appropriate indexing mechanisms for task structure cases
• Expand the functionalities of agent editor
• Automatically learn individual and team coordination patterns from team activity traces
![Page 36: Integrating Intelligent Assistants into Human Teams](https://reader035.fdocuments.net/reader035/viewer/2022062517/568134b1550346895d9bcce9/html5/thumbnails/36.jpg)
Plan for Integrating the Parts of CMU MURI
• Work with U. of Pittsburgh to identify additional agent requirements needed for agent-based team aiding
• U. of Pittsburgh will test the effectiveness of agent-based team aiding in ModSAF scenarios with human subjects
• Incorporate multimedia information from Informedia into agent-based team aiding
• Use the wearable computers as the platform for running the collaborative aircraft maintenance agents