Cognitive Task Analysis of Teams (Team CTA)
-
Upload
yogesh-tadwalkar -
Category
Technology
-
view
10.060 -
download
1
description
Transcript of Cognitive Task Analysis of Teams (Team CTA)
Team CTATeam CTA
Team CTATeam CTA
Cognitive Task Analysis
of Teams-Yogesh Tadwalkar
March 22, 2004
Team CTATeam CTA
Traditional Task Analysis
Task Analysis aims to optimize work performance by matching tasks with human capabilities and limitations
Human work is comprised of both physical and cognitive activities
Physical tasks are observable (Overt) However, cognitive processes that lead to
physical actions are not (Covert)
Team CTATeam CTA
Cognitive Task Analysis (CTA) Extension of traditional task analysis
techniques to uncover information about:– - Work domain, mental
demands on operator
– - Goal Structures – - Knowledge and Strategies– - Thought Processes and
Decision Making
Thus, providing a holistic, complete view of human work
Team CTATeam CTA
Goals of CTA
Performance improvement through: – Better Task and Interface Design– Ideal Personnel Selection– Comprehensive Training and
Instructions – Systematic Performance Assessment– Improved Planning
Team CTATeam CTA
Cognitive Task Analysis
of Teams
Team CTATeam CTA
Why Conduct Team CTA? Because 2 + 2 ≠ 4
– Individuals are often required to perform in teams
– Optimum individual performances may not lead to final mission accomplishment if team co-ordination is unsatisfactory
Thus, individual CTA would not be sufficient for designing optimum task performance
Team CTATeam CTA
Team CTA
As defined by Klein:– A reflection and representation of the
team mind / Knowledge – Description of the cognitive skills
required for effective team performance– Elaboration of the team decision
requirements– Determination of types of expertise
found in effective teams
Team CTATeam CTA
Objectives of Team CTA
Team performance enhancement through: – Restructuring and Resizing the team – Designing better:
• Information technology• Information management strategies• Human-computer interfaces• Decision support systems and Communication
– Developing methods for team training and selection
Team CTATeam CTA
Team Environment
Team CTATeam CTA
Team Environment Tasks are decomposed and
assigned to different team members
Team members may or may not have direct contact with each other (co-located vs. dispersed teams)
Tasks can be performed serially or
in parallel or overlapping fashion
Team members possess, acquire and process information, make decisions, solve problems and make plans
Team CTATeam CTA
Team Environment
Teams are intelligent entities (Thordsen and Klein, 1989)– Team members possess, acquire and
process information – Communicate with each other – Make decisions – Solve problems and – Make plans
Team CTATeam CTA
Types of Teams
Planning Teams– Military Command– Emergency Response at nuclear plants
Action Teams– Air Traffic Control– SWAT
Team CTATeam CTA
Cognitive Processes of TeamsWhat to measure through Team CTA methods
Team CTATeam CTA
Cognitive Processes of Teams Methods for team CTA should focus at capturing
these 5 processes (Klein):
1. Shared mental models- Team Goals, team member roles
2. Control of attention- Information seeking, filtering, communication
3. Shared situation awareness- Assessment of dynamism, prediction of future
4. Application of strategies - To make decisions, solve problems and plan
5. Metacognition- Self-monitoring, sensing trouble, making alternative plans
Team CTATeam CTA
A Model for Team Cognition
Knowledge
Mental Models
Attitudes
Expectations
Skills
Member A
Member CMember B
Team Performance
Pre-task Knowledge• Shared Mental Models• Strategies
Dynamic Knowledge• Control of Attention• Shared Situation Awareness• Metacognition
(Stout, Cannon-Bowers, and Salas, 1996)
Team CTATeam CTA
1. Shared Mental Models The degree to which team members have
the same understanding of key processes (Cannon-Bowers, Sales, & Converse, 1992) - Knowledge about overall team
goals and mission objectives e.g. Football game
- Roles and responsibility of each team member
- Team-mate characteristics and preferences
- Task Procedures, sequences and timing - Use and relationship of equipment
Team CTATeam CTA
Shared Mental Models and Team Performance Shared mental models affect team
performance significantly (Oransu, 1990): – Anticipating team member behaviours and
information requirements– Performing tasks from a common frame of
reference– Attending to, interpreting, communicating
about, and responding to the world more similarly than individuals with discrepant knowledge (Rentsch & Hall, 1994)
Team CTATeam CTA
Methods to Elicit and RepresentShared Mental Models
Team CTATeam CTAVariation of Goal-Directed Analysis (Woods & Hollnagel, 1987)
To solicit knowledge of Team Goals and Task Objectives
– Document Analysis and Observation– Separate interviews with each team
member, followed by group interview – Representation: Agreement Metric
Task Goal
Sub-Goal 1
Sub-Goal 2
Sub-Goal 3
Task 1 Agreement Metric
Team Goals
For Each Member For Whole Team
Task 2
Team CTATeam CTAVariation of Task-Action Mapping (Coury et al., 1991)
To solicit knowledge of Task Procedures, Sequence and Timing
– Document Analysis and Observation– Ask each team member to describe key
team concepts, associated tasks and specific actions
– Representation: Concept Maps
Description
Task 1
Task 2
Task 3
Action 1• What (Steps)• Who• WhenConcept 1
e.g. EvacuationAction 1• What (Steps)• Who• When
Team CTATeam CTA
Bootstrapping (Hoffman, Shadbolt, Burton & Klein 1995)
To solicit knowledge of Roles and Responsibilities
Bootstrapping
Documentation
Individual Interviews
ObservationGroup
Interviews
May not adequately capture dynamic, contextual data like misinterpretations, omissions due to environmental factors
Team CTATeam CTAAdapted Critical Incident Method(Flanagan, 1954; Critical Decision Method by Klein, 1989)
To solicit knowledge of Roles and Responsibilities, Strategies, Attitudes and Personal Characteristics– Document Analysis and Recall
of a critical past event e.g. fire, accident, high-stake mission.
– Ask each team member to describe various decisions taken, action triggers, who did what vs. who was supposed to do what
– Describe how each teammate reacted to the situation (perception, prediction, response time, stress, flexibility in co-ordination, etc.)
Team CTATeam CTA
2. Control of Attention The way a team engages in information
management - Information seeking - Filtering irrelevant information - Allocating attention to
important functions - Properly distributing messages
Refers to the team’s working memory– The way it uses limited resources for processing
simultaneous messages (Klein)
Team CTATeam CTA
Methods to Elicit and RepresentControl of Attention
Team CTATeam CTA
Scenario based Questionnaires (Blickensderfer, 1997)
To solicit knowledge of Information Management Strategies, and Communication Skills
– Document Analysis, observations and interviews to create a questionnaire covering a team work scenario, given to each team member
– Answers to outline relevant information protocol for the scenario, missing information, cues for identifying missing data, strategies to acquire and distribute information, and possible communication bottlenecks
Team CTATeam CTA
3. Shared Situation Awareness Extent to which team members have the
same interpretation of ongoing events (Cannon-Bowers, Sales, & Converse, 1992)
Is giving full data to each member a remedy for discrepant situation Awareness?
–How different team members perceive dynamic situations and events–How they form divergent impressions e.g. police in a riot situation–How they update each other
Team CTATeam CTA
Methods to Elicit and RepresentShared Situation Awareness
Team CTATeam CTA
Adapted Situation Awareness Global Assessment Technique (SAGAT)(Endsley, 1995)
To solicit knowledge of compatible situation awareness– Bootstrapping to create a mission simulation
– Asking team members to perform mission
– Freezing mission after specific intervals
– Asking each team member to analyze the situation and predict outcome and strategize
– Compare results for all members and assess compatibility and reasons for discrepancy
Team CTATeam CTA
Situation Awareness Linked Indicators Adapted to Novel Tasks (SALIANT) (Muniz et al. , 1998)
To solicit knowledge of compatible situation awareness 1. Delineate behaviours theoretically linked to team situations
(24 behaviours, 5 clusters).• Being able to predict next event• Managing resources (technical, system , internal, team)• Managing time• Feeling of being in control; able to implement elegant solutions• Taking the right decision at the best moment • Detecting mismatches, etc.
2. Development of task scenarios3. Identification of specific, observable responses for each
scenario4. Development of a script (to be administered to a team
member) 5. Observation form: Presence or Absence of response behavior
Team CTATeam CTA
4. Application of Strategies Every team has a prepared
list of steps, and routines for decision making, problem solving and planning
However, a team also develops shortcuts and workarounds not codified in procedures, as it gains experience
A skillful team knows its:– Key decision makers
(to be supported)– Key sources of expertise
(to be consulted) Time is critical in strategy
execution
Team CTATeam CTA
Methods to Elicit and RepresentTeam Strategies
Team CTATeam CTA
Retrospective Protocol Analysis (Means, 1993)
To solicit knowledge of agreement on strategies
– Based on non-interruption– Use video-tapes of similar mission or
actual team performance recorded previously
– Asking each team member to analyze the situation and justify strategies taken
– Analyze team agreement
Team CTATeam CTA
5. Metacognition A team needs to
monitor itself during ongoing task performance
To determine whether and when it is running into difficulty
Predict consequences of the difficulty
Determine when and how it needs to shift its strategies e.g. Police at riot situation
Team CTATeam CTA
Methods to Elicit and RepresentMetacognition
Team CTATeam CTA
Mission Simulation (Klein)
To solicit knowledge of self-monitoring or metacognition
– Bootstrapping to create a mission simulation– Asking team members to perform mission – Observing how team members recognize and
collect more information about critical incidents
– Conducting individual interviews with each team member to analyze the video-recording
– Analyzing gaps in team agreement
Team CTATeam CTA
Representing Team CTA Results
Team CTATeam CTA
Methods for Representing Team CTA Results Dealing with the ‘Envisioned World Problem’ (Woods, in
press)
– Creating prototypes and simulations to gauge how team CTA would impact the future
– Iterative testing
World Team CTA Prototypes of Changed
World
Team CTATeam CTA
Limitations / Challenges in Applying Team CTA Methods
Team CTATeam CTA
Challenges in Applying Team CTA Methods Knowledge elicitation
– Knowledge is internalized and even experts find it hard to verbalize it
– Retrospective account of work may differ from reality
Capturing dynamism – In many cases, team strategies differ
vastly from one situation to another – Generalizing based on a few situations
may be inappropriate
Team CTATeam CTA
Summary
Team CTA is critical to obtain a holistic (both overt and covert tasks) view of human work
Teams are intelligent entities with 5 key cognitive processes (Klein)
1. Shared mental models2. Control of attention3. Shared situation awareness4. Application of strategies to make decisions,
solve problems and plan5. Metacognition
Team CTATeam CTA
Summary
The 5 key processes can be spread over 2 types of team knowledge
– Pre-task knowledge– Dynamic Knowledge
Team CTA Methods should aim at eliciting team knowledge of these 5 processes
Team CTATeam CTA
Summary
Team CTA can yield significant performance improvement of teams by:
– Restructuring and Resizing the team – Designing better:
• Information technology• Information management strategies• Human-computer interfaces• Decision support systems and Communication
– Developing methods for team training and selection
Team CTATeam CTA
References Gary Klein
– Cognitive Task Analysis of Teams (1995) Cannon-Bowers, Sales, Baker
– Analyzing knowledge requirements in team tasks– Shared Mental Models in Expert Team Decision Making (1993)
Hoffman, R.R.– Use of Critical Incidents to Elicit Knowledge (1995)
Oransu, J.– Shared Mental Models and Crew Performance (1990)
Coury, B.G., Motte,S., & Seiford, L.M. – Capturing and representing decision processes in the design of
an information system Muniz, E.J., Stout, R.J., Bowers & Salas
– A methodology for measuring team situation awareness: Situation Awareness Linked Indicators Adapted to Novel Tasks (SALIANT) (1998)
Woods, D.D. & Hollnagel, E. – Mapping cognitive demands in complex problem solving worlds
(1987)
Team CTATeam CTA
Questions ?
Team CTATeam CTA
Cognitive Task Analysisof Teams
Thank You !