Improving Cockpit Task Management Performance:
description
Transcript of Improving Cockpit Task Management Performance:
![Page 1: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/1.jpg)
Improving Cockpit Task Management Performance:
Aviate
Navigate
Communicate
Manage Systems
The AgendaManagerTraining Pilots to Prioritize Tasks
![Page 2: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/2.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
2
Observation: Cockpit Task Management Errors
• Cockpit (flight deck) is a multitask environment– aviate– navigate– communicate– manage systems
• Results of distraction, preoccupation– Everglades L-1011 accident– many incidents
• Hypotheses: – flightcrew must manage as well as perform tasks:
Cockpit Task Management (CTM) – CTM is a significant factor in flight safety
![Page 3: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/3.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
3
Preliminary Normative Theory of CTM
• initiate tasks to achieve goals
• assess status of all tasks
• terminate completed and ‘obsolete’ tasks
• prioritize remaining tasks based on– importance:
• aviate
• navigate
• communicate
• manage systems
– urgency
– other factors (?)
• allocate resources (attend) to tasks in order of priority
![Page 4: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/4.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
4
Cockpit Task Management Research
• CTM Errors in Aircraft Accidents (1991)– 80 CTM errors in 76 (23%) of 324 accidents
• CTM Errors in Critical, In-Flight Incidents (1993)– 349 CTM errors in 231 (49%) of 470 incident reports
• Part-Task Flight Simulator Study (1996)– CTM error rate increases with workload
• ASRS Study of CTM and Automation (1998)– Task prioritization error rate higher in advanced technology reports
• Findings:– CTM is a significant factor in flight safety
– CTM can potentially be improved
![Page 5: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/5.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Improving CTM Through Technology:
The AgendaManager
5
![Page 6: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/6.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
6
Statement of Needs and Requirements Definition
• CTM aid shall– maintain a current model of aircraft state and current
cockpit tasks,
– monitor task state and status,
– compute task priority,
– remind the flightcrew of all tasks that should be in progress, and
– suggest that the flightcrew attend to tasks that do not show satisfactory progress.
– leave the pilot in control
![Page 7: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/7.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
7
System Analysis
• Generic, twin-engine transport aircraft– major subsystems
• power plant
• fuel system
• electrical system
• hydraulic system
• adverse weather system
• autoflight system
• flight management system.
– state variables of importance to pilot
specifications for simulator
![Page 8: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/8.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
8
Basic and Detailed Design of
The AgendaManager• Object-Oriented Design
– things & activities from IDEF0 models objects
• Multi-Agent Approach– AMgt functions are complex, cognitive functions AI– AMgt is complex interplay of many entities DAI
• System Agents• Actor Agents• Goal Agents• Function Agents• Agenda Agent• Agenda Manager Interface
• Display Design– general display design guidelines alternative display designs– consistency with EICAS final display design
![Page 9: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/9.jpg)
Autoflight
L Engine
Hyd System
Fuel System
Aircraft
Pilot
Aircraft Agent
Autoflight Agent
L Engine Agent
Hyd System Agent
Fuel System Agent
Flightcrew Agent
Simulator
AMgr display
Verbex ASR
AgendaManager
descend to 9,000 ft G & F Agents
descend to 8,000 ft G & F Agents
extinguish L ENGINE FIRE G & F Agents
restore C HYD PRESS G & F Agents
correct FUEL BALANCE G & F Agents
Goal & Function AgentsSystem AgentsSystem Models
reduce to 240 kt G & F Agents
maintain 070 deg G & F Agents
maintain 070 deg G & F Agents
reduce to 240 kt Goal & Function Agents
information flow
unsatisfactory functionsconflicting goals
satisfactory functions
AMgr Architecture and Function
![Page 10: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/10.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
10
Simulator(with EICAS)
![Page 11: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/11.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
11
AMgr Display(replaced EICAS)
![Page 12: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/12.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
12
AMgr Operation
• simulator runs• pilot declares goals via ATC acknowledgements• System & Actor Agents instantiate Goal Agents• Goal Agents watch for goal conflicts• Function Agents assess function status• AgendaManager informs pilot via display
![Page 13: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/13.jpg)
extinguish L engine fire not OK -> continuing
slow to 240 kt fast -> acceleratingmaintain 070 degdescend to 7,000 ft A/F alt goal conflict
correct fuel balance L heavy -> increasing
AgendaManager Display Design
![Page 14: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/14.jpg)
extinguish L engine fire not OK -> continuing
slow to 240 kt fast -> acceleratingmaintain 070 degdescend to 7,000 ft A/F alt goal conflict
correct fuel balance L heavy -> increasing
extremely important, urgent goals
(highest priority) trend info
aviate goals(high priority)
system goals(lower priority)
gray = OKamber = not OKred = important/urgent not OK
![Page 15: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/15.jpg)
maintain 280 ktmaintain 120 degmaintain15,000 ft
Initial Conditions:altitude = 15,000 ftheading = 120 degspeed = 280 ktall systems normal
![Page 16: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/16.jpg)
maintain 280 ktmaintain 120 degdescend to 11,000 ft high -> descending
ATC: “... descend and maintain 11,000 ft”pilot: “Roger, “... descend and maintain 11,000 ft”
sets A/F altitude to 11,000 ftdescent begins
![Page 17: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/17.jpg)
maintain 280 ktturn L to 070 deg right of -> turning Lmaintain 11,000 ft
ATC: “... turn left heading 070”pilot: “Roger, “... turn left heading 070”
begins turnlevels off at 11,000 ft
![Page 18: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/18.jpg)
maintain 280 ktmaintain 070 degmaintain 11,000 ft
correct fuel balance L heavy -> unbalancing
pilot: rolls out on 070 degAMgr:detects fuel imbalance & displays it
![Page 19: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/19.jpg)
slow to 240 kt fast -> slowingmaintain 070 degdescend to 9,000 ft high -> descending
correct fuel balance L heavy -> balancing
pilot: begins fuel crossfeedATC: “... descend and maintain 9,000 ft; reduce
speed to 240 kt”pilot: “Roger ... descend and maintain 9,000 ft;
reduce speed to 240 kt”sets altitude to 9,000 ft, descent beginsreduces throttles, aircraft slows
![Page 20: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/20.jpg)
extinguish L engine fire not OK -> continuing
slow to 240 kt fast -> acceleratingmaintain 070 degdescend to 7,000 ft A/F alt goal conflict
correct fuel balance L heavy -> balancing
AMgr: detects left engine firepilot: “... we have a problem ...”ATC: “... descend and maintain 7,000 ft”pilot: “Roger ... descend and maintain 7,000 ft”
mis-sets altitude to 6,000 ftspeed increases
![Page 21: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/21.jpg)
maintain 240 ktmaintain 070 degmaintain 7,000 ft
correct fuel balance R heavy -> unbalancing
fire outspeed controlledpilot: sets A/F to 7,000 ft
forgets to secure crossfeed when fuel balanced
![Page 22: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/22.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
22
Test and Evaluation (1)
• Objective: compare AMgt performance (AMgr vs EICAS)
• Apparatus– flight simulator
– AMgr
• Subjects: 8 line pilots
• Scenarios:– EUG to PDX
– PDX to Eugene
• Primary factor: monitoring and alerting condition– AMgr
– EICAS
![Page 23: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/23.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
23
Test and Evaluation (2)
• General Procedure– subject introduction– automatic Speech Recognition system training– flight training (using MCP)– subsystem training (fault correction)
– EICAS/AMgr training
• Trials– Scenario 1 (EICAS/AMgr)
• experimenter/ATC controller gives clearances, induces faults, induces goal conflicts
• subject acknowledges clearances, flies simulator, corrects faults, detects and resolves goal conflicts
– Scenario 2 (AMgr/EICAS)
![Page 24: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/24.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
24
Response variable AMgr EICAS sig. levelwithin subsystem correct prioritization 100% 100% NS
susbsystem fault correction time (sec) 19.5 19.6 NS
autoflight system programming time (sec) 7.0 5.9 NS
goal conflicts corrected percentage 100% 70% 0.10
goal conflict resolution time (sec) 34.7 53.6 0.10
subsystem/aviate correct prioritization 72% 46% 0.05
average number of unsatisfactory functions 0.64 0.85 0.05
percentage of time all functions satisfactory 65% 52% 0.05
mean subject effectiveness rating (-5 to 5) 4.8 2.5 0.05
Evaluation Results
![Page 25: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/25.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
25
Conclusions
• CTM is a significant factor in flight safety.• CTM can be facilitated (e.g., AMgr).• Future success of knowledge-based avionics depends
on a systematic approach to development:– systematic identification of problems, needs, and
opportunities
– appropriate application of appropriate technology
– evaluation of systems based on operationally relevant performance measures
![Page 26: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/26.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Improving CTM Through Training:
Training Pilots to Prioritize Tasks
26
![Page 27: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/27.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
ResearchMotivation and Objective
• Is task prioritization trainable?• Evidence suggests that voluntary control of
attention is a trainable skill– e.g., Gopher (1992)
• Objective– Develop and evaluate a CTM training program to
improve task prioritization performance.
![Page 28: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/28.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTMMethodology
• Participants– 12 General Aviation pilots, IFR rated, with at least 100 hrs “pilot-
in-command” total time.
– Recruited through flyers and word of mouth
– Oregon State (Corvallis, Albany, Salem, Eugene, Portland)
• Apparatus: Microsoft Flight Simulator 2000– 3 monitors, Flight Yoke, Throttles, and Rudder Pedals
– IFR conditions
– Two flight scenarios
![Page 29: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/29.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Lab Setup
![Page 30: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/30.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Participant Display(C-182RG)
![Page 31: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/31.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Experimenter’s Display
![Page 32: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/32.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Experimental Groups
• Control Group: No Training• Descriptive Group: CTM lecture
• Multi-tasking• Attention• CTM• Task Prioritization errors• Accident/Incident examples• What to be aware of.
• Prescriptive Group:– CTM lecture – “APE” procedure
![Page 33: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/33.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
APE:Assess Prioritize Execute
• Let the APE help you– Assess the situation:
• aircraft systems, environment, tasks, procedures• “What’s going on?” “What should I be doing?”
– Prioritize your tasks:1. Aviate: “Is my aircraft in control?”2. Navigate: “Do I know where I am and where I’m going?”3. Communicate: “Have I communicated or received important information?”4. Manage systems: “Are my systems okay?”
– Execute the high priority tasks Now.
• Invoke the APE frequently.• Think out loud.
A P E
![Page 34: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/34.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Experimental Procedure
1. Initial briefing, informed consent
2. Initial 30-minute simulator training
3. Pre-training flight
4. CTM training (break for control group)
5. Additional 30-minute simulator training
6. Post-training flight (different scenario)
7. Post-experiment questionnaire
![Page 35: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/35.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Dependent Measures
• Task prioritization error rate– 19 Task prioritization challenges, e.g.
• clearance near end of climb
• “bust” altitude? (+/- 200 ft)
• Prospective memory recall rate– 5 Memory recall challenges (prospective memory), e.g.,
• “report crossing SHONE [intersection]”
• remember to report?
![Page 36: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/36.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Data Collection
• Flight Data Recorder• Videotape• Observation• Data reduction to:
– task prioritization error rate
– prospective memory recall rate
![Page 37: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/37.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Results: ANOVA(task prioritization error rate)
Effect df SS df MS F-ratio p-value
Group 2 .1914125 9 .0799194 2.395 .147
Flight 1 .2053500 9 .0088806 23.123 .001
Group x Flight 2 .0429125 9 .0088806 4.832 .038
![Page 38: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/38.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Interaction Plot(task prioritization error rate)
Interaction Plot
Flight
Err
or R
ate
GroupControlDescriptivePrescriptive
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Pre Training Post Training
Control
Descriptive
Prescriptive
![Page 39: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/39.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Results: ANOVA(prospective memory recall rate)
Effect df SS df MS F-ratio p-value
Group 2 .017 9 .028 .603 .568
Flight 1 .074 9 .034 2.181 .174
Group x Flight 2 .171 9 .034 5.055 .034
![Page 40: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/40.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Interaction Plot(prospective memory recall rate)
Interaction Plot
Flight
Mem
ory
Tas
ks
GroupControlDescriptivePrescriptive
0.5
0.6
0.7
0.8
0.9
1
Pre Training Post Training
Prescriptive
Descriptive
Control
![Page 41: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/41.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Paired t-tests
• Prescriptive training group improved• Task prioritization error rate
• Prospective memory recall rate
• Descriptive training group improved• Task prioritization error rate
• Control group did not significantly improve
![Page 42: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/42.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Discussion
• Task Prioritization Error rate– Reduced, perhaps, due to (Prescriptive) CTM training.– Significant interaction and post-hoc tests support that
hypothesis.
• Prospective Memory Recall rate– Increased, perhaps, due to (Descriptive & Prescriptive) CTM
training.– Significant interaction and post-hoc tests support that
hypothesis.
![Page 43: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/43.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Possible Interpretations
• Results may have two interpretations:1. CTM training did improve task prioritization performance.
2. CTM training did not improve task prioritization.• Floor effect
• MSFS experience
• Age
• Research favors first interpretation• ANOVA results
• t-tests
• Potential for better control group performance was there.
• Additional tests
![Page 44: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/44.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
Final Comments
• CTM performance significant to flight safety• Results are encouraging• Evidence suggests that task prioritization is a
trainable skill• Follow-up experiment underway to resolve
ambiguities• If successful, would provide evidence that CTM
training can reduce risk of CTM errors and subsequent accidents
![Page 45: Improving Cockpit Task Management Performance:](https://reader036.fdocuments.net/reader036/viewer/2022070418/568159df550346895dc729cb/html5/thumbnails/45.jpg)
Aviate
Navigate
Communicate
Manage Systems
CTM
The AMgr: a KBS 45
The Cockpit Task Management Website
http://flightdeck.ie.orst.edu/CTM/