James Klinect, Ph.D.

Chief Executive Officer

IASS 2013 - Washington, DC - October 29-31, 2013

Introduction: James Klinect The University of Texas – LOSA/TEM Research

FAA funded (AAR-100) (presentations and publications)

Role: TEM / LOSA Project Manager (1996-2008)

Coauthored primary publications on LOSA

ICAO Handbook 9803 and FAA Advisory Circular 120.90)

The LOSA Collaborative – LOSA Implementation / TEM Training

Role: Founder and CEO

LOSA Registry of Airlines

Home to the LOSA Archive

Presentation Outline TEM Insights: Purpose and Applications Solidified

TEM Framework: A Visual Walk-Through

LOSA Insights: 10 Operating Characteristics Still Intact

LOSA Data Insights: Providing EBT Targets

Post-LOSA Insights: The Plight of Using Normal Ops Data

LOSA Resources

Question and Answer

What do you want your pilots to do to safely fly from A to B?

Common Response Disconnect?

Aviation (process focused) Passengers (outcome focused)

Use good airmanship

Manage fatigue

Follow SOP

Openly communicate

Practice good CRM

Manage problems

Avoid making mistakes

Manage mistakes

TEM provides a structured link between process and outcome

Threat Management: Crews Must Manage Operational Complexity

Crew

Factors

Crew Factors: Attributes that each

pilot brings with them to the cockpit

Anticipate or Recognize / Mitigate

Threat Threat: External event or error that occurs

outside the influence of the flight crew but

requires their attention if safety margins are

to be maintained

Aircraft

HardwareCRM LuckMonitor /

Cross-Check ChecklistsDeviation

Callouts AirmanshipPolicies /

Procedures

TEM Defenses Derived by Our LOSA Experience

Can increase or decrease the number of defenses

ex) CRM could be split into several different defenses

InquiryWorkload

Management LuckMonitor /

Cross-Check ChecklistsContingency

Management AirmanshipCaptain

Leadership

TEM Defenses are not Fixed; They are Plug and Play

Example: Linking TEM and LOSA Data

26% of LOSA Archive flights have a Checklist error

Most error prone checklists occur during Predeparture

Checklists

Monitor /

Cross-Check

Flight crews rated Poor/Marginal in Monitor/Cross-Check are 2 times more likely to have mismanaged errors and undesired aircraft states

Threat

Anticipate or Recognize / Mitigate

Crew

Factors

Pilot

Error

Detect / Correct

Unavoidable

Incident / Accident

Error Management: Crews Must Detect and Correct Their Errors

Undesired

Aircraft

State

Avoidable

Incident /

Accident

Recognize / RecoverCrew

Factors

Pilot

Error

Detect / CorrectThreat

Anticipate or Recognize / Mitigate

Unavoidable

Incident / Accident

UAS Management: Crews Must Manage Aircraft Deviations

Crew Factors Affect TEM Performance

Crew factors can increase

or decrease hole size in

defenses

Crew factor examples:

Knowledge

Skills

Fatigue

Personality

Personal Issues

Attitude

Procedures /

Policies

Monitor /

Cross-Check

Undesired

Aircraft

State

Avoidable

Incident /

Accident

Recognize / RecoverCrew

Factors

Pilot

Error

Detect / CorrectThreat

Anticipate or Recognize / Mitigate

Unavoidable

Incident / Accident

TEM Framework

Multiple Applications of TEM

Flight Crews

Structured and measurable “mindset” for pilots during flight

Storytelling / Sensemaking – Common Language & Framework

Incidents and accident analysis

Conduit for training concept delivery

Basis for flight crew evaluation (e.g., AQP)

Semi-calibrated pilot-to-pilot storytelling (briefings)

SMS Data Collection

A “leading” predictive performance indicator, not lagging

LOSA Operating Characteristics

1. Jumpseat observations during regular operations

2. Anonymous, confidential, and non-punitive data collection

3. Voluntary crew participation

4. Trusted and trained observers

5. Joint management / pilots association sponsorship

6. Systematic observation

instrument based on TEM

7. Secure data collection repository

8. Data verification roundtables

9. Data-derived targets for

enhancement

10. Feedback of results to line pilots

If your project has less than 10 characteristics, need to find

another acronym because you are not doing a LOSA as designed

100+ LOSA Projects Since 1996AeroMexico

Air Canada

Air Methods (HEMS)

Asiana

Alaska Airlines

Air France

Air New Zealand

Air Nelson

Air Niugini

Air Transat

ANA

ANA Network (4)

ASA

Braathens

Cathay Pacific

IndiGo

Japan Airlines

JetBlue

LACSA

QANTAS

Qatar Airways

Malaysia Airlines

Mexicana

Mt. Cook

REX Australia

Royal Jordanian

Republic

Saudia

Shuttle America

Chautauqua

China Airlines

Click Mexicana

COPA

Continental

Continental Micronesia

Delta

DHL Air

Dragonair

EVA Air

FedEx

Emirates

ExpressJet

Frontier

Horizon Air

Silk Air

Singapore Airlines

Singapore Cargo

Southwest

TACA

TACA Peru

TAP Portugal

Thomas Cook

UNI Air

United

US Airways

USAF Air Mobility Command

Virgin Australia

Volaris

WestJet

20,000+ LOSA Observations

LOSA Supports EBT With Archive Findings

LOSA: Some EBT Threat Management Targets

Challenging ATC speed clearances (13% mismanagement)

Three or more instructions in one ATC call (19% mismanagement)

Thunderstorms (12% mismanagement) Weather radar usage issues

Crews are typically unclear about when and how to far to deviate

Pop-Up Aircraft Malfunctions (16% mismanagement) Crews applying shortcuts rather than following procedures

Mostly during Predeparture

LOSA: Some EBT Error Management Targets

Automation error detection

1. Failure to / Wrong mode executed (46% mismanagement)

2. Wrong altitude dialed (40% mismanagement)

3. FMS entry errors during Predeparture (35% mismanagement)

Primary altimeters (46% mismanagement)

Mostly associated with low transitions below 8,000 ft.

Hand flying speed deviations (81% mismanagement)

Mostly during Descent/Approach

Rarely called out by PM even with SOP deviation calls

Stabilized Approach SOP

In LOSA observer training, we typically ask observers to

define their stabilized approach SOP

In response, we are often met by silence, blank stares, or

disagreement

Why is this?

Unstable Approaches

4% of LOSA flights have

an Unstable Approach

87% continued and landed without issue

10% continued and landed long, short, or significantly off

centerline

3% executed a missed approach

LOSA Stabilized Approach Definition

Stable by 1000 ft. IMC or 500 ft. VMC

• Speed between +10 and -5 target speed

• Sink rate no greater than 1000 fpm

• Vertical: No greater than 1 dot off GS

• Lateral: No greater than 1 dot off LOC

Unstable Approach Analysis 30% were linked to threats

ATC - Controller requiring crew to fly higher, faster, or shorter

routings (Challenging clearances)

Weather – Tailwind, low altitude windshear, or gusty conditions

PM’s are 20% more likely to speak up if the unstable

approach is linked to a threat

Emerging best practice: FDM callbacks to crew and

stabilized approach gates that start at 1500’ for speed

corrections

Intentional Noncompliance Observables

1. Committed multiple times during one phase of flight (e.g.,

missing multiple altitude callouts during descent)

2. Crew openly discusses they are intentionally committing an

action that is against published SOP in front of an observer

3. Observer justifies that the crew is time-optimizing SOP when

time is otherwise available

4. Observer justifies that an aircraft handling error involved an

increase in risk when more conservative options were

available (e.g., intentionally ducking under a glideslope).

Most Frequent Intentional Noncompliance

Error Prevalence - 49% of flights

Error Mismanagement - 19% link to other errors and/or

undesired aircraft state

Most Frequent Intentional Noncompliance

Checklist performed from memory / Use of nonstandard checklist protocol

Omitted altitude callouts

Failure to execute a mandatory missed approach

PF makes own flight guidance changes while hand flying

Taxi duties performed before leaving runway

LOSA Raw Data: Intentional NoncomplianceB777-300ER AAA to BBB PF: FO

Error Description: Established on ILS final at 2800ft, ATC required 160

knots to 5 miles. The CA instructed the FO to ignore it so that the aircraft

could be stable by 1500. F30 was selected and the MCP speed reduced to

126 by 5 miles. With the runway in sight, the FO offered to leave the speed

selected at 160 to 5, but the CA was adamant that it was to be ignored.

The FO had suggested a F25 landing to help with this situation before

descent, but the CA did not accept any of his suggestions. The CA did not

inform ATC of his intentions at any stage.

Error: Intentional Speed Deviation without ATC clearance / Speed too Low

Intentional Noncompliance is an Indicator of Eroding Safety Margins

Crew with at least one intentional noncompliance error

are 2-3 times more likely to mismanage threats and errors

TEM Indicator

Flights with zero

Intentional

Noncompliance errors

Flights with one

Intentional

Noncompliance error

Flights with two or more

Intentional Noncompliance

errors

% of observations 51% 24% 25%

Average number of threats per flight 4.5 4.7 4.8

Average number of errors per flight 2.1 3.9 7.5

% of flights with a mismanaged threat 26% 40% 54%

% of flights with a mismanaged error 29% 47% 67%

% of flights with an UAS 27% 43% 60%

Does your SMS address this issue?

Problem: LOSA Data Users Are Overloaded

Flight Ops – Manages safety crises, industrial issues, schedule

changes, and everything else.

Standards – Writes new SOP, attempts to integrate with

Manufacturer SOP, which contains lots of pilot discretion procedures.

Training – Frequently chasing their tails with current expansion

programs and trying to get pilots inducted and cleared to fly.

Result: Stakeholders are typically not motivated to address proactive

safety data issues because:

No crisis or urgency are attached to the results

There is no time for stakeholders to strategize and design fixes

Solution: What Airlines Are Doing toEffectively Use LOSA Data

See LOSA and other normal ops data as establishing a multi-year safety strategy

Form a LOSA review board (mostly line pilots) that can the study findings and present solutions to management

Flight Safety or QA must hold people accountable that enhancements are being made

Incentivized to use LOSA results (insurance discounts, line check credits, IOSA audit credits, or regulator happiness)

LOSA change management must supported and pushed by top management

LOSA Resources

Web Resources

The LOSA Collaborative

www.losacollaborative.com

(New site in December 2013)

LOSA Advisory Circular 190.90

Best public doc available

ICAO Document 9803

Information is good but

needs an update / TEM

content is out of date

Question and Answer

LOSA Observer

Wilbur

Threat and Error

Manager Orville

LOSA and TEM in 1903