Line Operations Safety Audit (LOSA) - Leading Edge … Dubai.pdf · collect data about flight crew...

Dr Graham EdkinsGeneral Manager Safety Systems & Education

Corporate Safety Department

Line Operations Safety Audit (LOSA)

“… a program that uses expert and highly trained observers to collect data about flight crew behaviour and situational factors

on normal flights”.

Qantas LOSA Dubai, 14 October 2002


IN BRIEF IN BRIEF

Business rationale for LOSA at Qantas.

How did Qantas implement LOSA?

Lessons learned.

Future plans.


BUSINESS RATIONALEBUSINESS RATIONALE

Crews’ may not always report safety issues.

Crews’ normal behaviour may be different when being

observed during line/route checks.

Current safety information tends to be reliant on

system and crew failure - something bad has to happen!

Lack of data about flight crew strategies for managing

error.

Lack of data about what crew do well.


THREAT & ERROR MANAGEMENT


CONVINCING THE BEANCOUNTERS!

DEVELOPING A BUSINESS CASE

CONVINCING THE BEANCOUNTERS!

DEVELOPING A BUSINESS CASE

Approval signatures

Executive Summary

Purpose and scope of project

Business requirement, benefits and risks

Recommended project approach

Project cost

Project Milestones/Deliverables

Business Case

Line Operations Safety Audit (LOSA) implementationstrategy for Flight Operations


28 December 2001


STRATEGY FOR LOSA IMPLEMENTATION

STRATEGY FOR LOSA IMPLEMENTATION

Option 1. The utilisation of the University of Texas to provide LOSA planning, data collection, analysis and report writing services.

Option 2. The utilisation of an Australian university to assist in the development of data collection and analysis expertise within Qantas.

Option 3. The development and implementation of LOSA in-house with no external assistance.

Option 4. A combination of options 1-3.

Option 1. The utilisation of the University of Texas to provide LOSA planning, data collection, analysis and report writing services.

Option 2. The utilisation of an Australian university to assist in the development of data collection and analysis expertise within Qantas.

Option 3. The development and implementation of LOSA in-house with no external assistance.

Option 4. A combination of options 1-3.


IPLEMENTATION STRATEGYIPLEMENTATION STRATEGYAdopted a “shotgun” approach.

“Non-blinkered” approach - no pre-analysis of FOQA,

Check or Safety Incident data.

Mimimum of 50 observation per fleet (B737, B767,

B744).

Consideration of sample design issues.

Mix of internal and external observers.

Strive to maintain full integrity of the LOSA process,

eg. minimal QF changes.


WHAT DOES IT COST?WHAT DOES IT COST?

Direct Cost.LOSA Collaborative Fee.

LOSA Collaborative expenses (travel, accommodation, daily

expenses) - AUD $30,000.00 *

Indirect Cost.Unproductive costs of crew (observer) = direct cost X 4 **

* Based on 3 X LOSA Collaborative observers.** Based on 20 X QF Observers conducting a total of 200 observations.


Risks Potential Impact Likelihood Mitigation StrategyNon acceptance ofthe project by linecrew.

Line crew mistrustand scepticism ofLOSA as anotherline check.

Medium Produce regular and clear information to line crewabout the objectives of LOSA. Produce a MOUco-signed by AIPA, GGM Corporate Safety andGGM Flight Operations, clearly outlining howLOSA results will be used.

Non acceptance ofthe project by AIPA.

Delay in thedevelopment ofthe project.

Low Involve AIPA in the selection of LOSA observers.Provide regular updates on LOSAimplementation.

Selection of LOSAobservers that havepoor credibility.

Line crew mistrustand scepticism ofLOSA.

Medium Select LOSA observers using a process agreedto by management and AIPA.

Mis-handling ofsensitive informationcollected throughaudits.

Breach ofconfidentiality ofline crewundergoingLOSA.

Medium Restrict the handling and storage of LOSA data toobservers and UT. Communicate policy to linecrew about observation of non-compliance withSOP’s.

Poor training ofobservers inbehaviouralobservationstrategies.

Lack of valid data. Medium Develop behavioural observation expertise withincore group of LOSA observers. Utilise theUniversity of Texas to conduct observer training.

CONSIDER THE RISKS TO SUCCESSCONSIDER THE RISKS TO SUCCESS


LOSA Observer Quick Reference GuideCaptain Brief (a.k.a., Sales job)

• Ask for permission to ride – show the letters• Stress de-identified and confidential data• Ask if you can to take notes• Schedule the crew interview and time away from the cockpit

The Four Observer Tasks 1. Write a narrative on “what happened and how / why”2. Threats Threat Management Threat Outcome3. Errors Error Management Error Outcome4. Rate CRM performance on how the crew managed threats and errors

Narratives – three questions – keep concise and simple 1. What did the crew do well as risk managers?2. What did the crew do poorly as risk managers?3. How did the crew manage threats, errors, or other significant events?

Threat Management – three questions – follow the model 1. What was the threat?2. How did the crew respond to the threat?3. What was the outcome of the threat?

Error Management – four questions – follow the model 1. What was the error?2. How did the crew respond to the error?3. What was the outcome of the error?4. Did the crew get into any undesired aircraft states?

CRM Performance • Don’t have to rate everything – rate markers with confidence• Justify behavioral markers with ratings 1,2, or 4

Crew Interview • Ask questions first and demographics last

Blue Box Information • Was the approach briefed before the TOD?• Did the crew begin the descent before or at the TOD?• Did the aircraft get significantly above / below FMS path?• Approach flown?

o Visual, Precision, Non-precision, or precision to visualo Hand-flown or automation flown?

• Weather (VMC or IMC)

• Stabilized approach parameters (1500, 1000, and 500 HAT)o Airspeed – CAC to Vref +20o ROD <1000o Engines spooled appropriate to conditionso +1/-1 dot on G/S and localizero Proper landing configuration before 500’

• Landing runway and did the crew brief the landing runway?

SEND OBSERVATIONS: [email protected]

SAVE FILE AS:1 .ObserverID.ObservationNumber.doc2. ObserverID.ObservationNumber.ExtraThreats.doc3. ObserverID.ObservationNumber.ExtraErrors.doc

Examples) 3059.#1.doc or 99997.#5.ExtraThreats.doc

James Klinect contact: [email protected]

Pat Murray contact: [email protected] Home: 0755 029 748Cell Phone: 0402 019 747Mail: PO Box 241, Sanctuary Cove, QLD 4212

Ian Topfer Numbers:Mobile: 0417 265 313 Home: 9983 9994 Work: 9691 1881


26 April 2002

Re: Letter of Introduction for LOSA Observers

Dear Qantas Captain,

This letter introduces a Line Operations Safety Audit (LOSA) observer, who is conductingsafety observation flights during May and June 2002 across Qantas B737, B767 and B747operations.

LOSA is a new safety audit program being introduced by Flight Operations in conjunctionwith the Corporate Safety Department and the University of Texas. Trained observers collectinformation on normal flights about how crew identify potential threats to safety and howthese threats are managed.

By gaining an understanding of these issues, Qantas will be better able to identify safety risks,training needs, and high threat operations. This information will complement our wellestablished programs such as FOQA and safety reporting.

The trained observers are not check and training oriented. They are there to observe andrecord data that will be de-identified and sent directly to the University of Texas. No names,flight numbers or dates are collected.

If any Captains have concerns about LOSA they should contact Captain Ian Topfer on (02)9691 1881 or 0417 265 313.

The LOSA program has the full support of both Flight Operations Management and AIPA.

Yours sincerely,

Captain Ian Lucas Captain Chris ManningGGM Flight Operations & Chief Pilot President, AIPA


TARGETED AND CONDUCTED OBSERVATIONS

TARGETED AND CONDUCTED OBSERVATIONS

Fleet Minimum ObservationsTargeted

Actual Observations

B 737 70 77

B 747-400 50 69

B 767 80 116

TOTALS 200 262

* Observers conducted observation on at least 2 of 3 fleets


OBSERVER DEMOGRAPHICSOBSERVER DEMOGRAPHICS

ObserverPosition

Number ofObservers

Number ofObservations

Captain 10 107

First Officer 6 77

Other QantasPilots/Personnel

4 49

LOSA Collaborative 3 29

TOTALS 23 262


TIME (2002) TASKJan-April Education/communication activities, sample

design issues, finalise selection of LOSAobservers

April 30 Observer training course

May-June Conduct of 200 + observations

June Data entry of observation forms (UT)

August Data cleaning (UT)

September Briefing by UT to Qantas on results

October Develop strategy for action on LOSA results

November Briefing to QF Board Safety Committee

IMPLEMENTING LOSAIMPLEMENTING LOSA


LESSONS LEARNEDLESSONS LEARNED

Lack of B737 observers.

Inform CSM’s about LOSA program earlier.

Warn Captains about LOSA flight.

Be better prepared for the use of external observers.

Should have conducted observations on B747 “classic”.

Lack of B737 observers.

Inform CSM’s about LOSA program earlier.

Warn Captains about LOSA flight.

Be better prepared for the use of external observers.

Should have conducted observations on B747 “classic”.


FUTURE PLANSFUTURE PLANS

Have appointed Lead Project Manager to develop action strategy for LOSA findings.

Qantas Link LOSA implementation 2003.

The monitoring of safe behaviour in airport ramp operations:

CASA funded Phd Scholarship - Michael Piotrowitz - University of Newcastle.

Cabin Crew LOSA.

Have appointed Lead Project Manager to develop action strategy for LOSA findings.

Qantas Link LOSA implementation 2003.

The monitoring of safe behaviour in airport ramp operations:

CASA funded Phd Scholarship - Michael Piotrowitz - University of Newcastle.

Cabin Crew LOSA.


APPLICATION ISSUESThere are a number of logistical problems in

conducting a cabin crew type LOSA:

APPLICATION ISSUESThere are a number of logistical problems in

conducting a cabin crew type LOSA:

) Cabin environment is not as contained as a cockpit.

) Double deck aircraft.) All information goes through CSM. ) Observers are more obtrusive in cabin.) Errors tend to be less consequential in

the cabin. ) Impact on customers & service. ) Multiple crew to observe.) Less external threats to inflight safety

in the cabin.

) Cabin environment is not as contained as a cockpit.

) Double deck aircraft.) All information goes through CSM. ) Observers are more obtrusive in cabin.) Errors tend to be less consequential in

the cabin. ) Impact on customers & service. ) Multiple crew to observe.) Less external threats to inflight safety

in the cabin.

Dr Graham EdkinsGeneral Manager Safety Systems & Education


+ 61 2 9691 8095

[email protected]


CONTACT DETAILSCONTACT DETAILS

Line Operations Safety Audit (LOSA) - Leading Edge … Dubai.pdf · collect data about flight crew...

Documents

Transcript of Line Operations Safety Audit (LOSA) - Leading Edge … Dubai.pdf · collect data about flight crew...