Edited Measuring Safety Performancecaa.co.za/Safety Seminars and Presentations/Measuring... ·...

Measuring Safety Performance

Presented by: Keboitihetse Fredy TongDate: 2017/04/04

SMS – managing safety in the interest of all…

Safety Performance Measurement (SPM):SPI & ALoSP Development

Agenda• Definition.• Why measure safety performance?• Alert Level Setting.• Alert Level Trigger.• Target Level Setting (planned improvement).• Target Achievement.• Acceptable Level of Safety performance (ALoSP).


Safety Performance Indicator

Ø Safety Performance Indicator: A data-based safetyparameter used for monitoring and assessing safetyperformance. ALL OPERATOR-REPORTED INCIDENTS/ACCIDENTS


0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

jan feb mar apr may jun jul aug sep oct nov decPreceding Year Combined Operators Monthly Reportable Incident rate (per 1000FH)Preceeding Year Average (Ave)


ØAlert Level: An alert level is a common criterion todelineate the acceptable from the unacceptableperformance regions for a particular safety indicator.Ø Target Level: The required level of safety

performance of a system, during aviation activitiesthat are centred around the set organisational safetyobjective.



ØAcceptable Level of Safety Performance (ALoSP):The minimum level of safety performance of civilaviation in a State, as defined in its State SafetyProgramme, or of a service provider, as defined in itssafety management system, expressed in terms ofsafety performance targets and safety performanceindicators.


Measure What Matters Most

Ø Safety Performance Indicators (SPIs)◦ Reduce complexity to a small number of keyindicators.

Ø This is the same approach we use in our personalhealth◦ Blood pressure, cholesterol levels, heart rate.

ØMost aviation accidents are caused by human error◦ Suggestion: Measure factors related to human

error.


SPI TRAP

ØAVOID THIS TRAP!!◦ Identify everything that is easy to measure andcount◦ Report the data on everything easy to measureand count◦ End up confused and overwhelmed with data.


SPIs to Consider

Ø How often are these happening, and why?◦ Unstable Approaches◦ Procedural Errors or Lapses◦ Working Fatigued◦ Minimum Fuel Events◦ TCAS RA Events◦ EGPWS or TAWS Alerts/Warnings◦ Events related to LOCI, for example:

• Low speed/stall alert or warning• Bank Angle alert or warning.


Safety Performance Target

Ø The planned or intended objective for safetyperformance indicator(s) over a given period.◦ Technique

• Gather data over a one-year period• Compute average• Set a reasonable Goal (Target) to improve.


Hi-Consequence Indicators

Ø SPIs pertaining to the monitoring and measurementof high-consequence occurrences, such asaccidents or serious incidents, i.e. MandatoryOccurrence Reports (MORs).

Ø High-consequence indicators are sometimesreferred to as reactive indicators.


Low-Consequence Indicator

Ø SPIs pertaining to the monitoring and measurementof lower-consequence occurrences, events oractivities such as incidents, non-conformancefindings or deviations.

Ø Lower-consequence indicators are sometimesreferred to as proactive/predictive indicators.


SPI Alert and Target Values

Ø Safety performance is expressed by SPIs and theircorresponding alert and target values.Ø Monitor the performance of SPI trends to identify

any abnormal changes in safety performance.Ø Target and alert settings should take into

consideration recent historical performance for agiven indicator.Ø Targets should be realistic and achievable (SMART).


Past Performance

Ø Past performance may be an indicator of futureperformanceØ Employ trend analyses to track safety performance

over timeØ Where deficiencies have been found and corrected,

ensure the effectiveness of corrective actions.


SPI and Performance Monitoring

Ø Normally depicted in the form of charts or graphs

Ø Target Example: 5% Better than Last Year AverageØ Alerts related to Data Points (DP) & Standard Deviations (SD)Ø ALERT:

◦ 1 DP > 3 SD◦ 2 DP > 2 SD◦ 3 DP > 1 SD


Ave+1 SD

Ave+2 SD

Ave+3 SD

Target

dec jan feb mar apr may jun jul aug sep oct nov dec

Current Year Combined Operators Monthly Reportable Incident rate (per1000 FH)Current Year Target Average

Performance Summaries

Ø Once SPIs and corresponding targets and alertsettings have been defined, the performanceoutcome of each indicator should be updated andmonitored on a regular basis.Ø A summary of the overall target and alert

performance outcome of the complete safetyperformance indicators package may be aggregatedfor a given monitoring period.


How to Use SPI ResultsØ SPIs are NOT simply metrics used to get a better scoreØ SPIs are to be utilised to improve safety performanceØ Results include collection, analysis, and interpretation of

SPIsØ It is important that these results are used by management

for decision and action.Ø These results should be presented at regular meetings and

communicated to everyone in the organisationØ Actions should not focus on certain indicators in isolation,

but on optimising your organisation’s overall safetyperformance.


Evaluate SPIs

Ø Periodically review and evaluate your SPIs toconsider:◦ the value of experience gained,◦ new safety issues identified,◦ changes in the nature of risk,◦ changes in the safety policy, objectives, and

priorities,◦ changes in applicable regulations, and◦ organisational changes, etc.


Summary

Ø Measuring Safety Performance◦ SPI’s must be meaningful◦ SPI’s must relate to the safe operation of aircraft

• Flight Operations• Maintenance• Dispatch


Edited Measuring Safety Performancecaa.co.za/Safety Seminars and Presentations/Measuring... ·...

Documents

Transcript of Edited Measuring Safety Performancecaa.co.za/Safety Seminars and Presentations/Measuring... ·...