CAR/SAM Regional Guidance Material on Air Traffic Services Quality Assurance Programmes NAM/CAR/SAM...

CAR/SAM Regional Guidance Material CAR/SAM Regional Guidance Material on Air Traffic Services Quality on Air Traffic Services Quality

Assurance ProgrammesAssurance Programmes

NAM/CAR/SAM Quality Assurance WorkshopNAM/CAR/SAM Quality Assurance Workshop

Mark RíosMark RíosAir Traffic Management OfficerAir Traffic Management Officer

ICAO NACC Regional OfficeICAO NACC Regional Office

NAM/CAR/SAM Air Traffic Services Quality Assurance Workshop(Port of Spain, Trinidad and Tobago, 26 to 30 November 2001

NAM/CAR/SAM Air Traffic Services Quality Assurance Workshop

(Port of Spain, Trinidad and Tobago, 26 to 30 November 2001

INTERNATIONAL CIVIL AVIATION ORGANIZATIONINTERNATIONAL CIVIL AVIATION ORGANIZATION

NORTH AMERICAN, CENTRAL AMERICAN AND CARIBBEAN OFFICENORTH AMERICAN, CENTRAL AMERICAN AND CARIBBEAN OFFICE

CAR/SAM REGIONAL GUIDANCE MATERIAL ONCAR/SAM REGIONAL GUIDANCE MATERIAL ON

AIR TRAFFIC SERVICESAIR TRAFFIC SERVICES

QUALITY ASSURANCE PROGRAMMESQUALITY ASSURANCE PROGRAMMES

Chapters 5 – 8Chapters 5 – 8

Version 1.0Version 1.0

November 2001November 2001



ContentsContents

Chapter 5.Chapter 5. Air Traffic Incident Reporting, Investigations, and Air Traffic Incident Reporting, Investigations, and Investigation TeamsInvestigation Teams

Chapter 6.Chapter 6. Air Traffic Incident Prevention ProgrammesAir Traffic Incident Prevention Programmes

Chapter 7.Chapter 7. Air Traffic Services Evaluation ProgrammeAir Traffic Services Evaluation Programme

Chapter 8.Chapter 8. Quality of Services Improvement ProgrammesQuality of Services Improvement Programmes



Chapter 5. Air Traffic Incident Reporting, Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation TeamsInvestigations, and Investigation Teams

IntroductionIntroduction– Air traffic incidentsAir traffic incidents

AIRPROXAIRPROX

Operational errorOperational error

Operational deviationOperational deviation

Runway incursionRunway incursion

Pilot deviationPilot deviation

– Casual factorCasual factorProceduralProcedural

Ground facilities (equipment)Ground facilities (equipment)

HumanHuman




General reporting proceduresGeneral reporting procedures– Reporting by pilotsReporting by pilots– Reporting by ATSReporting by ATS– ATS incident report form for use by pilots and ATS incident report form for use by pilots and

controllerscontrollers

Investigation and documentationInvestigation and documentationAnalysis of ATS incidentAnalysis of ATS incidentRelease of informationRelease of information– In the interest of accident/incident preventionIn the interest of accident/incident prevention– Publish report as soon as possiblePublish report as soon as possible




Air traffic incident investigation processAir traffic incident investigation processStep-by-step processStep-by-step processSuspected ATS incident eventSuspected ATS incident event– Identify and report ATS deficienciesIdentify and report ATS deficiencies– That problems could be correctedThat problems could be corrected– Reporting of ATS incidentsReporting of ATS incidents

Requirement within StatesRequirement within StatesResponsibility of all ATS personnelResponsibility of all ATS personnelImmediately report suspected ATS incidents to ATS Immediately report suspected ATS incidents to ATS supervisor or appropriate ATS administration officialsupervisor or appropriate ATS administration officialWithout fear of reprisalWithout fear of reprisal




Preliminary ATS incident investigationPreliminary ATS incident investigation– Fact findingFact finding– Determine what occurred in the ATS systemDetermine what occurred in the ATS system– Determine validity of the suspected ATS incidentDetermine validity of the suspected ATS incident– Confirmation of ATS incidentConfirmation of ATS incident

Take appropriate actionTake appropriate action

Advise other ATS unit if involvedAdvise other ATS unit if involved

Relieve controller(s) may be involvedRelieve controller(s) may be involved

Gather pertinent information (e.g. flight progress strips, Gather pertinent information (e.g. flight progress strips, etc.)etc.)

Review voice recordings and prepare working copyReview voice recordings and prepare working copy




Review computer and radar dataReview computer and radar dataConduct preliminary interviewsConduct preliminary interviewsNotify ATS managementNotify ATS managementComplete preliminary ATS Incident Investigation Complete preliminary ATS Incident Investigation WorksheetWorksheet

Follow-up investigationFollow-up investigation– ATS unit chief shouldATS unit chief should

Ensure ATS incident investigations are conductedEnsure ATS incident investigations are conductedCoordinate with other ATS unit chief if involvedCoordinate with other ATS unit chief if involvedDesignate ATS incident Investigator-In-Charge (ATS-IIIC)Designate ATS incident Investigator-In-Charge (ATS-IIIC)Assemble ATS incident investigation teamAssemble ATS incident investigation teamEnsure complete, thorough, objective and timely Ensure complete, thorough, objective and timely investigationinvestigation




– ATS-IIICATS-IIICConduct complete investigationConduct complete investigationForward findings and recommendations to ATS unit Forward findings and recommendations to ATS unit chiefchiefDetermine scope of other ATS unit investigation if Determine scope of other ATS unit investigation if involved and exchange informationinvolved and exchange informationAssign duties to team membersAssign duties to team membersConduct interviews in accordance with national and Conduct interviews in accordance with national and local requirementslocal requirements

– ATS incident investigation teamATS incident investigation teamAssist ATS-IIICAssist ATS-IIICRemain under guidance of ATS-IIIC during investigation Remain under guidance of ATS-IIIC during investigation processprocess




ProceduresProceduresTrainingTrainingSupervisionSupervisionEquipmentEquipmentControl room environmentControl room environmentExternal factorsExternal factorsController actionsController actionsAirspace configurationAirspace configurationTraffic flowsTraffic flowsPilot actionsPilot actions

Consequence of Consequence of ACAS/TCAS eventACAS/TCAS eventMeteorological conditionsMeteorological conditionsControl position Control position configurationconfigurationCoordination proceduresCoordination proceduresAerodrome environmentAerodrome environmentHuman factorsHuman factorsAccuracy of automated Accuracy of automated radar systemsradar systemsRadar dataRadar data

•Investigation process•Fact finding•In-depth inquiry into all causal factors




Investigation process (cont..)Investigation process (cont..)– InterviewsInterviews

Conduct interviews with all possible personnel involved Conduct interviews with all possible personnel involved or who may have knowledgeable of the incidentor who may have knowledgeable of the incident

Conduct interview in atmosphere of shared concernConduct interview in atmosphere of shared concern

Consider national and/or local interview proceduresConsider national and/or local interview procedures

Allow written comments and recommendations from Allow written comments and recommendations from intervieweeinterviewee

ATS-IIIC or investigation team should conduct interviewsATS-IIIC or investigation team should conduct interviews



Chapter 5. Air Traffic Incident Reporting, Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation TeamsInvestigations, and Investigation TeamsInvestigation process (cont.)Investigation process (cont.)– Voice recordingsVoice recordings

Review voice recordingReview voice recordingMake certified copy of original recordingMake certified copy of original recording5 minutes before to 5 minutes after last contact5 minutes before to 5 minutes after last contactInclude track time if availableInclude track time if availableWritten transcript of communications only when neededWritten transcript of communications only when needed

– Radar dataRadar dataRadar data may be used to validate occurrence of ATS Radar data may be used to validate occurrence of ATS incidentincidentWritten procedures should be developed explaining Written procedures should be developed explaining processprocessConsider automation capabilities and accuracy of dataConsider automation capabilities and accuracy of data




Reclassification of ATS incidentReclassification of ATS incident– What if during the investigation data What if during the investigation data

indicates a reclassification is warranted?indicates a reclassification is warranted?– The incident may be reclassified as:The incident may be reclassified as:

Pilot deviationPilot deviation

Other ATS unit deviationOther ATS unit deviation

From operational error to operational deviationFrom operational error to operational deviation

From operational deviation to operational errorFrom operational deviation to operational error

No occurrenceNo occurrence




Return to operational dutiesReturn to operational duties– ATS chiefs should remain involved in the ATS chiefs should remain involved in the

post ATS incident process to ensure post ATS incident process to ensure complete and consistent handlingcomplete and consistent handling

Reasons for suspension of controller’s rating Reasons for suspension of controller’s rating and assignment of proficiency trainingand assignment of proficiency training– Assist and enhance controller’s performanceAssist and enhance controller’s performance– Best of his/her abilitiesBest of his/her abilities– Meet minimum quality standardsMeet minimum quality standards– Improve overall performance and quality of ATSImprove overall performance and quality of ATS




Return to operational duties (cont.)Return to operational duties (cont.)– Consider the following when returning a controller to Consider the following when returning a controller to

operational dutiesoperational dutiesEvents leading up to and surrounding ATS incidentEvents leading up to and surrounding ATS incidentController’s statementController’s statementProcedure or separation minimum involvedProcedure or separation minimum involvedVoice recordingsVoice recordingsController’s training recordController’s training recordController proficiency checksController proficiency checksProficiency trainingProficiency trainingController currency on the control positionController currency on the control positionComputer dataComputer dataInvolvement in previous ATS incidentsInvolvement in previous ATS incidentsResults of interviews and recommendations made by ATS-IIIC Results of interviews and recommendations made by ATS-IIIC and investigation teamand investigation team




Return to operational duties (cont.)Return to operational duties (cont.)Deficiencies in controller’s performanceDeficiencies in controller’s performance

Determine whether to suspend controller’s ratingDetermine whether to suspend controller’s rating

Suspension of rating not based solely on involvement in Suspension of rating not based solely on involvement in ATS incident but based on performanceATS incident but based on performance

Determine appropriate actions and refresher trainingDetermine appropriate actions and refresher training

Complete refresher trainingComplete refresher training

Re-certify controller through controller proficiency Re-certify controller through controller proficiency checkcheck




Final ATS incident reportsFinal ATS incident reports– ATS unit chiefATS unit chief

Analyze data submitted by ATS-IIIC and Analyze data submitted by ATS-IIIC and investigation team in Final ATS Incident investigation team in Final ATS Incident Investigation Report to determine and/or Investigation Report to determine and/or endorse classification of incident andendorse classification of incident and– Categorization of ATS incident (i.e, procedural, Categorization of ATS incident (i.e, procedural,

facility, human factor, etc.)facility, human factor, etc.)– Causal factorsCausal factors– Recommendation and corrective actionsRecommendation and corrective actions

Complete Final ATS Incident Report within 30 Complete Final ATS Incident Report within 30 daysdays




ATS incident tracking processATS incident tracking process– Annual ATS incidents tracking processAnnual ATS incidents tracking process– Will assist CAAs with follow-up investigations and trend analysisWill assist CAAs with follow-up investigations and trend analysis– SampleSample

ATS Incident Report # XXXX-A-00-001ATS Incident Report # XXXX-A-00-001ATS Incident Report # XXXX-T-00-002ATS Incident Report # XXXX-T-00-002ATS Incident Report # XXXX-F-00-003ATS Incident Report # XXXX-F-00-003

Legend:Legend:““XXXX” = ATS unit identificationXXXX” = ATS unit identification““A, T, or F” = Type of ATS unit (A = Area (ACC); T = Approach (APP) andA, T, or F” = Type of ATS unit (A = Area (ACC); T = Approach (APP) andAerodrome Control Tower; F = FIC, AFIS, etc.)Aerodrome Control Tower; F = FIC, AFIS, etc.)““00” = Last two digits of the year00” = Last two digits of the year““001” = ATS incident number in sequence by year for the unit001” = ATS incident number in sequence by year for the unit




Training and proficiency recordsTraining and proficiency records– Document in training and proficiency recordsDocument in training and proficiency records

ATS incident records retentionATS incident records retention– Determine retention periodDetermine retention period– 2 ½ years is recommended2 ½ years is recommended

National ATS authority responsibilitiesNational ATS authority responsibilities– Maintain central source of ATS incident dataMaintain central source of ATS incident data– Review final ATS incident investigation reportsReview final ATS incident investigation reports– Develop and distribute annual ATS incident analysis reportDevelop and distribute annual ATS incident analysis report– Determine effectiveness and efficiency of programmeDetermine effectiveness and efficiency of programme– Maintain liaison with ATS unit chiefsMaintain liaison with ATS unit chiefs– Maintain oversight of ATS incident reportsMaintain oversight of ATS incident reports



Chapter 6. Air Traffic Incident Prevention Chapter 6. Air Traffic Incident Prevention ProgrammesProgrammes

IntroductionIntroduction– Establish policies and programmes that result in quality Establish policies and programmes that result in quality

incident free ATSincident free ATS– Requires commitment of all ATS personnelRequires commitment of all ATS personnel– Consistently perform duties and responsibilities in Consistently perform duties and responsibilities in

accordance with ICAO SARPs, CAA and ATS unit directives accordance with ICAO SARPs, CAA and ATS unit directives and proceduresand procedures

– Accept responsibility for maintaining the safest and highest Accept responsibility for maintaining the safest and highest levels of operational performancelevels of operational performance

– Goals/objectivesGoals/objectivesATS incident free environmentATS incident free environmentSafe, orderly, and expeditious ATS to usersSafe, orderly, and expeditious ATS to usersEveryone is responsible for ATS incident preventionEveryone is responsible for ATS incident preventionKnow, apply, and adhere to requirements in performance of Know, apply, and adhere to requirements in performance of operational duties and responsibilitiesoperational duties and responsibilities




ATS incident preventionATS incident prevention– Many initiatives can be pursuedMany initiatives can be pursued– Five primary areasFive primary areas

CommunicationsCommunications

PhraseologyPhraseology

SupervisionSupervision

TeamworkTeamwork

ATC proficiencyATC proficiency




Identify activities/efforts which Identify activities/efforts which enhance individual, ATS unit, enhance individual, ATS unit, and system performanceand system performanceRecognize commendable Recognize commendable activities/effortsactivities/effortsIdentify deficiencies (individual, Identify deficiencies (individual, equipment, procedural)equipment, procedural)Promptly correct deficienciesPromptly correct deficienciesEnsure actions taken corrected Ensure actions taken corrected the deficiencies or problemsthe deficiencies or problemsInitiatives to ensure effective Initiatives to ensure effective controller performance, controller performance, teamwork, communicationsteamwork, communicationsCommunicate performance Communicate performance expectationsexpectations

Maintain summary of ATS Maintain summary of ATS incidents and incorporate into incidents and incorporate into training training Brief ATS personnel on ATS Brief ATS personnel on ATS incidentsincidentsMonitor/evaluate voice Monitor/evaluate voice recordingsrecordingsReview local/national/regional Review local/national/regional ATS incidentsATS incidentsAnnual QA reviewAnnual QA reviewATS unit staff meetingsATS unit staff meetingsImprove communications Improve communications among all ATS personnel to among all ATS personnel to create atmosphere conducive to create atmosphere conducive to sharing informationsharing informationAttendance to local, national, Attendance to local, national, regional QA seminars, regional QA seminars, conferences, etc.conferences, etc.

Prevention objectives




Communicate performance Communicate performance expectations with expectations with controllerscontrollersTake prompt action when Take prompt action when performance does not meet performance does not meet expectations (on-the-spot expectations (on-the-spot corrections, etc.)corrections, etc.)Address individual/team Address individual/team accountabilityaccountabilityEfficient/consistent Efficient/consistent oversight of ATS unit oversight of ATS unit operationoperationEnsure distractions and Ensure distractions and noise levels in ATS unit are noise levels in ATS unit are kept at a minimumkept at a minimum

Require personnel to Require personnel to maintain high degree of maintain high degree of professionalism, teamwork, professionalism, teamwork, control position discipline, control position discipline, awarenessawarenessPromote open flow of Promote open flow of communicationscommunicationsConduct ATS unit team Conduct ATS unit team meetingsmeetingsPlace emphasis on Place emphasis on hearback/readbackhearback/readbackKeep ATS unit chief/support Keep ATS unit chief/support staff informed (operations staff informed (operations concerns, etc.)concerns, etc.)

ATS supervisors




Random voice recording Random voice recording reviewsreviewsBrief ATS unit on trends and Brief ATS unit on trends and causal factors related to ATS causal factors related to ATS incidents, random recording incidents, random recording reviews, evaluations, etc.reviews, evaluations, etc.Maintain awareness of ATS Maintain awareness of ATS operational environmentoperational environmentProvide staff expertise and Provide staff expertise and supportsupportAssist in ensuring QA goals Assist in ensuring QA goals and objectives are being metand objectives are being met

Ensure scenarios involving Ensure scenarios involving hearback/readback errors hearback/readback errors are included in trainingare included in trainingPrepare/distribute ATS Prepare/distribute ATS incident summaries to all incident summaries to all ATS personnelATS personnelEnsure ATS unit maintains Ensure ATS unit maintains summary of ATS units, summary of ATS units, causal factors, trends, etc.causal factors, trends, etc.Incorporate into classroom Incorporate into classroom and annual proficiency and annual proficiency trainingtraining

ATS quality assurance officers/specialist




Keep ATS supervisors Keep ATS supervisors advised of traffic advised of traffic problems and problems and equipment limitationsequipment limitations

Suggest improvements Suggest improvements to ATS unit and/or to ATS unit and/or incident preventionincident prevention

Maintain situational Maintain situational awarenessawareness

Assist busier control Assist busier control positionspositions

Assist busier control Assist busier control positionspositions

Review own operating Review own operating techniques and ATS techniques and ATS unit procedures to unit procedures to effect highest quality of effect highest quality of performanceperformance

Promptly report ATS Promptly report ATS incidentsincidents

Utilize memory aidsUtilize memory aids

ATC personnel




Voice recording evaluationsVoice recording evaluations– To ensure proper phraseology, good operating To ensure proper phraseology, good operating

practices, adherence to ICAO standards, practices, adherence to ICAO standards, national/local directives and practicesnational/local directives and practices

– GuidelinesGuidelinesConduct semi-annually on all ATS operational personnelConduct semi-annually on all ATS operational personnelReview of voice recording by ATS supervisor/ATS QA Review of voice recording by ATS supervisor/ATS QA specialistspecialistDocument commentsDocument commentsDevelop action plan to address performance deficienciesDevelop action plan to address performance deficienciesATS supervisor/QA specialist review and discuss voice ATS supervisor/QA specialist review and discuss voice recording with controllerrecording with controller




Review of ATC procedures and Review of ATC procedures and practices that may contribute to ATS practices that may contribute to ATS incidentsincidents– Constant review of ATC procedures and Constant review of ATC procedures and

practicespractices– Identify, report, recommend, and Identify, report, recommend, and

implement amendmentsimplement amendments– Ensure safety of aviation and prevention of Ensure safety of aviation and prevention of

ATS incidentsATS incidents




ATS incident review groupsATS incident review groups– Implement ATS incident review groupsImplement ATS incident review groups– National/local levelsNational/local levels– PurposePurpose

Periodic reviews of prior ATS incidentsPeriodic reviews of prior ATS incidentsIdentify other potential problem areas requiring attentionIdentify other potential problem areas requiring attentionEffort to prevent ATS incidentsEffort to prevent ATS incidents

– CompositionCompositionATS unit chiefATS unit chiefControllersControllersOther ATS staffOther ATS staffATS unit QA specialistATS unit QA specialistPilots/airlines when appropriatePilots/airlines when appropriate




Safety information sharing networks and Safety information sharing networks and international organizationsinternational organizations– Several safety information networks availableSeveral safety information networks available– Facilitate free exchange of information of actual Facilitate free exchange of information of actual

and potential safety deficienciesand potential safety deficiencies– Expanding focus to include air traffic control Expanding focus to include air traffic control

servicesservices– GoalsGoals

Collect dataCollect dataShare informationShare informationAssist with identification and prevention of incidents Assist with identification and prevention of incidents and accidentsand accidents




Safety information sharing networks and Safety information sharing networks and international organizations (cont.)international organizations (cont.)– ADREPADREP

ICAO Accident/Incident Reporting Manual, Doc 9156ICAO Accident/Incident Reporting Manual, Doc 9156– Global Aviation Safety Plan (GASP)Global Aviation Safety Plan (GASP)

Endorsed by 32nd Session of the ICAO AssemblyEndorsed by 32nd Session of the ICAO AssemblyObjectivesObjectives– Safety initiatives that offer best “safety dividend”Safety initiatives that offer best “safety dividend”– Annual reviews of accident statisticsAnnual reviews of accident statistics– Decrease in worldwide accident rateDecrease in worldwide accident rate– Enhance identification of short comings and deficienciesEnhance identification of short comings and deficiencies– Increase and improve ICAO’s own capability to compile, Increase and improve ICAO’s own capability to compile,

assess and disseminate safety-related informationassess and disseminate safety-related informationICAO Website: ICAO Website: www.icao.intwww.icao.int

http://www.icao.int/




Global Aviation Information NetworkGlobal Aviation Information Network– To encourage the sharing of aviation safety data worldwide To encourage the sharing of aviation safety data worldwide

in an effort to further reduce accidentsin an effort to further reduce accidents– Plans to expand programme to include air traffic controlPlans to expand programme to include air traffic control– Website: Website: www.gainweb.orgwww.gainweb.org

Aviation Safety Reporting Programme (ASRP)Aviation Safety Reporting Programme (ASRP)– Reporting programme designed to identify unsafe operating Reporting programme designed to identify unsafe operating

conditionsconditions– Deficiencies and discrepanciesDeficiencies and discrepancies– Limited immunityLimited immunity– Voluntary and confidentialVoluntary and confidential

http://www.gainweb.org/




PAN American Aviation Safety Team (PAAST)PAN American Aviation Safety Team (PAAST)– Multi-organizational teamMulti-organizational team– Pool essential resourcesPool essential resources– Addresses priority safety areasAddresses priority safety areas– Deliver practical products to aviation community Deliver practical products to aviation community

in Latin American and Caribbean Regionsin Latin American and Caribbean Regions

Participation in international organization Participation in international organization study groups to analyze and recommend study groups to analyze and recommend solutions to ATS incidents is encouragedsolutions to ATS incidents is encouraged




Runway incursion prevention planRunway incursion prevention plan– Increase in runway incursionsIncrease in runway incursions– Aerodrome operations increasing throughout the Aerodrome operations increasing throughout the

worldworld– Potential for runway incursions increasePotential for runway incursions increase– Need for prevention plan which specifically Need for prevention plan which specifically

addresses runway incursionsaddresses runway incursions– StrategiesStrategies

Position relief briefingsPosition relief briefingsPosition relief checklistPosition relief checklistProficiency training – Prevention of runway incursionsProficiency training – Prevention of runway incursionsFeedbackFeedback




Runway incursion prevention plan (cont.)Runway incursion prevention plan (cont.)– Memory aidsMemory aids

Use of flight progress strips/boardsUse of flight progress strips/boardsRunway in use signRunway in use signReview usage of memory aids on semi-annual basisReview usage of memory aids on semi-annual basis

– ProceduresProceduresReview aerodrome movement operating practicesReview aerodrome movement operating practicesDesignate runway crossing point(s)Designate runway crossing point(s)Place special emphasis on safe runway crossingsPlace special emphasis on safe runway crossingsUse of proper procedures in runway crossings including Use of proper procedures in runway crossings including interphone procedures, runway in use sign by ATS interphone procedures, runway in use sign by ATS supervisors and controllerssupervisors and controllers




– Procedures (cont.)Procedures (cont.)Avoid use of conditional clearances except as Avoid use of conditional clearances except as authorized by ICAO SARPsauthorized by ICAO SARPsCertain conditional clearances are permitted; however, Certain conditional clearances are permitted; however, they should be kept to a minimum, the exception rather they should be kept to a minimum, the exception rather than the norm should be practiced, and under extreme than the norm should be practiced, and under extreme cautioncautionInclude runway check procedures in letter of agreementInclude runway check procedures in letter of agreementDefine vehicular movement procedures/areas in letter of Define vehicular movement procedures/areas in letter of agreementagreementVehicular movements should be restricted to perimeter Vehicular movements should be restricted to perimeter roads, vehicle access roads and other non-movement roads, vehicle access roads and other non-movement areasareasDescribe closed runway proceduresDescribe closed runway procedures




Quality Assurance Review (QAR)Quality Assurance Review (QAR)– Essential to the effectiveness of the ATS Essential to the effectiveness of the ATS

System and prevention of ATS Incidents; System and prevention of ATS Incidents; that all incidents be investigatedthat all incidents be investigated

– QARs provide for identification, QARs provide for identification, investigation and resolutions of these investigation and resolutions of these incidents through corrective training of incidents through corrective training of controller performance deficienciescontroller performance deficiencies




Quality Assurance Review (QAR) Quality Assurance Review (QAR) (cont’d)(cont’d)– Conduct QARs when:Conduct QARs when:

Aircraft accidentsAircraft accidentsOther incidents not categorized as ATS Other incidents not categorized as ATS incidentsincidentsACAS/TCAS eventACAS/TCAS eventOther miscellaneous incident reports that Other miscellaneous incident reports that involve loss of separationsinvolve loss of separationsAircraft go-aroundsAircraft go-aroundsPublic inquiries regarding ATSPublic inquiries regarding ATS




Quality Assurance Review (QAR) Quality Assurance Review (QAR) (cont’d)(cont’d)– Process:Process:

Designate person to conduct QAR’sDesignate person to conduct QAR’sConduct QAR in depth so as to assess Conduct QAR in depth so as to assess controller performancecontroller performanceAccomplish corrective training for all identified Accomplish corrective training for all identified controller performance deficienciescontroller performance deficienciesCommunicate QAR conclusionsCommunicate QAR conclusionsRecord notification and conclusion of QAR in Record notification and conclusion of QAR in ATS unit daily operations logATS unit daily operations log



Chapter 7. ATS EvaluationChapter 7. ATS Evaluation

IntroductionIntroduction– Chapter deals with need for constant and Chapter deals with need for constant and

continuous evaluation of individual ATS continuous evaluation of individual ATS units and of the overall ATS systemunits and of the overall ATS system

Purpose and scopePurpose and scope– Ensure that provision of service is Ensure that provision of service is

maintained at the highest standardmaintained at the highest standard– Units/personnel apply polices, standards, Units/personnel apply polices, standards,

rules, procedures and separation minimum rules, procedures and separation minimum in approved mannerin approved manner




ATS evaluation activities includeATS evaluation activities include– Assessing service provided to usersAssessing service provided to users– Ensuring operating procedures conform with standardsEnsuring operating procedures conform with standards– Assessing operational requirementsAssessing operational requirements– Identifying potential unsafe procedures or operating Identifying potential unsafe procedures or operating

practicespractices– Detecting problem areas or deficienciesDetecting problem areas or deficiencies– Examining effectiveness of intra/inter-unit communications Examining effectiveness of intra/inter-unit communications

and coordinationand coordination– Examining personnel utilization, position workload and Examining personnel utilization, position workload and

establishments to ensure compatibilityestablishments to ensure compatibility– Recommending corrective actionsRecommending corrective actions




Conduct of evaluationConduct of evaluation– Designate team of experienced personnel to Designate team of experienced personnel to

conduct ATS evaluationsconduct ATS evaluations– At least once every two yearsAt least once every two years– Notify ATS unit chief prior to conducting Notify ATS unit chief prior to conducting

evaluationevaluation– Meeting at completion of evaluation with ATS unit Meeting at completion of evaluation with ATS unit

chief to discuss significant findings and chief to discuss significant findings and recommendationsrecommendations

– Prepare written report – ATS evaluation teamPrepare written report – ATS evaluation team– Prepare action plan – ATS unitPrepare action plan – ATS unit




ATS evaluation proceduresATS evaluation proceduresThis section provides standardized procedures for This section provides standardized procedures for evaluating compliance with ICAO SARPs and specified evaluating compliance with ICAO SARPs and specified directives and procedures at the national and local ATS directives and procedures at the national and local ATS unit level within a Stateunit level within a State

– Evaluation processEvaluation processFull unit ATS evaluationFull unit ATS evaluationPreparation and notificationPreparation and notification– Use ATS Unit Evaluation ChecklistUse ATS Unit Evaluation Checklist– Notify ATS unit chief 30 days prior to evaluationNotify ATS unit chief 30 days prior to evaluation– Request data from ATS unit for pre-evaluation reviewRequest data from ATS unit for pre-evaluation review– Solicit special interest items that ATS unit would like Solicit special interest items that ATS unit would like

assessedassessed




– Evaluation process (cont.)Evaluation process (cont.)In-briefingIn-briefing– Introduce team membersIntroduce team members

– Evaluation schedule and activitiesEvaluation schedule and activities

– ATS unit chief staffATS unit chief staff

Conducting the evaluationConducting the evaluation– Direct observationDirect observation

– Control position and/or voice recording or data monitorControl position and/or voice recording or data monitor

– Attendance at staff meetingsAttendance at staff meetings

– Observation of trainingObservation of training

– Review of administrative recordsReview of administrative records

– Interviews/discussionsInterviews/discussions

– Review of in-flight/preflight evaluation reportsReview of in-flight/preflight evaluation reports




– Evaluation process (cont.)Evaluation process (cont.)Daily briefingsDaily briefings– Progress of evaluationProgress of evaluation– Provided by lead ATS unit evaluator to ATS unit chiefProvided by lead ATS unit evaluator to ATS unit chief

Out-briefingOut-briefing– Briefing of ATS evaluator/team’s findingsBriefing of ATS evaluator/team’s findings– AttendanceAttendance

ATS unit chiefATS unit chiefATS unit personnel encouragedATS unit personnel encouraged

– Present draft copy of ATS evaluation reportPresent draft copy of ATS evaluation report

Evaluation critiqueEvaluation critique– Critique form to be completed by ATS unit chiefCritique form to be completed by ATS unit chief




Follow-up evaluationsFollow-up evaluations– Preparation and notificationPreparation and notification

Unannounced or minimum notificationUnannounced or minimum notificationOn-site evaluationOn-site evaluationDesk auditDesk auditCombination of bothCombination of bothConduct no earlier than 6 months after full unit Conduct no earlier than 6 months after full unit ATS evaluationATS evaluationMay request data for pre-evaluation reviewMay request data for pre-evaluation reviewBasically follow same process as full unit ATS Basically follow same process as full unit ATS evaluationevaluation




In-flight evaluationsIn-flight evaluations– Conduct at least 2 per year on each ATS unitConduct at least 2 per year on each ATS unit– Conducted by ATS evaluators and other authorized ATS Conducted by ATS evaluators and other authorized ATS

personnelpersonnel– Use airlines or other operators in which an agreement to Use airlines or other operators in which an agreement to

conduct such observations is in placeconduct such observations is in place– The following air traffic services may be assessed:The following air traffic services may be assessed:

Air traffic services reporting officeAir traffic services reporting officeAutomatic terminal information serviceAutomatic terminal information serviceAerodrome control serviceAerodrome control serviceApproach control serviceApproach control serviceAir traffic advisory serviceAir traffic advisory serviceOther services deemed appropriateOther services deemed appropriate




Special evaluationsSpecial evaluations– May be conducted when deemed necessary by May be conducted when deemed necessary by

the ATS authority or if requested by the ATS unitthe ATS authority or if requested by the ATS unit

Evaluation reportsEvaluation reports– Document results of all ATS evaluationsDocument results of all ATS evaluations– Complete and distribute final ATS evaluation Complete and distribute final ATS evaluation

reports within 30 days of the out-briefingreports within 30 days of the out-briefing– Contents of final ATS evaluation reportsContents of final ATS evaluation reports

Results of evaluation pertaining to areas assessedResults of evaluation pertaining to areas assessedDescribe reportable itemsDescribe reportable itemsAssign tracking control numbers to all items identifiedAssign tracking control numbers to all items identified




Responses to ATS unit evaluationsResponses to ATS unit evaluations– Required on all items rated less than satisfactoryRequired on all items rated less than satisfactory– Three-step closure processThree-step closure process

Action planAction plan– Develop and forward action plans to appropriate ATS Develop and forward action plans to appropriate ATS

authority within 30 days of receiving final ATS unit authority within 30 days of receiving final ATS unit evaluation reportevaluation report

First responseFirst response– Completed and distributed to ATS authority with 60 days of Completed and distributed to ATS authority with 60 days of

ATS unit out-briefingATS unit out-briefing

Second responseSecond response– Completed and distributed with 180 days after date of the Completed and distributed with 180 days after date of the

ATS unit evaluation out-briefing and every 180 days ATS unit evaluation out-briefing and every 180 days thereafter until all items are closedthereafter until all items are closed



Three-step Closure processThree-step Closure process

Three-step closure processThree-step closure process.. The three-step closure process is The three-step closure process is the method by which less than satisfactory items of an the method by which less than satisfactory items of an evaluation are to be closed. The required responses should be evaluation are to be closed. The required responses should be available after 60 days and 180 calendar days and should available after 60 days and 180 calendar days and should describe the three steps as follows:describe the three steps as follows:a)a) Corrective actionCorrective action. The initial action taken by the ATS . The initial action taken by the ATS unit to correct the discrepancy;unit to correct the discrepancy;b)b) Follow-up actionFollow-up action. The action taken over a period of . The action taken over a period of time to validate that the initial action corrected the discrepancy. time to validate that the initial action corrected the discrepancy. Include the date(s) when this was accomplished and the Include the date(s) when this was accomplished and the results; andresults; andc)c) Management controlManagement control. The action taken by the ATS . The action taken by the ATS Authority or ATS Unit that will remain in place to ensure the Authority or ATS Unit that will remain in place to ensure the problem does not reoccur. The action should include problem does not reoccur. The action should include identifying those posts within the ATS unit that have identifying those posts within the ATS unit that have responsibility to periodically review the corrected discrepancy responsibility to periodically review the corrected discrepancy and when the review will be accomplished.and when the review will be accomplished.



Chapter 8. Quality of Services Improvement Chapter 8. Quality of Services Improvement ProgrammesProgrammes

IntroductionIntroduction– Initiatives that can improve the overall quality of Initiatives that can improve the overall quality of

air traffic services being providedair traffic services being provided

Random voice recording reviewsRandom voice recording reviews– ATS unit quality assurance officers/specialists ATS unit quality assurance officers/specialists

should complete several random voice recording should complete several random voice recording reviews on a monthly basisreviews on a monthly basis

Ensure overall quality of ATS is maintainedEnsure overall quality of ATS is maintained




ICAO phraseology improvement programmeICAO phraseology improvement programme– Implement ATS unit phraseology improvement Implement ATS unit phraseology improvement

programmeprogramme– Benefit to pilots and controllersBenefit to pilots and controllers– Results can improve quality of service and Results can improve quality of service and

prevention of ATS incidentsprevention of ATS incidents– MethodMethod

Random voice recording reviewsRandom voice recording reviewsVoice recording monitor evaluationsVoice recording monitor evaluationsDirect observationsDirect observations

– Recognition of ATS personnel who demonstrate Recognition of ATS personnel who demonstrate outstanding phraseology or significant outstanding phraseology or significant improvementimprovement




ATS customer service/feedbackATS customer service/feedback– Important to establish good communications between ATS Important to establish good communications between ATS

authorities/units and usersauthorities/units and users– Users can offer valuable feedbackUsers can offer valuable feedback– Feedback from others organizations/departments/internallyFeedback from others organizations/departments/internally– Obtain feedback through surveysObtain feedback through surveys– ATS quality assurance surveyATS quality assurance survey

Conduct annuallyConduct annuallyExternalExternalInternalInternalAnalyzed and validate data collectedAnalyzed and validate data collectedShare results with ATS personnelShare results with ATS personnelIssues affecting quality of service should be addressedIssues affecting quality of service should be addressed




Pilot/controller user forumsPilot/controller user forums– Conduct annuallyConduct annually– Can produce good relations and enhance communications Can produce good relations and enhance communications

between ATS authorities, pilots and controllersbetween ATS authorities, pilots and controllers– Main objectiveMain objective

Bring together people in the ATS system that work together on Bring together people in the ATS system that work together on a daily basisa daily basis– Pilot on the flight deckPilot on the flight deck– Controller in the control room/towerController in the control room/tower– Better understanding of each other’s responsibilities and dutiesBetter understanding of each other’s responsibilities and duties

– Avoid conducting in a meeting type formatAvoid conducting in a meeting type format– Can also be used to present and explain information on the Can also be used to present and explain information on the

local or national ATS system and procedureslocal or national ATS system and procedures




Participation in pilot safety seminarsParticipation in pilot safety seminars– Participation of ATS authorities in pilot safety seminarsParticipation of ATS authorities in pilot safety seminars– Present information regarding the ATS system, safety and quality Present information regarding the ATS system, safety and quality

assuranceassurancePilot visits to ATS unitsPilot visits to ATS units– Encourage pilot visits to ATS unitsEncourage pilot visits to ATS units– ATS system familiarizationATS system familiarization

ATS system familiarization/education for pilotsATS system familiarization/education for pilots– Consider developing an ATS system education programme for Consider developing an ATS system education programme for

pilotspilots– Education pilots in how to best utilize the ATS systemEducation pilots in how to best utilize the ATS system– FunctionsFunctions– ResponsibilitiesResponsibilities– BenefitsBenefits– ServicesServices




Familiarization training flights for ATS Familiarization training flights for ATS personnelpersonnel– Establish programmes with airlinesEstablish programmes with airlines– ATS supervisors and controllers should be ATS supervisors and controllers should be

encourage to participateencourage to participate– Allows ATS unit personnel to experience Allows ATS unit personnel to experience

first hand the activities on the flight deckfirst hand the activities on the flight deck– View programme as proficiency trainingView programme as proficiency training




Recognition of quality performanceRecognition of quality performance– Recognition of positive quality Recognition of positive quality

performance is as important as identifying performance is as important as identifying deficienciesdeficiencies

– Recognition as individuals or as a teamRecognition as individuals or as a team– Providing a high standard of performance Providing a high standard of performance

and quality of serviceand quality of service




ATS performance measurementsATS performance measurements– Important that ATS providers find ways to continually Important that ATS providers find ways to continually

improve the safety and efficiency of ATS servicesimprove the safety and efficiency of ATS services– Factors when measuring performance and quality of ATS Factors when measuring performance and quality of ATS

services:services:SafetySafetyDelayDelayPredictabilityPredictabilityFlexibilityFlexibilityEfficiencyEfficiencyAvailabilityAvailabilityAccessAccessCost of ServiceCost of ServiceRunway CapacityRunway Capacity

THANK YOUTHE END

CAR/SAM Regional Guidance Material on Air Traffic Services Quality Assurance Programmes NAM/CAR/SAM...

Documents

Transcript of CAR/SAM Regional Guidance Material on Air Traffic Services Quality Assurance Programmes NAM/CAR/SAM...