Federal Aviation Administration The United States Government’s USOAP Experience By: Michele...

Federal AviationAdministration

The United States Government’s USOAP Experience

By: Michele Cappelle, National USOAP Coordinator

Date: December 15, 2007

2Federal AviationAdministration

The United States Government’s USOAP experience

December 12, 2007

Safety oversight audit process

Location

Time Frame

Activities

Montreal

Pre-audit

Sec Gen letter to State

12 months

SPO handover to TL

Team meeting

Update SAAQ & CC

Post-audit

Montreal

Final report

9 months+

days

On-siteAudit

Opening meeting

Closing meeting



December 12, 2007

04/19/23 3

Preparation Recommendations

• Select a Coordinator with strong team building skills and knowledge of the State’s organizational and legal structure

• Ensure the Coordinator takes the ICAO Regional Audit Preparation Seminar/Workshop– Outlines USOAP process and ICAO expectations– Opportunity to meet others who have been audited



December 12, 2007

04/19/23 4


• Build a strong inter-agency working group to respond to each of the audit areas– Include all State entities involved with safety

oversight as well as representatives from the appropriate offices in the CAA

– Ensure the legal offices are involved in the coordination and development of State responses



December 12, 2007

5


• Complete the compliance checklists and the State Activity Questionnaire in a timely manner– Offer clear, accurate and concise explanations– Answers should represent the “official” government

position– There are a lot of overlapping and repetitious

questions in the State Activity Questionnaire and in the Protocols. Ensure that the answers are compatible



December 12, 2007

04/19/23 6


• Develop answers to the audit protocols questions in advance of the audit– For internal use– Coordinate answers– Identify and organize evidence to support answer – Identify the subject-matter-expert for each question– Identify the appropriate person to respond to the

auditor for each question



December 12, 2007

04/19/23 7


• Brief high-level government officials as well as the CAA staff– Ensure they understand the importance of the audit– Explain the USOAP comprehensive systems

approach– Prepare them for the extensive resource demands

for the pre-audit activities, on-site audit activities and post-audit activities



December 12, 2007

04/19/23 8


• Brief industry representatives for ICAO industry visits– Explain USOAP process and purpose of the visit



December 12, 2007

04/19/23 9


• Learn to use and navigate audit tools on the SOA website

• Learn how ICAO determines a State’s level of compliance with the critical elements– Bias toward less complex civil aviation oversight

systems– Ensure high-level officials understand how this

number is developed



December 12, 2007

04/19/23 10


• Review audit results of other States– Focus on audits which were conducted by the same

auditors who will audit your State– Focus on audits of the most comparable oversight

systems to your State– Become familiar with typical findings and begin to

evaluate whether they would be applicable to your State as well



December 12, 2007

04/19/23 11


• Conduct practice audits for each audit area– Use people who have served as ICAO USOAP

auditors of other countries– Use compliance inspectors and train them on the

conduct of ICAO USOAP audits– Identify officials who will meet with the auditors and

have them participate in the practice audits– Identify weaknesses – Ensure internal auditors are familiar with other

State’s audit findings



December 12, 2007

04/19/23 12


• Manage expectations by developing a credible list of potential audit findings – Ensure that the management team is aware of the

potential findings– Avoid findings by correcting weaknesses where

possible



December 12, 2007

04/19/23 13

Lessons Learned

• The State’s USOAP team is critical to the success of the audit

• Strong internal coordination is essential

• Develop good communication w/ USOAP team



December 12, 2007

04/19/23 14

Lessons Learned

• Develop a detailed audit workplan – ensure each auditor is scheduled to meet with the

appropriate officials for answers to all protocol questions. This will avoid last-minute surprises and unprepared or unavailable interviewees

• Insist on organized daily debriefs– Support a clear understanding by all participants– Prevent the development of inaccurate audit findings– Helps manage the IOUs



December 12, 2007

04/19/23 15

Lessons Learned

• Use all the resources available to understand and prepare for the audit– ICAO SOA website, USOAP team leader, State auditing

expertise, audit results of other states

• Develop a strong internal communication strategy– Widely disseminate information concerning the USOAP– Brief important officials

• Ensure they are familiar with potential audit findings



December 12, 2007

04/19/23 16

Lessons Learned

• Train interviewees – On the USOAP process and expected attitude– To stick to the subject– To only answer questions which are in his/her

purview – On the limits of their authority to respond to ICAO on

behalf of the State



December 12, 2007

04/19/23 17

Lessons Learned• Test all components of the audit for

understanding and effectiveness– Are the compliance checklists complete and accurate– Do the State’s answers to the State Activity

Questionnaire completely reflect the State’s official government response to each issue?

– Are the protocol answers fully coordinated and supported by evidence?

– Are the interviewees conversant with the protocol answers and evidence?

– Do you know with whom the auditors need to meet and are those appointments scheduled?



December 12, 2007

04/19/23 18

U.S. Government USOAP Point of Contact

Michele Cappelle

National USOAP Coordinator

Tel: 202-385-8887

Email: [email protected]

Federal Aviation Administration The United States Government’s USOAP Experience By: Michele...

Documents

Transcript of Federal Aviation Administration The United States Government’s USOAP Experience By: Michele...