IT Audit powerpoint

1

IT Auditing AC 475

Session 10Bentley University

18 April 2013

© John W. Beveridge 2

Session 10 Agenda

1. Revisiting Closing the Loop Framework

2. Measuring IT Audit Performance3. Application System Audit Planning

REVISITING THE CLOSING THE LOOP FRAMEWORK

Section 1


© John Beveridge

Revisiting Closing the Loop Framework

Why revisit the CTL Framework? What is the relationship of the Framework to the

Team Project? How applicable is the CTL Framework to audits

other than IT audits? Where do IT auditing standards, quality assurance,

and audit risk come into play?

© John Beveridge

Revisiting Closing the Loop Framework

CTL Framework provides: structured approach for developing an audit work program

for internal control examinations. reinforces the value gained by closely linking control

objectives to business objectives, and controls to control objectives

promotes understanding of the benefit of controls and having an appropriate mix of controls

Strengthens audit work programs by distinguishing between “controls in place” and “controls in effect”

Helps in drawing conclusions and developing audit results

© John Beveridge

Closing the Loop Framework

CTL is a methodology to: Define audit objectives and audit criteria in relation to

control objectives and control practices, Develop targeted audit steps to meet audit evidence

requirements, Develop references, or work papers, to help draw

conclusions in line with control objectives and report audit results.

CTL Framework – Forward & Back

The idea of Closing the Loop is to tie in what we learn at each step in the process and to be able to link that information backwards and forwards.

By “Look Back”, we are referring back to the prior step as a point of reference and basis for what we do in the current step.

Closing the

Loop

F. Develop Audit Results

A. Define Control Objectives

B. Identifying Control Criteria

C. Develop Audit Objectives

E. Build Audit Steps

D. Define Audit Criteria

Framework Outline


A. Identifying operational and control objectives

B. Identifying and classifying control criteriaC. Developing audit objectivesD. Defining audit criteria in terms of

evidence requirements E. Building audit steps F. Developing audit results

Framework Outline


A. Identifying operational and control objectives

B. Identifying and classifying control criteriaC. Developing audit objectivesD. Defining audit criteria in terms of

evidence requirements E. Building audit steps F. Developing audit results

Au

dit

Pla

nn

ing

© John Beveridge

A. Defining

Control Objectives






Closing the

Loop


© John Beveridge

A. Defining the Control Objectives Internal Control

Controls are framed by what is to be attained (control objectives) and the means to attain those goals (the controls).

© John Beveridge

Control (as defined by COBIT)

The policies, procedures, practices and organizational structures designed to provide reasonable assurance that business objectives will be achieved and that undesired events will be prevented or detected and corrected.

A. Defining the Control Objectives IT Control Objective

A statement of desired result or purpose to be achieved by implementing control procedures in a particular IT activity

A. Defining the Control Objectives Identifying Control Objectives

Identification of relevant operational and control objectives: First, what needs to be achieved and

avoided. Based on importance and impact of risk

triage what the control requirements Then, select the control objectives

© John Beveridge

The auditor’s understanding of: what is important to the business, its

customers, and oversight bodies, and the reasons for why IT needs to be controlled

helps one focus on the Control Objectives and then the controls needed

Understanding Control Requirements

© John Beveridge

A. Defining the Control Objectives Understanding the Control Environment

Business organization, information systems, and supporting technology

Documenting the business operations (internal and external, CSFs, IT environment)

Identifying the key operational and control objectives

Assessing entity’s ethical climate – tone at the top Identifying and evaluating the appropriateness of

internal controls

To Achieve Business Objectives

To Avoid Risks, Threats and Exposures

Mission Statement

Business Objectives

Interviews with Managers

Business White Paper

Risk Assessment

Risk Management Reports

Incident Reports

Security Reports

Identifying Control

Objectives

© John Beveridge

A. Defining the Control Objectives Identify Important Information

Attributes

For the Data and Information, identify: Nature and type of information Business/process requirements of information

1. Identify reliability requirements2. Identify relevance requirements3. Security – (access, change, privacy)4. Accessibility5. Availability

© John Beveridge

Defining the Control Objectives

Select the control objectives that are most important in terms of what the organization needs to achieve and avoid.

Define and articulate the control objectives for clear understanding by auditee and auditor

© John Beveridge


Building Audit Criteria






Closing the

Loop


© John Beveridge

B. Identifying Control Criteria“Look Back”

Look Back to Control Objectives for our primary point of reference.

Controls are the policies, procedures, organizational structures, and mechanisms that help ensure that the control objectives are met.

© John Beveridge

Controls Link to Control Objectives

Control

Objective

Controls

Control Evidence

For Defined Control Objectives, Where Do We Go to Find the Controls?

Researching controls Start with researching the control

objectives or purpose to be achieved

Suggest a two-tier approach to follow

1. Obtain control lists from control models, control guidelines, standards, etc

2. Research individual controls through web-based and reference material

© John Beveridge

B. Identifying Control Criteria Complete the Control Classification

Prepare list of controls (with sources) for each control objective

Identify for each control the control category and control type

Populate table with control classification Identifying category and type of control is

the first step to ensuring the mix of controls will be sufficiently comprehensive

© John Beveridge


Control Classification

Policies (rules of the road) Procedures (“How To” do tasks and activities) Organizational (structure, span of control, unity of

command, segregation of duties, job descriptions, job responsibilities, points of accountability)

Practice (methods) Mechanisms (software tools, programmed

procedures, etc.) Documentation (audit trails, systems of record) Legal (statutory, regulatory, and contractual) Management Directives

B. Identifying Control Criteria Classify by Types of Controls

General controls Application controls Primary controls Secondary controls Preventive controls Detective Controls Corrective Controls Compensating controls

© John Beveridge

General Controls

General controls are controls which are over and within the entire data processing or IT environment, impacting all or most application systems. They are pervasive Reflect the control culture Management sensitive

© John Beveridge

Application Controls

Application controls are specific to individual application systems Primarily applied to input, processing, and output. Include data preparation controls, edit checks,

reasonableness limits, processing controls, restart and recovery, backup, and output distribution controls.

© John Beveridge


Control Benefit or Failure for Each Control

Our focus is now on “Control Value” What is the benefit derived from the

control? In place? In effect?

What is the impact when the control is not in place, or in effect?

Impact of control(s) not in effect

Control Objective

Control Criteria

Impact of control(s) not in place

Adverse Impact of Absence of Control

Control Objective: Physical Security

Control ControlCategory

Type of Control

Control Benefits

Adverse Impact of Control not in Place/Effect

Lock Mechanism PrimaryPreventive

Prevent access

Unauthorized access gained

Monitor, Camera

Mechanism SecondaryDetective

Unauthorized access gained

Guard(s) in lobby

Organizational

SecondaryPreventiveDetectiveCorrective


Table for Control Classification






Closing the

Loop


© John Beveridge


Audit Objectives

Depends on the type of audit Relate the audit objective to the control

objective. Link the audit objectives and audit procedures

to the control objectives and the controls (review and examination steps) to obtain sufficient audit evidence to draw conclusions

© John Beveridge


Audit Objectives

For control examinations, best phrased when focused on selected control objectives.

Use standard language to phrase the audit objective in relation to the control objective

“Determine whether adequate controls are in effect to provide reasonable assurance that the” Control Objective “will be met.”

© John Beveridge


Example Audit Objective

If the control objective is to ensure that all changes are authorized and tested.

Therefore, the audit objective would be “To determine whether adequate controls were in effect to provide reasonable assurance that all changes are authorized and tested.”






Closing the

Loop


© John Beveridge


Developing Audit Criteria in Terms of Evidence Requirements

defining evidence requirements for controls “in place”

defining evidence requirements for controls “in effect”

RISKS

To Achieve To AvoidBusiness Objectives

Controls

Control in Place

Control in Effect

ProcessAudit

© John Beveridge

To Achieve To AvoidBusiness Objectives

Controls

Control in Place

Control in Effect ControlAudit

RISKS

Control Objective: Safeguarding IT Resources

Control Evidence that Control is in Place

Evidence that Control is in Effect

Sign-out policy for notebook computers

Documented policy that all notebook computers are to be signed out.Policy is readily available in hardcopy and online

Understanding of policy by management and staffRegister maintained of all signed out notebooksDocumentation of signatures of parties to whom notebooks have been assignedReconciliation of sign-out documentation to IT resource inventory


Control Objective

Control Practices

Evidence Requirements

Audit Objective

Audit Strategy

Audit Procedures

Criteria

E. Building Audit Steps






Closing the Loop


© John Beveridge


Developing Audit Strategy

Determine exactly what evidence that you need to draw conclusions (prior CTL step really helps)

Identify and assess the reliability of the sources of audit evidence (from people to IT)

Objective: to develop, or review and amend if necessary, an audit plan to accomplish the audit objectives

© John Beveridge


Development of Audit Work Program

For IT Control Examinations, need to: define control objective and identify relevant

controls identify CSFs and Risk Factors identify control characteristics including desired

evidence establish audit objective(s) based on control

objective(s) develop audit procedures to capture and analyze

“evidence”

© John Beveridge


Development of Audit Work Program

The presence, or absence, of control evidence becomes our audit evidence

Need sufficient, competent audit evidence to serve as a basis for drawing audit conclusions and forming an opinion


Focus on Audit Evidence and Steps to Obtain and Analyze it

Determines:

Skills and knowledge requirements for the audit team

Whether technical assistance is needed Whether software tools are needed Impact on audit schedule

© John Beveridge


Audit Procedures

Audit steps should cover the full combination of controls (appropriate mix of categories and types)

Should include steps to assess the presence and effectiveness of assurance mechanisms

F. Developing Audit Results






Closing the

Loop


© John Beveridge


Control Conclusions

Review the detailed results by control area and audit objectives.

Review the audit result tables to identify which controls were found to be in place and effect, and which were not.

Generate control strengths and weaknesses list by control area.

Draw conclusions by audit area.

© John Beveridge


Control Assessment What is the control objective? What business objective is impacted? Appropriateness of the stated control? Identify the type of control (application or general;

primary or secondary; and preventive, detective, or corrective)

Number of components used to execute the control and number of subsystems or control objectives impacted?

Evidence that the control is in effect, or impact that it is not.

Benefit of control(s) in effect

Controls

Audit StepsImpact of control(s)

not in place

Evidence: control in

place

Evidence: control in

effect

Audit Steps

Benefit of control(s) in place

Impact of control(s) not in effect

© John Beveridge


Forming Conclusions and Opinions

Is the combination of controls in place and in effect to the degree determined to be sufficient to provide “reasonable assurance” that the control objective will be met?

Auditor’s judgment is paramount

MEASURING IT AUDIT PERFORMANCE

Section 2


IT Audit Performance

How well is IT Audit doing?Is Is IT Audit doing the Right Things?

Is IT Audit Effective?Are resources used Effectively and

Efficiently?

Performance Measurement

Process of quantifying the efficiency and effectiveness of past action

Metrics are specific representations of a capacity, process, or outcome relevant to performance assessment

A metric or performance measure should be quantifiable and be documented

Criteria for Effective Performance Measures

Is the measure relevant? Is the measure clearly defined? Is the data easy to obtain? Is there a tracking/reporting system? If yes, is it easy to access and use? Can reasonable control/influence be

exercised over performance related to the measure?

Criteria for Effective Performance Measures

Does the measure accurately reflect what is happening in the audit process?

Does the measure communicate how we are doing?

Does the measure allow one to demonstrate progress?

Is the measure useful to whoever can act on element being measured to improve performance?


What should be subject to performance measurement? Audit quality AIC and staff performance Report content and clarity Workpapers Audit Risk Compliance with auditing standards Impact to the organization

What else?


Cornerstone to effective management of audit assignments

Relevance and Reliability of Metrics Reliability of data related to time and

resources On-line audit management systems Work papers Use of automated workpapers (TeamMate) Time and attendance


Data that is Usually Recorded

Start date to end field date End of field date to informal exit date Informal exit to formal exit date Formal exit to report issuance date Estimated completion date Actual completion date Number of staff (and who) Number of audit topic areas

continued


Data that is Usually Recorded (2) Number of findings Number of Audit Follow-up Areas Audit name and audit number Total number of field days Number of Site Visits by Support Manager Number of Site Visits by Manager Date that Staff End-of-Job Evaluations are Completed and

Filed with HRD Date that all Completed (Reviewed and Signed Off)

Hardcopy Work Papers are Filed


What do Metrics Tell Us?

Total Field Days Range Average Field Days

Number of Audit Topics in Scope Range of Topics: 2 to 10 topics, etc. Average Number of Topics: 6.79



Number of Days from end of Field Work to Issuance of the Report (elapsed time) Range Average Elapsed Days Adjusted for Work Days

Number of Audit Findings Range of Topics: Average Number of Topics:



Number of Field Days per Audit Topic Range Average Days

What other statistics would tell us the effectiveness and efficiency of each audit?


Develop set of reliable and relevant metrics and targets

Establish performance measurement baseline Evaluate in order to manage performance Objective: better understand audit

performance and improve it

Example Administrative Items

IT Audit Survey Results Form Control Analysis Form Control Evidence Sheet Independence Certification Form Report Preparation Checklist Report Changes Tracking Form Audit Topic Areas Table IT Audit Administrative Review Form

APPLICATION SYSTEM AUDIT PLANNING

Section 3


Application System Audits

Application System Audits are one of the fundamental types of IT audits.

The other types of IT audits are general control examinations (used to be called facility audits), system under development audits, and IT technical audits.

Application audits, like other types of IT audits, can include performance and compliance audit work.

Application Controls

Application controls may be: Administrative or technical Manual or programmed

They need to address the integrity, security, maintainability, and availability of the system.

Application System Audit Objectives

May be focused on the reliability, security and availability of the entire system

May be tailored to specific functional areas or operational aspects of the application Example: billing module, data input, change control,

protection of PII, etc.


Before we gain and record an understanding of the system, we need to understand the audit entity in terms of: Mission and business objectives Business processes and data and information

requirements Legal mandates (law, regulation and contract) Physical and functional organization Entity’s control environment IT infrastructure

Is the Business Process subject to Change? Are changes in business strategy occurring, or about

to occur? Is there an effort underway, or planned, to enhance

value chain? Is there a business process engineering effort

underway, or planned?

Application System Changes?

Application System Audit Planning

To understand the need for the system, start with understanding the business and its customers.

Understand the macro and micro business environment within which the system operates. Gain and record an understanding system functions Perform risk assessment Brainstorm – fraud issues Obtain and review stated controls

Develop proposed scope, objectives and audit strategy


As with CTL, start with the operational and control objectives. What should the system do – what is its purpose? Address business process support and functional

attributes (integrity, security and availability) As with controls (under CTL) identify functional value

and adverse impact of functional failure. What is the evidence of functional success and failure?

Understanding the System

Understanding the system: What is the name of the application? Where does it reside? Applications do not just sit on

mainframes. With what systems does it interface? Is it a feeder system? When and by whom was the system developed? How long has the system been in production? Legal requirements (regulatory – contractual)?

Understanding the System (2)

Understanding the system Define process boundaries Identify mission-critical functions Assess data sensitivity, integrity and availability

requirements Identify significant types of transactions Identify the level of documentation Identify the required IT infrastructure needed to

support the operation of the system?

Identifying “Who” is Involved and Impacted by the system

View this from a RACI chart perspective “Who” needs to be informed regarding the system? Who needs to be consulted regarding the system? Who is responsible for the integrity, operation, security,

maintenance and availability of the system? Who is accountable for the system?

Who relies on the system? Who monitors and evaluates the system? Who measures the performance of the system? Who are the primary and secondary users?

Compliance with Laws and Regulations Determine whether there are external

compliance requirements: Identify external requirements Document pertinent laws and regulations Assess whether management and the IT function have

considered the relevant external requirements and the actions taken (policies, procedures, training, etc.)

Review business department documents that address adherence to applicable laws and regulations

Application Documentation

User manuals Functional specs Detailed specs DFDs Flowcharts Data dictionaries System narratives Policy and procedures Source documents Screen formats System reports


QuestionsQuestions

Questions

IT Audit powerpoint

Documents

Transcript of IT Audit powerpoint