IT Audit Benchmarking Study

8/9/2019 IT Audit Benchmarking Study

1/82

2009 IT AUDIT BENCHMARKING STUDY Executive Summary and Report

March 2009

Version 1.2


2/82

i

DISCLAIMER

Copyright © 2009 by The Institute of Internal Auditors’ (IIA’s) and The IIA Research Foundation’s (IIARF’s)Global Audit Information Network (GAIN) located at 247 Maitland Avenue, Altamonte Springs, Fla. 32701. All rights reserved. Published in the United States of America.

Except for the purposes intended by this publication, readers of this document may not reproduce, redistribute,display, rent, lend, resell, commercially exploit, or adapt the statistical and other data contained herein withoutthe permission of GAIN or The IIARF.

The information included in this document is general in nature and is not intended to address any particularindividual, internal audit activity, or organization. Based on the date of issuance and changing environments, noindividual, internal audit activity, or organization should act on the information provided in this document withoutappropriate consultation or examination.

ABOUT THIS REPORT

As part of its services, The Institute of Internal Auditors’ (IIA’s) and IIA Research Foundation’s (IIARF’s) Global Audit Information Network (GAIN) will develop a series of benchmarking studies on specific subjects internalauditors can use to share, compare, and validate their work and specific business practices. This reportprovides a summary of key study findings and recommendations from IIA members to help those lookingto establish an effective IT audit process and acquire technology-based audit tools to maximize their internalaudit efforts.

This study is not a statistically based survey and its results are not representative of the entire population ofinternal auditors. Rather, this is a benchmarking study based on the responses of chief audit executives andother internal audit professionals who are members of GAIN and it is solely intended to provide information(i.e., tools, resources, and/or other knowledge) that is based on the responses of survey participants.

ACKNOWLEDGEMENTS

The IIA would like to thank Don Sparks, CIA, CISA, ARM, and Cesar Martinez, CIA, CGAP, for theircontributions in developing the 2009 IT Audit Benchmarking Study from which this Executive Summaryand Report is based.


3/82


4/82

1

EXECUTIVE SUMMARY

According to IIA Standard 1210.A3: Proficiency, internal auditors must have sufficient knowledge of key IT risksand controls and available technology-based audit techniques to perform their work. As many long-time users oftechnology-based audit techniques know, having the right application can expedite and maximize internal auditefforts significantly. However, whether an in-house or third-party tool is used, it is important that organizations

incorporate IT audit activities as part of the internal audit plan.

To determine the extent of IT audit planning efforts, the profile of IT audit functions, and the software toolscurrently in use, The IIA’s and IIARF’s GAIN department conducted its first annual IT Audit Benchmarking Studyin February 2009. Of the 138 organizations represented in the study, an overwhelming majority — 94.8 percent — incorporate IT audit activities as part of the internal audit plan. When asked to explain theprocess used to incorporate IT audit activities into the audit plan, 52.9 percent use an integrated planningapproach in which potential IT audit areas are determined as part of the risk assessment process or annualaudit planning process. In addition, many of these organizations use software to support extraction, dataanalysis, and risk assessment efforts, among other activities.

In terms of years of IT audit experience, respondents stated they have an average of 2.9 years of expertise inthis area. In addition, most IT audit functions consisted of 1 –3 internal auditors dedicated solely to this task and

25 percent of participants indicated their internal audit function has been performing IT audits for 1 –3 years. Additionally, the vast majority (83.2 percent) of respondents indicated the IT audit function reports to the CAEor head of internal auditing followed by the audit committee (8.8 percent).

The study also asked respondents to specify whether their organization co-sources, outsources, or both co-sources and outsources any of its IT audit activities. More than half of study participants (52.2 percent) statedthey performed none of these three activities — that is, IT audit activities are performed solely by the internalaudit group. Of the remaining responses, 23.9 percent both co-source and outsource their IT audit activities,followed by 17.4 percent that co-source only and 6.5 percent that outsource only. The top five reasons whyIT audit activities are either co-sourced or outsourced include having better access to subject-matter experts(79.5 percent), internal audit staff limitations (75 percent), cost-effectiveness of the co-sourcing or outsourcingactivity (43.2 percent), lack of internal audit staff knowledge on the IT systems used in the organization(36.4 percent), and difficulty in recruiting qualified IT audit staff (22.7 percent).

Furthermore, respondents were asked to list the top three issues that will impact IT audits the most within thenext 24 months. These three issues include IT audit project limitations due to budget restrictions, lack of internalresources or time, increasing travel costs, and lack of overall knowledge to perform an IT audit (43.5 percent);data security and privacy (37.7 percent); and being unable to add value to the organization due to the increasingcomplexity of IT systems (23.2 percent). Based on these responses, study participants were asked if they hadthe skills and training to address the issues that will impact IT audits the most. The vast majority of participantsresponded “yes” to both questions — 71.7 percent and 72.2 percent of internal audit activities represented in thestudy have the skills and training, respectively, to address the issues that will impact IT audits the most withinthe next 24 months.

Similarly, participants were asked to identify the latest three technology innovations that have eased theperformance of IT audits the most within the last three years. These include use of computer assisted audittechniques (CAATs), availability of many systems online, and guidance on specific IT audit areas or guidance

that is tailored to noncomplex IT environments. In terms of training, the primary source of IT audit knowledgeduring the past 24 months is participation in seminars, workshops, and conferences offered by a professionalorganization (44 percent), followed by individual research gathered from online resources, and books or self-study courses.

http://www.theiia.org/guidance/standards-and-guidance/ippf/standards/standards-items/?C=3093&i=8247http://www.theiia.org/guidance/standards-and-guidance/ippf/standards/standards-items/?C=3093&i=8247http://www.theiia.org/guidance/standards-and-guidance/ippf/standards/standards-items/?C=3093&i=8247http://www.theiia.org/guidance/standards-and-guidance/ippf/standards/standards-items/?C=3093&i=8247


5/82


6/82

3

Besides working in small to mid-size internal audit departments, the low number of IT auditors could becorrelated to how long this function has been present within the organization. According to study results,25 percent of participants indicated their internal audit function has been performing IT audits for 1 –3 yearsfollowed by 4 –6 years (19.9 percent). These percentages continue to decrease until they hit the ―19 yearsor more‖ category, where they increase back to 19.9 percent (refer to figure 4 for a breakdown).

Figure 4: Years internal audit function has been performing IT audits

Finally, respondents were asked to determine to whom the IT audit function reports. The majority (83.2 percent)indicated the IT audit function reports to the CAE or head of internal auditing followed by the audit committee(8.8 percent). Figure 5 summarizes these responses.

Figure 5: IT audit function reporting line


7/82

4

THE IT AUDIT ACTIVITY

To determine how IT audits are planned throughout the year, study participants were asked whether theyincorporate IT audit activities as part of the internal audit plan. Nearly all respondents (94.8 percent) indicatedthey incorporate IT audit activities during the internal audit planning process while only 5.2 percent indicatedthey did not. A cross-tabulation of the data was performed to determine whether IT audit and internal audit staff

sizes were correlated to the incorporation of IT audit activities as part of the audit plan.

The assumption was made that the larger the size of the internal audit or IT audit activity, the more likely theorganization would be not to incorporate IT audit activities as part of the internal audit plan due to the presenceof a dedicated group of IT auditors. However, this was not case. Of the 94.8 percent of organizations thatincorporate IT audit activities as part of the internal audit plan, 81 percent have an IT audit function consisting of1 –3 IT auditors and 37.1 percent have an internal audit activity ranging from 3 –6 full-time internal auditors.

3

When asked to explain the process to incorporate IT audit activities into the internal audit plan, 52.9 percent usean integrated planning approach in which potential IT audit areas are determined as part of the risk assessmentprocess or annual audit planning process performed to determine all audit universe components. Only a feworganizations perform a separate IT audit risk assessment to identify the IT audit areas to be audited throughoutthe year (10.9 percent) or identify IT auditable components based on core business functions or processes

(5.8 percent) (refer to table 1 for a summary of these responses).

Gary Allen, CIA, CISA, IT audit manager for Berkshire Life Insurance Company of America, in Pittsfield, Mass.,is one of the respondents using a risk-based, integrated planning approach that incorporates IT auditcomponents into the annual audit plan. The entire internal audit staff participates in system developmentprojects as a way to help the organization mitigate risks in addition to performing transactional and operationalaudits. ―If the sales department is designing and building a new system, for instance, our team would devoteaudit resources to the project to give advice on designing controls in the new system,‖ he explains. ―This way,our auditors help get the process right to begin with and it is less expensive than coming in later and pointing outall the controls that got left out of the new process or system.‖

4

Process Used to Incorporate IT Audit Activities as Part of the Internal Audit Plan

The in ternal audit activity takes an integrated IT audit plann ing approach in which potential IT audit areas are

determined as part of the risk assessment process or annual audit planning process performed to determine all audit universecomponents. Once an IT audit universe is determined based on areas of high risk, a schedule is created to monitor or reviewIT audit universe components on a specific timeframe. These IT audit universe components are either incorporated into the

annual audit plan or kept as a separate IT audit plan. For example,

A universe of IT audits is created as part of the normal audit planning process, in which IT audit areas are

risk-ranked.

The highest risk-ranked audits are included in the overall audit plan to the extent that the internal audit

department has the IT resources to allocate to them.

Risk assessment interviews are also performed, including interviews with IT management.

IT components that are ranked for risk include system applications, and operations, access, and change

management controls (73 responses).

The internal audi t activity performs a separate IT audi t ri sk assessment to identify the IT audit areas to be auditedthroughout the year. These areas are added to the overall annual audit plan (15 responses).

I T audi ts are determined based on core business functions and processes (8 responses).Table 1: Process used to incorporate IT audit activities into the internal audit planning process

3 These percentages represent the choices with the highest number of responses.4 According to The IIA’s International Standards for the Professional Practice of Internal Auditing (refer to all Standards with the letter“C” for those pertaining to consulting activities), internal auditors can act in a consulting capacity as long as doing so does not hinder theinternal auditor’s independence and objectivity to later assess the effectiveness of the same activity. For instance, as long as auditors only

provide advice regarding the design of controls and no help is given in the actual development of the detailed controls, this consulting

activity should not breach any independence issue. If, however, the internal auditor were to develop the con trols that form part of thesystem, then as stated by the Standards, he or she could not audit that particular area for a period of at least 12 months.

http://www.theiia.org/guidance/standards-and-guidance/ippf/standards/full-standards/http://www.theiia.org/guidance/standards-and-guidance/ippf/standards/full-standards/http://www.theiia.org/guidance/standards-and-guidance/ippf/standards/full-standards/http://www.theiia.org/guidance/standards-and-guidance/ippf/standards/full-standards/


8/82

5

Respondents indicating they did not incorporate IT auditactivities as part of the internal audit plan were alsoasked to elaborate on their response. Reasons why ITaudit activities are not incorporated as part of the internalaudit plan include:

The internal audit activity does not have theskills or financial resources necessary to performIT audits.IT management does not provide the informationnecessary for the internal audit group to reviewIT activities and processes.IT audits are outsourced.

Study questions also asked respondents to specifywhether their organization co-sources, outsources,or both co-sources and outsources any of its IT auditactivities. More than half of study participants(52.2 percent) stated they performed none of these three

activities — that is, IT audit activities are performedsolely by the audit group. Of the remaining responses,23.9 percent both co-source and outsource their IT auditactivities, followed by 17.4 percent that co-source ITaudit efforts only and 6.5 percent that outsource them.

Again, a cross-tabulation of the data was performed todetermine whether IT audit and internal audit staff sizeswere correlated to the co-sourcing or outsourcing of ITaudit activities. The assumption was made that thesmaller the size of the internal audit activity or IT auditfunction, the more likely the organization would be to co-source, outsource, or both co-source and outsource theirIT audit activities. This was somewhat the case: 5

Of the 17.4 percent of internal audit departmentsrepresented in the study that co-source their ITaudit activities, 13.2 percent and 6.5 percenthave 1 –3 IT auditors and 3 –6 full-time internalauditors, respectively.Of the 6.5 percent of audit departments thatoutsource their IT audit activities, 4.4 percentand 5.4 percent have 1 –3 IT auditors and 3 –6full-time internal auditors, respectively.Of the 23.9 percent of audit departments thatboth co-source and outsource their IT auditactivities, 23.1 percent and 9.8 percent have

1 –3 IT auditors and 7 –15 full-time internalauditors, respectively.Of the 52.2 percent of audit departments that perform their own IT audits, 44 percent and19.6 percent have 1 –3 IT auditors and 3 –6full-time internal auditors, respectively.

5 Percentages listed in the bulleted list represent the choices with the highest number of responses.

Leading Practices for IT Audit Planning

According to The IIA’s Global Technology Audit

Guide, Developing the IT Audit Plan (2008), creating an

effective IT audit plan is a four-step process. The first

step is to understand the business. This means

identifying the strategies, company objectives, and

business models that will enable CAEs to understand the

organization’s unique business risks, as well as

understanding how existing business operations and IT

service functions support the organization.

The remaining steps in the IT audit planning process are:

Defining the IT audit universe following a top-

down approach that identifies key business

objectives and processes, significant applications

supporting the business processes, the

infrastructure needed for all business applications,

the organization’s service support model for IT,and the role of common supporting technologies

such as network devices.

Performing a risk assessment that ranks audit

subjects using IT risk factors and assesses risk

using business risk factors.

Formalizing the IT audit plan within the

constraints of the internal audit activity’s operating

budget and available resources. The IT audit plan

should be integrated into the organization’s overall

annual audit plan. Levels of integration may vary

depending on the organization’s needs and

available resources. For instance, CAEs may

choose to have a low-integrated IT audit plan (i.e.,a stand-alone plan under the responsibility of the

IT audit team that is organized by IT subject areas,

is generally isolated from non-IT audit activities,

and includes the review of applications); a partially

integrated audit plan (i.e., a plan that outlines IT

audit engagements available by a core IT audit

team, as well as provides an additional set of

planned engagements distributed across non-IT

audit teams and coordinated with other process

reviews); or a highly integrated audit plan (i.e., a

plan where IT audit activities are incorporated as

part of business process engagements).

http://www.theiia.org/guidance/standards-and-guidance/ippf/practice-guides/gtag/gtag11/http://www.theiia.org/guidance/standards-and-guidance/ippf/practice-guides/gtag/gtag11/http://www.theiia.org/guidance/standards-and-guidance/ippf/practice-guides/gtag/gtag11/


9/82

6

Furthermore, participants were asked to specify how much of their IT audit activities are co-sourced,outsourced, or both. According to study results:

88.8 percent of the 17.4 percent of organizations that co-source IT audits, co-source anywhere from1 percent to 25 percent of their IT audit efforts.50 percent of the 6.5 percent of organizations that outsource IT audits, outsource anywhere from

76 percent to 100 percent of their IT audit efforts.65.1 percent of the 23.9 percent of organizations that both co-source and outsource IT audits, co-sourceand outsource anywhere from 10 percent to 75 percent of their IT audit efforts (refer to figures 6 –8 for abreakdown of these percentages by type of sourcing activity).

s

Figures 6 –7: Percent of IT audits that are co-sourced (left) and outsourced (right)

Figure 8: Percent of IT audits that are both co-sourced and outsourced

The top five reasons why IT audit activities are either co-sourced or outsourced include having better access tosubject-matter experts (79.5 percent), internal audit staff limitations (75 percent), cost-effectiveness of the co-sourcing or outsourcing activity (43.2 percent), lack of internal audit staff knowledge on the IT systems used inthe organization (36.4 percent), and difficulty in recruiting qualified IT audit staff (22.7 percent). Respondentsalso were asked to rate the ability of their in-house staff to evaluate the quality, effectiveness, and efficiency oftheir IT audit activities, as well as their overall satisfaction with their organization’s IT audit efforts. Overall, themajority of study participants provided positive ratings in each category:

79.5 percent rated the ability of their in-house audit staff to evaluate the quality of their outsourced orco-sourced IT audit work as good to excellent.

63.1 percent rated the effectiveness of their IT audit activities as effective to highly effective.51.7 percent rated the efficiency of their organization’s IT audit activities as efficient to highly efficient.57.6 percent were satisfied to highly satisfied with their organization’s overall IT audit efforts.


10/82

7

Reasons provided for these ratings include:

Ability of staff (e.g., level of experience of internal audit manager, CAE, and IT audit staff, and thepresence of excellent lines of communication among audit staff with service providers and auditees).IT audit activity effectiveness (e.g., good communication among IT auditors, the IT department, andthe board; presence of well-trained staff; and an excellent vendor and risk assessment process).

Efficiency of IT audit activities (e.g., presence of highly qualified staff; excellent working relationshipswith management and senior auditors; and presence of a continuous improvement and review process).Overall satisfaction with IT audit activity (e.g., presence of trained, qualified staff with informationsystems experience and the use of technology that meets business ’ needs. (A detailed summary ofeach rating and why each rating was chosen are provided in the Appendix to this report.)

I T Audit Skill s and Sources of Traini ng

____________________________________

According to GAIN’s Annual Benchmarking Study — a year-round survey that compiles the responses of morethan 600 CAEs from organizations around the world — 47 percent of general auditors are encouraged toreceive IT training by attending internal or external formal training sessions, while 47 percent of audit staffthoroughly understand IT concepts and test IT general controls as part of their audit reviews.6 To obtain more

in-depth information regarding these statistics, this year’s IT audit study participants were asked questionsregarding the IT skills and level of training present in their respective internal audit function.

First, respondents were asked to list the top three issues that will impact IT audits the most within the next 24months. These three issues include IT audit project limitations due to budget restrictions, lack of internalresources or time, increasing travel costs, and lack of overall knowledge to perform an IT audit (43.5 percent);data security and privacy (37.7 percent); and being unable to add value to the organization due to the increasingcomplexity of IT systems (23.2 percent). Table 2 provides a detailed summary of each issue.

Top Three Issues That Will Impact IT Audit the Most Within the Next 24 Months

I T audit project limitations due to budget restrictions caused by the current economic downturn or

shifting organizational priorities; time constraints; lack of internal resources to perform the IT audit,

such as lack of qualified staff due to turnover or budget cuts; increasing travel costs; and lack of overall

knowledge to perform an IT audit (60 responses).

Data secur ity and pri vacy : compliance with data security and privacy laws and regulations (e.g.,

compliance with the Payment Card Industry Data Security Standard) and information security and data

privacy practices within the organization (e.g., user provisioning, data access, and change

management) (52 responses).

Being unable to add value to the organization due to the increasing complexity of I T systems , which

prevents the internal audit activity from keeping up with technological changes and innovations, as

well as not having the knowledge to audit and provide support during new system implementations as

a result of out-of-date technology, replacement of legacy systems, and automation of existing controls

(32 responses).

Table 2: Top three issues that will impact IT audits the most within the next 24 months

6 These statistics were obtained from 569 organizations representing the gamut of industries, company types, annual revenues, and assetsizes that participated in the Annual Benchmarking Study from June 30, 2007 until Dec. 31, 2008.

http://www.theiia.org/research/benchmarking/gain/http://www.theiia.org/research/benchmarking/gain/http://www.theiia.org/research/benchmarking/gain/http://www.theiia.org/research/benchmarking/gain/


11/82

8

Based on these responses, study participants were asked if they had the skills and training to address theissues that will impact IT audits the most. The vast majority of participants responded “yes” to both questions — 71.7 percent and 72.2 percent of internal audit activities represented in the study have the skills and training,respectively, to address the issues that will impact IT audits the most within the next 24 months. In terms ofskills, participants stated their internal audit activity has a dedicated group of IT auditors or internal auditors withsufficient training to perform IT audits or with IT-specific certifications, such as the ISACA’s Certified Information

Systems Auditor, while training criteria identified include providing staff with the continuing education needed toperform their work and the presence of a training plan that addresses the needs of each auditor.

Furthermore, participants were asked to identify the latest three technology innovations that have eased theperformance of IT audits the most within the last three years. These include:

1. Use of CAATs, such as audit administration tools and documentation software; automated changemanagement applications; new audit tracking software, and help desk audit software.

2. Availability of many systems online, which enables remote audit activities.3. Guidance on specific IT audit areas or guidance that is tailored to noncomplex IT environments.

By far the primary source of IT audit knowledge during the past 24 months is participation in seminars,workshops, and conferences offered by a professional organization (44 percent), remotely followed by individualresearch gathered from online resources (e.g., The IIA and ISACA), and books or self-study courses (refer tofigure 9 for a summary of all responses). The top organization selected as the first choice for increasing IT auditknowledge was ISACA (47.8 percent) followed by The IIA (20 percent), the MIS Training Institute(18.9 percent), the SANS Institute (4.4 percent), and the American Institute of Certified Public Accountants(2.2 percent).

Figure 9: Primary source of IT audit knowledge


12/82

9

IT AUDIT TOOLS AND TECHNIQUES

To identify the kinds of tools in place for different internal audit processes, study participants were asked toprovide information on the primary and secondary software tools used for the following activities:

Extraction.

Data analysis.Fraud detection or investigation. Automated working papers.Control self-assessments.Compliance.Continuous auditing.Risk assessments for the annual audit plan.

Following is a summary of all responses for eachof these software categories. (For tips on howCAEs and other internal auditors can demonstratethe benefits of using technology-based audittechniques to their audit committee and senior

management team, read ―Showing the Value ofIT Audit Tools‖ on this page.)

Extraction Software ____________________________________

According to study results, 63 percent ofrespondents use software for extraction comparedto 32.6 percent who do not and 4.3 percent whoanswered ―not applicable.‖ The top three primarysoftware or tools identified by study participantsfor extraction are ACL, application queries, andSAP and SQL (tied for third place). Similarly,

study respondents stated that the secondarysoftware tools used for extraction are Excel, ACL,IDEA, Access, and SQL.

To get a better idea of how these software toolshave helped internal audit activities perform moreeffectively, respondents were asked to identifyhow the use of the software has improved theirinternal audit capabilities, a success story or bestpractice linked to the use of the software, and anexample of a barrier or challenge presented bythe use of the software. Table 3 summarizes the responses to these three questions.

Showing the Value of IT Audit Tools Demonstrating the value of using a particular IT audit software

or tool is not easy, especially in organizations where cost-saving

initiatives are taking place. One way to show the value these

tools can bring to the internal audit function is by setting clear

expectations regarding the tool’s cost as well as the amount of

time necessary for setting up the tool correctly and learning how

to use it.

Another way to show the value of using IT audit tools is by

demonstrating how the tool can be leveraged by otherdepartments or teams in their day-to-day work. Not only does

this allow for the tool’s cost to be absorbed by more than one

business unit, but it enables the organization to use one commonset of metrics for a specific activity.

Finally, CAEs can demonstrate the financial value of using a

particular tool to their audit committee and senior management

team. For instance, CAEs can discuss:

The number of work hours audit staff will save by

using the tool as well as how this free time can be used

in support of other audit and business projects.

The amount of money the organization will save on a

monthly, quarterly, or yearly basis by using the tool.

The extra number of audits the organization will be

able to perform once the tool is implemented properlywithout the need for extra staff.

The amount of money the organization will save by

standardizing key metrics and processes that cross

several business functions through the use of the same

audit tool.

Source: GAIN’s The I nternal Audit Activity: Cur rent Tr ends, I ssues,

and Practices report (March 2009)


13/82

10

How has the use of the software identified previously improved your internal audit capabilities?

Enables audits consisting of 100 percent of the population (20 responses).

Improves productivity and efficiency of work (i.e., better able to extract, analyze, and acquire data from corporate systems; has drill-downcapabilities; and reduces the amount of time required to identify potential problems) (18 responses).

Enables continuous monitoring of data (2 responses).

Please provide a success story or best practice linked to the use of the software identified previously:

Has enabled the use of exception reports and tests that identify fraud, misuse of expense reports, and staff who didn't charge leavetime, as well as test pricing invoices and internal controls (12 responses).

Analyzes the entire population rather than a sample and identifies true error rates (6 responses).

Identifies financial savings to the organization (1 response).

Please provide an example of a barrier or challenge presented by the use of the software identified previously:

Learning curve and training (e.g., training staff to use the system; the system is cumbersome to work with) (14 responses).

Getting data in the proper format (e.g., using the system requires the use of SQL querying) (7 responses).

The tool doesn't work or integrate well with other systems and is only used by internal audit department (5 responses).

Difficulty accessing data; takes more time to access data than it should; or difficulty in getting access from data owners (5 responses).

Table 3: Responses to questions on software improvement areas, success stories, and challenges presented

Data Analysis Software ____________________________________

Data analysis tools are the highest type of software used by study participants — 76.1 percent of participantsuse data analysis compared to 19.6 percent who do not and 4.3 percent who answered ―not applicable.‖ The topthree primary software or tools identified by study participants for data analysis are ACL, Excel, and Access.The secondary software tools used by study respondents for data analysis also include Access, ACL, and Excel.In addition, ways the software has improved internal audit capabilities or posed a challenge are listed in table 4.


Has increased the efficiency of audits (e.g., more detailed analysis of data; helps analyze data for trends; helps to identify what data aresaying easily; and provide timely analysis of data) (11 responses).

Enables the sorting, viewing, and analysis of large amounts of data or 100 percent of all data (11 responses).

Established independence of internal audit function (e.g., internal auditors now provide data to clients or external auditors) (2 responses).

Helps internal auditors obtain frequency of errors and detect fraudulent activities (2 responses).

Helps to continuously monitor control (1 response).


Reduced audits and testing to areas of importance by helping auditors focus fieldwork on data identified with reference to anomalies,red flags, potential fraud, or other issues otherwise not found without the tool (6 responses).

Has enabled the review of user access, areas leading to losses of revenue, and segregation of duty conflicts (2 responses).

Able to continuously monitor 100 percent of all data (1 response).

Able to conduct inventory analysis of multiple sites (1 response).


The tool is not user friendly and requires a high level of training (5 responses).

Accessing data (i.e., data is saved in formats that are not conducive to software analysis and it is difficult to obtain data stored in twosystems or legacy systems) (6 responses).

Older versions of Excel or Access do not have the bandwidth to analyze large volumes of data (2 responses).



14/82

11

Fraud Detection or I nvestigation Software ____________________________________

In terms of fraud detection or investigation software, responses are split in terms of its use — 46.7 percent ofparticipants use fraud detection or investigation software compared to 48.9 percent who do not and 4.3 percentwho answered ―not applicable.‖ The top three primary software or tools identified by study participants for fraud

detection or investigation are ACL, Excel, and IDEA, while the secondary software tools used by studyrespondents for fraud detection or investigation include Access, Crystal Reports, Excel, IDEA, and Nuix. Waysthe software has improved internal audit capabilities or posed a challenge are listed in table 5.


Enables reporting of fraudulent transactions and abnormal activities and identifies users processing transactions on an ongoing basis(e.g., on a quarterly basis) (4 responses).

Helps auditors test the entire population in less time (3 responses).

Has the ability to manipulate data for analysis of trends or compare data within different systems or tables (2 responses).

Enables access to data more easily and network access to new IT system information (2 responses).

Is able to investigate fraud more efficiently and save time during fraud investigations (2 responses).

Auditors are able to run CAATs for external auditors (1 response).


Found bogus vendors or addresses (e.g., matched vendor addresses with employee addresses) (2 responses).

Mapped data to users (e.g., mapping of e-mails to determine their content, who received the e-mail, and the actions taken by recipients;mapping data to identify noncompliant cases) (2 responses).

Provides more visibility of information (1 response).

Increased number of fraud items for investigations (1 response).


Issues with the data (e.g., obtaining data in the first place; getting files to import from IT in a timely basis; and defining data to conductthe investigation) (3 responses).

High volume of data to analyze (1 response).

Learning curve to use software properly (1 response).

Cannot perform transactions in real time (1 response).


Automated Working Paper Software

____________________________________

A little more than half of study respondents (52.2 percent) use software to automate their working paperscompared to 47.8 percent who do not. The top three primary software or tools identified by study participants fordata analysis are TeamMate, AutoAudit, and Word, and the secondary software tools used by study participantsfor automated working papers include Access, Excel, OpenPages, and TeamMate. In addition, ways thesoftware has improved internal audit capabilities or posed a challenge are listed in table 6.


15/82

12


Use of standard templates (e.g., standardization of templates provides high-productivity and efficiency of work and for the automation of

audit processes and consistency between projects) (14 responses).

Improves quality of review, audit program, and work papers (e.g., reduces review times for audit files and facilitates the sharing and remote

reviews of working papers) (7 responses).

Better organization and access of information (e.g., provides a centralized storage of audit working papers) (6 responses).

Enhances the follow up of audits, tracking of audits, and repeatability of audits (4 responses).Reduces planning time and staff work (e.g., time used to ensure working paper documentation complies with IIA Standards is significantlyreduced) (4 responses).

Provides better coordination of all audits (e.g., coordination of audits with Sarbanes-Oxley audits and automated reporting of Sarbanes-Oxley work and other internal audits) (2 responses).

Increases audit penetration (1 response).

Improves data protection (1 response).


Facilitated the automation of work (e.g., has enabled the automation of tracking issues and responses from responsible parties, as well asthe automation of draft reports generated after completion of field work) (2 responses).

Enabled auditors to review work papers from remote locations (2 responses).

Eliminated used hard copies (i.e., all work papers are saved electronically, which saves space and reduces waste) (2 responses).

Provided consistency of work (e.g., enables consistency of work papers by allowing auditors to choose which fields to use) (2 responses).

Increased the efficiency of compliance reviews with IIA Standards (2 responses).

Enabled more than one auditor to work on the same project (1 response).Eased the documentation of work papers (1 response).


Software can be cumbersome to use, replicate in the existing environment, or integrate with other software, which leads to an inefficient

use of time (5 responses).

Auditors have lost work occasionally due to bugs within the system and lack of customer service support from vendor (2 responses).

Because the software automates all work papers, too much information is kept, which can be overwhelming (2 responses).

Use of the software and review of work papers still requires human interaction, which introduces inconsistencies unless the internal auditdepartment has a standard infrastructure in place (2 responses).

It is difficult to access work papers in locations where bandwidth is an issue (1 response).

Cost of training (1 response).

Cannot monitor action items in an automated fashion (1 response).


Control Self -assessment Software ____________________________________

Unlike the other types of software, tools for control-self assessments are not used widely. In fact, 65.2 percent ofstudy respondents do not use them at all, compared to 21.7 percent who do and 13 percent who stated they are―not applicable‖ to their work. The top primary software or tool identified by study participants for control self-assessment is Excel. Other primary tools listed were only identified by one respondent each and include AutoAudit, Axentis, FCM, Lumigent Audit DB, Movaris, Option Finder, PolicyIQ, Risk Navigator, SharpeDecision, TeamRisk, and Word.

Reasons given by study participants for the use of Excel include its simplicity and availability within theorganization. The only secondary software tool used by respondents for control self-assessment is the TurningPoint Audience Response System. Ways control self-assessment software has improved internal audit

capabilities or posed a challenge are listed in table 7.


16/82

13


The tool has made the control self-assessment process more efficient and less costly (3 responses).

Results are immediately summarized and graphs produced, which has resulted in significantly reduced time in summarizing controlself-assessment results (1 response).


Other groups in the organization are able to design their own questions, which has cut costs of external resources for management testing

to less than 10 percent (1 response). Self-assessments are performed on a more timely basis, and it is easier to provide assessment information as needed (1 response).


The control self-assessment’s tool questionnaire design has presented problems (1 response).

Third-party software has response issues (1 response).

Vendor is not helpful in automating tasks and there are consulting fees associated with assisting in data uploads (1 response).

Not all divisions in the company are using the software (1 response).

The audit team is unable to run reports off of the information received (1 response).


Compli ance Software ____________________________________

Similar to the use of control-self assessment software, compliance tools are not widely used — 55.4 percent ofstudy respondents do not use compliance software compared to 23.9 percent who do and 20.7 percent whostated this type of software is ―not applicable‖ to their work. While no single software tool was listed by morethan one study participant as the primary software used for compliance, applications identified include Access, ACL, Compliance 300, Excel, IDEA, Implexus, Movaris, Oracle’s Apex Application, Oracle GRC, PolicyIQ,Resolver Risk, Showcase Quary, and Word. The only secondary software listed by respondents for complianceis Access.

Ways compliance software has improved internal audit capabilities or posed a challenge are listed in table 8.


Our compliance area primarily uses this software (1 response).

Identify specific transactions of possible concern (1 response).

Directs audit work (1 response).

Provides a common centralized approach to performing compliance audits (1 response).Compliance audits are timely, easy, and effective (1 response).


Provided a common centralized approach to performing compliance audits (1 response).


It is not easy to update use compliance logs (1 response).


Continuous Audi t Software ____________________________________

Continuous audit software is also not widely popular. According to study results, 59.8 percent of respondents donot use continuous audit software compared to 25 percent who do and 15.2 percent who stated this type ofsoftware is ―not applicable‖ to their work. The top primary software or tool identified by survey participants forcontinuous auditing is ACL. Other primary tools listed were only identified by one respondent each and includeExcel, IDEA, Oracle Apex Database, PeopleSoft, Proprietary Data Extraction, and Showcase Query. Reasonsgiven for the use of ACL by study participants include its ability to look at control weaknesses, the ease withwhich users can evaluate data, and its ability to provide exception reports.

Two tools were identified as the secondary software applications used by respondents for continuous auditing.These are ARC and Access, in order of importance. Ways compliance software has improved internal auditcapabilities or posed a challenge are listed in table 9.


17/82

14


Auditors are alerted of issues as they occur (i.e., there is no lag time to identify issues) and tool creates exception reports (2 responses).

Tool audits 100 percent of the population rather than a sample (1 response).

Allowed the internal audit activity to create preventive controls for process owners (1 response).


Ability to quickly identify a number of irregularities including fraudulent transactions (1 response).

Please provide an example of a barrier or challenge presented by the use of the software identified previously: Process takes a while to implement correctly, based on the organization’s needs and system changes (2 responses).

Auditors need to have detailed knowledge of the underlying data structures to use the tool correctly (1 response).

Auditors have to determine the parameters to be used (1 response).

The organization has a hard time accepting reports generated by the tool (1 response).


Software Used to Assess Risks for the Annual Audit Plan ____________________________________

Finally, study participants were asked to identify the primary and secondary software tools used to assess risksfor the annual audit plan. Again, the majority of study participants do not use software to assess annual auditplan risks — 57.6 percent of respondents do not use this kind of software compared to 39.1 percent who do and3.3 percent who stated this type of software is ―not applicable‖ to their work. The top three primary software ortools identified by study participants to assess risks are Excel, TeamMate, and Team Risk. Other primary toolslisted were only identified by one respondent each and include ACL, AutoAudit, and Crystal Reports. In addition,two tools were identified by study participants as the secondary software applications used for assessing risks.These are Crystal Reports and TeamMate. Ways risk assessment software has improved internal auditcapabilities or posed a challenge are listed in table 10.


The tool enables auditors to track risks consistently and provides a standard format for all risk assessments and risk calculations, which

makes it easier to compare risks across the organization (5 responses).

Provides a central tracking location (2 responses).

Saves time when performing the risk assessment (e.g., sort for different types of risks) (2 responses).

Software illustrates risk assessment results graphically and by using standard reports (1 response).


Helps in the audit planning process (e.g., annual audit plan is prepared promptly by the use of the risk assessment system) (2 responses). Provides a common communication mechanism of risk assessment results (1 response).

Is enabling our internal audit activity to develop a mathematical risk assessment model (1 response).

Risk criteria are consistently updated as audits are performed, which enables auditors to determine which audit areas are considered highrisk at a glance because of the control environment or whether an audit has not been performed in a while (1 response).

Enables use of the Monte Carlo technique for risk assessments (1 response).


Software is cumbersome to use and could use additional automation (2 responses).

It is difficult to incorporate changes to a spreadsheet (1 response).

Risk assessment process is still subjective (1 response).

Lack of adequate resources to patch or upgrade the system to eliminate problem areas and to build new features (1 response).

Table10: Responses to questions on software improvement areas, success stories, and challenges presented


18/82

15

CLOSING THOUGHTS

Establishing an effective IT audit function should be a carefully thought-out process that not only incorporatesexisting internal audit resources, but meets the organization’s IT audit needs. Respondents to this study seem tobe moving in the right direction in ter ms of their IT audit activities the vast majority of study participantsincorporate IT audits as part of the internal audit plan; the majority of internal audit groups represented in the

study have the skills and knowledge necessary to evaluate the quality, effectiveness, and efficiency of theorganization’s IT audit activities; and overall satisfaction with IT audit efforts is positive. In addition, more than 70percent of study respondents indicated their internal audit activity has the skills and training needed to addressthe issues that will impact IT audits the most within the next 24 months. This is particularly important giventoday’s economic downturn, which is affecting many organizations’ ability to provide the training needed to keepup with today’s technological innovations.

If there is one area for improvement, it is in the use of audit software. In particular, most study respondentsindicated they do not use software to detect or investigate fraud, perform control self-assessments, monitorcompliance activities, partake in continuous auditing, and assess risks for the annual audit plan. While noreasons were given regarding the lack of software used for these activities, technology-based audit techniquescan greatly maximize internal audit efforts. This is especially true in large-size organizations, where continuousaudit software, for instance, can increase the scope of internal audit activities to cover as much as 100 percent

of all auditable universe components, and in small internal audit groups, where audit software can help internalauditors perform faster, more effective audits.


19/82

16

APPENDIX: IT AUDIT BENCHMARKING STUDY RESULTS ____________________________________

IT Audit Benchmarking StudyType: Executive Summary Report

Date: February 2009Total number of invitations: 1,709

Total number of responses collected: 138 (8.1 percent)

1: Are you the chief audit executive or top internal audit authority of your organization? (Respondents could only choose a single response)

Response Chart Frequency Count

Yes 70.3% 97

No 29.7% 41

Valid Responses 138

Total Responses 138

2: Does your organization incorporate IT audit activities as part of the internal audit plan? (Respondents could only choose a single response)


Yes 94.8% 92

No 5.2% 5

Valid Responses 97

Total Responses 97

2a: Please explain the process used to incorporate IT audit activities as part of the internalaudit plan:

Response (Yes)

The internal audit activity takes an integrated IT audit planning approach in which potential IT audit areasare determined as part of the risk assessment process or annual audit planning process to determine all audituniverse components. Once an IT audit universe is determined based on areas of high risk, a schedule iscreated to monitor/review IT audit universe components on a specific timeframe. These IT audit universecomponents are either incorporated into the annual audit plan or kept as a separate IT audit plan (e.g., auniverse of IT audits is created as part of the normal audit planning process, in which IT audit areas are risk-ranked. The highest risk-ranked audits are included in the overall audit plan to the extent that the internalaudit department has the IT resources to allocate to them. Risk assessment interviews are also performed,including interviews with IT management. IT components that are ranked for risk include systemapplications, as well as operations, access, and change management controls) (73 response).The internal audit activity performs a separate IT audit risk assessment to identify the IT audit areas to beaudited throughout the year. These areas are added to the overall annual audit plan (15 responses).

IT audits are determined based on core business functions and processes (8 responses).


20/82

17

2b: Please explain why you do not incorporate IT audit activities as part of the internal auditplan:

Response

The internal audit activity does not have the skills or financial resources necessary to perform IT audits(1 response).

IT management does not provide the information necessary for the internal audit activity to review ITactivities and processes (1 response).

IT audits are outsourced (1 response).

3: Please identify whether your organization co-sources or outsources any of its IT auditactivities. (Respondents could only choose a single response)


Co-source 17.4% 16

Outsource 6.5% 6

Both co-source and outsource 23.9% 22

None 52.2% 48

Valid Responses 92

Total Responses 92

3a: How much of your organization's IT audit activities are co-sourced? (Respondents could only choose a single response)


Less than 10% 44.4% 8

10% – 25% 44.4% 8

26% –50% 0.0% 0

51% –75% 0.0% 0

76% –99% 0.0% 0

100% 11.1% 2

Valid Responses 18

Total Responses 18


21/82

18

3b: How much of your organization's IT audit activities are outsourced? (Respondents could only choose a single response)


Less than 10% 16.7% 1

10% –25% 16.7% 1

26% –50% 0.0% 0

51% –75% 16.7% 1

76% – 99% 33.3% 2

100% 16.7% 1

Valid Responses 6

Total Responses 6

3c: How much of your organization's IT audit activities are both co-sourced and outsourced? (Respondents could only choose a single response)


Less than 10% 17.4% 4

10% – 25% 21.7% 5

26% –

50% 21.7% 5

51% – 75% 21.7% 5

76% –99% 13.0% 3

100% 4.3% 1

Valid Responses 23

Total Responses 23


22/82

19

4: Why is the IT audit activity co-sourced or outsourced in your organization? (Respondents were allowed to choose multiple responses)


Internal staff limitations 75.0% 33

Budget limitations 9.1% 4

More cost effective 43.2% 19

Better access to subject-

matter experts79.5% 35

Audit committee requirement 0.0% 0

Regulatory requirement 4.5% 2

Internal auditors do not havesufficient knowledge on theIT systems used in theorganization

36.4% 16

Difficulty in recruitingqualified IT audit staff

22.7% 10

Difficulty in retaining qualifiedIT audit staff

13.6% 6

Other (explained in 4.1) 0.0% 0

Valid Responses 44

Total Responses 44

4.1: Why is the IT audit activity co-sourced or outsourced in your organization?

Response - None

5: Please rate the ability of your in-house audit staff to evaluate the quality of the outsourcedor co-sourced IT audit work performed and explain why you chose the rating: (Respondents could only choose a single response)


Unacceptable (explained in 5.1) 2.3% 1

Needs major improvement

(explained in 5.2)2.3% 1

Needs some improvement(explained in 5.3)

4.5% 2

Fair (explained in 5.4) 11.4% 5

Good (explained in 5.5) 47.7% 21

Excellent (explained in 5.6) 31.8% 14

Valid Responses 44

Total Responses 44


23/82

20

5.1: Please explain why you rated the ability of your in-house audit staff as unacceptable:

Response

Staff do not evaluate the IT audit work; work is 100 percent outsourced and only evaluated or reviewed bythe CAE (1 response).

5.2: Please explain why you rated the ability of your in-house audit staff as needs majorimprovement:

Response

None of the in-house staff have an IT background (1 response).

5.3: Please explain why you rated the ability of your in-house audit staff as needing someimprovement:

Response

Staff have general IT knowledge and background, but are not fully technically competent (1 response).

Working with consultants is a new skill set for my staff, and we continue to work with managing their workand their reporting activities (1 response).

5.4: Please explain why you rated the ability of your in-house audit staff as fair:

Response

Staff have limited technology knowledge (2 responses).

5.5: Please explain why you rated the ability of your in-house audit staff as good:

Response

Experienced IT auditor(s) (7 responses).

Experience of internal audit manager and CAE (4 responses).

Good contract management skills (6 responses).

Good peer references and feedback from the auditees (1 response).

5.6: Please explain why you rated the ability of your in-house audit staff as excellent:

Response

Experienced IT auditor works at the organization (9 responses).

Excellent communications with service providers (3 responses).


24/82

21

6: Please rate the effectiveness of your organization’s IT audit activities and explain why youchose the rating: (Respondents could only choose a single response)


Highly ineffective (explained in

6.1) 1.1% 1

Ineffective (explained in 6.2) 3.3% 3

Moderately ineffective(explained in 6.3)

8.7% 8

Moderately effective(explained in 6.4)

23.9% 22

Effective (explained in 6.5) 45.7% 42

Highly effective (explained in6.6)

17.4% 16

Valid Responses 92

Total Responses 92

6.1: Please explain why you rated the effectiveness of your organization’s IT audit activities ashighly ineffective:

Response - None

6.2: Please explain why you rated the effectiveness of your organization’s IT audit activities asineffective:

Response

Reduced or understaffed (1 response).

Without expertise in this area, it is difficult to conduct audits other than access control audits, whichcan be done by a non-IT auditor (1 response).

6.3: Please explain why you rated the effectiveness of your organization’s IT audit activities asmoderately ineffective:

Response

Not enough skilled IT audit staff available (4 responses).

Inexperienced staff (2 responses).

IT general control audits have not been completed for quite some time. Other audit work was identified onan ad hoc basis. I am new to my position and making significant changes to our processes (1 response).

Limited in-house resources and knowledge (1 response).


25/82

22

6.4: Please explain why you rated the effectiveness of your organization’s IT audit activities asmoderately effective:

Response

Lack of solid skills (6 responses).

IT auditing is a new function, and we have had some trouble in the IT area (2 responses).

The organization’s IT audit executive has acquired knowledge of the IT environment and control situationsduring the audit exercise and has an established relationship with IT management personnel (1 response).

Need for better coordination with IT department for mapping the IT universe and following up onrecommendations (1 response).

Not well-led and not using current technology; we are several generations behind. This is because thecompany is not heavily IT dependent, which is proven based on the frequency of outages (1 response).

Provides a level of insight to our CIO that previously did not exist (1 response).

Generally, these activities are effective but we have issues with the ISO position (1 response).

We have done a comprehensive review but have not touched on key controls (1 response).

We just expanded our staff from one to three people and we are still improving our processes(1 response).

We need a formalized risk model that will increase the effectiveness of our IT audit activities (1 response).

We review key risks, but probably need a lot more focus around information security (1 response).

6.5 Please explain why you rated the effectiveness of your organization’s IT audit activities aseffective:

Response

Good communications and response from function, IT department, and the board (16 responses).

Excellent vendor and risk assessment process (5 responses).

Solid knowledge and provides value-added recommendations (4 responses).

Changes are made to improve controls (3 responses).

All risks are covered on a risk-based cycle (1 response).

I believe we hit the high-risk areas, but we could do more if we had more audit resources (1 response).

IT reviews are operational but not technical (1 response).

Key IT general controls seem to be working; IT governance was weak but appears to be improving toacceptable standards. Key applications are rigidly maintained as they support government-regulatedprocesses (1 response).

Findings are always relevant and helpful to the organization (1 response).

We are effective for control testing, but not necessarily for operational efficiencies (1 response).

We have the audit committee’s and CEO attention and CIO is making changes (1 response).


26/82

23

6.6: Please explain why you rated the effectiveness of your organization’s IT audit activities ashighly effective:

Response

Excellent and well-trained staff (7 responses).

All areas are subject to audit including IT activities (2 responses).

Areas to be audited are mutually agreed upon by internal auditing and the CIO (1 response).

We get positive feedback from auditees and the audit committee (1 response).

Reduces external audit fee and IS agree with recommendations (1 response).

Good collaboration between IS management, internal audit management, and vendors (1 response).

We look at controls; others review technical issues (1 response).

7: Please rate the efficiency of your organization’s IT audit activities and explain why you chosethe rating: (Respondents could only choose a single response)


Highly inefficient (explained in7.1)

1.1% 1

Inefficient (explained in 7.2) 0.0% 0

Moderately inefficient(explained in 7.3)

9.9% 9

Moderately efficient(explained in 7.4) 37.4% 34

Efficient (explained in 7.5) 38.5% 35

Highly efficient (explained in7.6)

13.2% 12

Not Answered 1

Valid Responses 91

Total Responses 92

7.1: Please explain why you rated the efficiency of your organization’s IT audit activities as

highly ineffective:

Response - None

7.2: Please explain why you rated the efficiency of your organization’s IT audit activities as ineffective:

Response - None


27/82

24

7.3: Please explain why you rated the efficiency of your organization’s IT audit activities asmoderately ineffective:

Response

Inexperienced staff (4 responses).

Slow to respond; short staffed; internal cooperation (2 responses).

Sometimes too thoroughly audited (1 response).

We are efficient for our size and skill, but unable to address critical concerns due to staffing (1 response).

7.4: Please explain why you rated the efficiency of your organization’s IT audit activities asmoderately effective:

Response

IT audit area and plans are new and still in development (4 responses).

Staff experience and IT background is limited (3 responses).

Experience of IA audit manager and resources (2 responses).

Access to outsource providers supplements departmental needs (2 responses).

Challenge completing audits within budgeted time and given deadlines (2 responses).

Using co-sourcing partners is inefficient without adequate management oversight (2 responses).

Legacy systems and IT silos are key issues (1 response).

Significant research is needed for in-house audit procedure development (1 response).

Implementing an ERP (1 response).

The IT audits can be bigger and more challenging (1 response).

Inefficiency is caused by physical distance between our department and the IT department (1 response).

We can get better in timely completion of audit reports (1 response).

We have standardized IT audit procedures and testing is generally consistent (1 response).

We need a formalized risk model that will increase the efficiency of our IT audit activities (1 response).


28/82

25

7.5: Please explain why you rated the efficiency of your organization’s IT audit activities aseffective:

Response

Highly qualified staff (10 responses).

Excellent working relationship with management and external auditors (5 responses).

Continuous improvement and review (3 responses).

Excellent use of tools and methodologies (2 responses).

Use of risk ranking to guide audit performance (2 responses).

Use of rotation plan for GCC audits (1 response).

7.6: Please explain why you rated the efficiency of your organization’s IT audit activities ashighly effective:

Response

Well trained, dedicated, and certified staff (4 responses).

IT audit activities are risked-based and targeted to issue risks ranked as high (3 responses).

Excellent communications with audit committee and CIO (1 response).

Excellent outsourcing partner (1 response).

8: Please rate your overall satisfaction with your organization's IT audit activity and explain why

you chose the rating: (Respondents could only choose a single response)


Highly dissatisfied (explained in8.1)

1.1% 1

Dissatisfied (explained in 8.2) 8.7% 8

Moderately dissatisfied(explained in 8.3)

4.3% 4

Moderately satisfied (explainedin 8.4)

28.3% 26

Satisfied (explained in 8.5) 34.8% 32

Highly satisfied (explained in8.6)

22.8% 21

Valid Responses 92

Total Responses 92


29/82

26

8.1: Please explain why you rated your overall satisfaction as highly dissatisfied:

Response – None

8.2: Please explain why you rated your overall satisfaction as dissatisfied:

Response

Lack of professional staff and training (4 responses).

There are many IT security issues with limited audit staff (2 responses).

There is no chief information officer and therefore the quality is lacking (1 response).

8.3: Please explain why you rated your overall satisfaction as moderately dissatisfied:

Response

Limited and inexperienced staff (3 responses).

8.4: Please explain why you rated your overall satisfaction as moderately satisfied:

Response

Area is still in development with room for improvement (9 responses).

Excellent work but limited resources (2 responses).

Missing application audits (1 response).

Our available IT audit hours are not adequate to meet our audit plan (1 response).

Unrealistic audit committee's expectations for definitive audit opinions and ratings (1 response).

Audit recommendations have generally been accepted and implemented (1 response).

We do not have a large exposure to in-house development. Our risks are limited to third-party products.Our IT structures are decentralized and aligned with each business unit, so we have limited exposure toglobal problems (1 response).

We need a formalized risk model that will increase the quality of our IT audit activities (1 response).


30/82

27

8.5: Please explain why you rated your overall satisfaction as satisfied:

Response

Good work, experienced in systems, and technology that meets audit needs (6 responses).

Limited staff with excellent experience (4 responses).

Excellent communications with audit committee and executive management (1 response).

Experience of internal audit manager and resources (1 response).

Cost is a major consideration (1 response).

We have improved our efficiency by prioritizing IT risks (1 response).

Knowledge transfer is a key to learning the IT area (1 response).

Effective balance between in-house and co-sourced audits and our IT management team seeks ourassistance (1 response).

Need to incorporate technology into all of our audit activities (1 response).

Our IT audits have been improving over the past few years and are approaching highly satisfied(1 response).

Our surveys come back from the organization with high scores (1 response).

Provide basic coverage of key controls (1 response).

Staff from outsourced firm generally not outstanding, but their associate director makes sure everythingworks in the end (1 response).

We cover the management of critical applications and are able to assess ICT governance (1 response).

8.6: Please explain why you rated your overall satisfaction as highly satisfied: Response

Highly trained qualified staff (7 responses).

We provide good coverage of all major risks (1 response).

Excellent feedback from auditees and the audit committee (1 response).

IT is taking action on IT audit reports (1 response).

Material weaknesses are identified and addressed by management (1 response).

Strong relationship with IT; high-quality audit work (1 response).

Good outsourcing partner (1 response).

Viewed as a resource by IT management (1 response).


31/82

28

9.1: List the top three issues that will impact IT audits the most within the next 24 months:

ResponseIT audit project limitations due to budget restrictions caused by the current economic downturn orshifting organizational priorities; time constraints; lack of internal resources to perform the IT audit, suchas lack of qualified staff due to turnover or budget cuts; increasing travel costs; and lack of overall

knowledge to perform an IT audit (60 responses).Data security and privacy: 1) Compliance with data security and privacy laws and regulations (e.g.,compliance with the Payment Card Industry Data Security Standard) and 2) information security and dataprivacy practices within the organization (e.g., user provisioning, data access and change management)(52 responses).Being unable to add value to the organization due to the increasing complexity of IT systems,which prevents the internal audit activity from being able to keep up with technological changes andinnovations, as well as not having the knowledge to audit and provide support during new systemimplementations as a result of out-of-date technology, replacement of legacy systems, and automation ofexisting controls (32 responses).

10: Do you have the skills to address the issues that will impact IT audits the most within the

next 24 months? (Respondents could only choose a single response)


Yes (explained in 10.1) 71.7% 66

No (explained in 10.2) 28.3% 26

Valid Responses 92

Total Responses 92

10.1: Please explain why you selected ―Yes‖:

Response

The internal audit activity has a dedicated group of IT auditors or internal auditors with sufficient trainingto perform IT audits (29 responses).

The internal audit activity has internal auditors who have IT-specific certifications, such as CISA andCISP (15 responses).

The internal audit activity outsources its IT audit activities or co-sources them with other businessfunctions (4 responses).

The internal audit activity has the necessary resources, other than staff, to support the organization'sIT audit needs (3 responses).

10.2: Please explain why you selected ―No‖:

ResponseThe internal audit activity does not include IT auditors, auditors with the necessary IT training orknowledge (e.g., IT audits are a new area of work), or IT subject-matter experts; there is a lack ofcorrelation of specific IT skills to IT audit universe components (15 responses).The internal audit activity does not have 1) the financial resources or time to allow auditors to obtain thenecessary IT audit skills, 2) enough auditors to perform IT audits, and 3) time to perform IT audits (e.g.,due to shifting organizational priorities) (5 responses).The internal audit activity needs to outsource or co-source IT audit activities due to lack of resources(4 responses).


32/82

29

11: Do you have the training to address the issues that will impact IT audits the most withinthe next 24 months? (Respondents could only choose a single response)


Yes (explained in 11.1) 72.2% 65

No (explained in 11.2) 27.8% 25

Not Answered 2

Valid Responses 90

Total Responses 92

11.1: Please explain why you selected ―Yes‖:

Response

Auditors are provided with the necessary training through attendance at seminars, conferences, andother CPE earning events; self-study materials; and internal training (35 responses).

A training plan is developed that addresses the training needs of each auditor (2 responses).

IT audit activities are co-sourced or outsourced to learn from the work and methodologies of others(2 responses).

11.2: Please explain why you selected ―No‖:

Response

There is no budget for IT audit training and training locations are too far for travel to take place

(6 responses).

IT audit training will be addressed in the future (5 responses).

IT audit activity is outsourced and, consequently, there is no need for training (4 responses).

12.1: List the latest three technology innovations that have eased the performance of IT auditsthe most within the last three years:

Response

Use of CAATs, such as audit administration tools and documentation software (e.g., ACL, IDEA,TeamMate); automated change management applications; new tracking software; and help desk audit

software (56 responses).

Availability of many systems online, which enables remote audit activities (8 responses).

Guidance on specific IT audit areas or tailored to noncomplex IT environments (7 responses).


33/82

30

13: Which of the following has been your primary source of IT audit knowledge during the last24 months? (Respondents could only choose a single response)


Seminars, workshops, and/or

conferences offered by professionalorganizations

44.0% 40

In-house training by company employees 2.2% 2

External training offered by consultants ortraining companies

5.5% 5

Books or self-study courses 9.9% 9

On-the-job training 8.8% 8

Peer-to-peer assistance 6.6% 6

Private training offered by a consultant or

training company

2.2% 2

Individual research gathered from onlineresources (specified in 13.1)

13.2% 12

Other (specified in 13.2) 7.7% 7

Not Answered 1

Valid Responses 91

Total Responses 92

13.1: Please explain why you selected ―Individual research gathered from onlineresources‖ as your primary source of IT audit knowledge during the last 24 months:

Response

ISACA (4 responses)

The IIA (4 responses)

software suppliers' Web sites (2 responses)

ACUA (2 responses)

AICPA

ITIL

COBIT

13.2: Please explain why you selected ―Other‖ as your primary source of IT auditknowledge during the last 24 months:

Response

Outsourcing provider (2 responses)

COBIT

ISO

Working with our outsourcing consultants


34/82

31

14: Please select which organization would be your first choice as a source for increasingyour IT audit knowledge: (Respondents could only choose a single response)


American Institute of Certified Public

Accountants2.2% 2

The Institute of Internal Auditors (IIA) 20.0% 18

ISACA 47.8% 43

MIS Training Institute 18.9% 17

SANS Institute 4.4% 4

Other (specified below) 6.7% 6

Not Answered 2

Valid Responses 90

Total Responses 92

14.1: Please select which organization would be your first choice as a source for increasingyour IT audit knowledge:

Response

SAP (2 responses)

The IIA (Germany) (2 responses)

Vendor-specific training

14a: If not The IIA, why?

Response

Courses do not meet the needs or not enough in-depth training is offered (20 responses).

ISACA better meets the needs in IT knowledge (19 responses).

IIA IT courses are too expensive, especially in comparison to others (10 responses).

The IIA seems to lack the experience, expertise, and knowledge as the IT experts (6 responses).

In comparison, MIS offers better IT technical programs (5 responses).

The IIA would be the second choice (3 responses).

COBIT provides more comprehensive information (2 responses).

Use of Local IIA chapters (1 response).


35/82

32

15: Does your internal audit function use software for extraction? (Respondents could only choose a single response) Response Chart Frequency Count

Yes 63.0% 58

No 32.6% 30

Not applicable 4.3% 4

Valid Responses 92

Total Responses 92

15a: Please provide the name of the primary software used for extraction, skill level required, its usefulness to internal auditors,and why it is useful or not:

Software/ Numberof responses

7

Skill level and numberresponding

8

Is this softwareuseful to IA

Explain why software is useful or not useful

ACL (34)

Beginner level of expertise (8) Yes

Ease of sample selections and review of large amounts of data (2responses).

Can use ACL in a variety of audit activities not only IT-related.

Can take huge amounts of data and summarize it and exceptions.Helps us to standardize how we review the data so we are less systemdependent.

Key extracts add value.

Intermediate level of expertise(24)

Yes

Adaptable to multiple applications and easy to use (7 responses).Total monitoring of certain controls; ease of sample selection; unrestricted asto file size (4 responses).

Eases audit analysis in particular of large databases.

Increases efficiency and comprehensiveness of audits.We have direct access to source systems and do not depend on IT to giveus files.

Expert level (2) Yes The software helps to ease the auditor's analysis job.

7 For questions 15 – 22: Only one response was provided in cells where no number is shown.8 For questions 15 – 22: Only one response was provided in cells where no number is shown.


36/82

33

15a: (continued) Please provide the name of the primary software used for extraction, skill level required, its usefulness to internal auditors,and why it is useful or not:





Application queries(4)

Intermediate level of expertise Yes

Easy access to populated information.

Able to extract information from corporate systems for sampling or analysis.

Does not require code development.

Business Objects Beginner level of expertise Yes No comments provided.

Crystal Reports Intermediate level of expertise Yes It is the most flexible of our accounting software.

Excel (2)Expert Level Yes No comments provided.

Intermediate level of expertise Yes Small data extracts can be manipulated.

IDEA Intermediate level of expertise Yes Awesome ability to seek anomalies.

MS Access Expert Level Yes No comments provided.

Proprietary Yes No comments provided.

SAP (3) Beginner level of expertise Yes

Highly useful; we perform all our audits using SAP. We have also written

certain exception reporting applications.Intermediate level of expertise Yes No comments provided.

Expert Level Yes No comments provided.


37/82


38/82

35

15a: (continued) Please provide the name of the secondary software used for extraction, skill level required, its usefulness to internal auditorsand why it is useful or not:





Excel (6)

Beginner level of expertise (2) Yes Helps with smaller systems with less data content.

None chosen YesFlexibility provided to work with live data versus the limitations ofpreprogrammed queries or only being able to see totals and summaries.

Intermediate level of expertise Yes

Excel is limited to the number of records it can extract, plus the ODBCconnection can overwrite live data.

Easy to use.

Expert Level Yes No comments provided.

Focus Intermediate level of expertise Yes No comments provided.

IDEA (2)Beginner level of expertise Yes No comments provided.

Intermediate level of expertise Yes Used to analyze or sample transactions.

Microsoft Suite Intermediate level of expertise Yes No comments provided.

Monarch Intermediate level of expertise Yes Can pull data from almost any report or common file format.

MS Access (2) Intermediate level of expertise Yes Easy to use and flexible.

SAP Intermediate level of expertise No Occasional/infrequent use; we not totally familiar with the tool.

SQL (2) Intermediate level of expertise Yes No comments provided.

Expert Level Yes Ability to obtain other required information.

SharePoint Intermediate level of expertise Yes No comments provided.


39/82

36

15b. Please complete the following:How has the use of the previously identified software improved yourinternal audit capabilities?

Audit 100 percent of the population rather than doing a sample-based audit (20 responses).

Improved productivity and efficiency of work (i.e., better able toextract, analyze, and acquire data from corporate systems; drill-down capabilities; and reduced the amount of time required toidentify potential problems) (18 responses).

Continuous monitoring of data (2 responses).

Please provide a success story or best practice linked to the use of thesoftware identified previously:

Has enabled the use of exception reports and tests that identifyfraud, misuse of expense reports, and staff who didn't charge leavetime, as well as test pricing invoices and internal controls(12 responses).

Ability to analyze the entire population rather tha

IT Audit Benchmarking Study

Documents

Transcript of IT Audit Benchmarking Study