UWCISA Symposium 2007

J. Mack Robinson College of Business / Georgia State University

Auditing system development: Constructing the meaning of “systematic and rational” in the context of legacy code migration for vendor incentives

A. Faye BorthickGeorgia State University, borthick@gsu.edu

Paul L. BowenFlorida State University, pbowen@cob.fsu.edu

Simulation in response to learning needs

J. Mack Robinson College of Business / Georgia State University

1. Growing emphasis on controlling system development to comply with SOX 404

2. Scarcity of cases with data for practicing system development audits

The challenge

J. Mack Robinson College of Business / Georgia State University

Design and implement learning experiences for novices to develop expertise in auditing system development

Create authentic tasks (tasks that IS auditors would perform on the job) to simulate learning on-the-job

The approach

The task: Organofood system development audit

J. Mack Robinson College of Business / Georgia State University

• Organofood: Grocery chain

• Objective: Audit migration of vender incentive code to an enterprise system (ES) with database querying

• Approach: Audit vendor incentive accounts through analysis of the provided data in the context of the situation

Organofood: An authentic task in a simulation

J. Mack Robinson College of Business / Georgia State University

1. Situation narrative: Context in conversation

2. Transaction data: For analysis with database or audit software querying

3. Links to background information

4. SDLC approval for migration project

5. Reporting template

6. Readiness questions for Saving Sergeant Pabletti epiphany

Situation narrative: Context in conversation

J. Mack Robinson College of Business / Georgia State University

The scene: Auditors planning a system development in two parts:

1. Audit of migration of legacy code for vendor incentives to an enterprise system (ES) for a grocer

2. Audit of system development generally

1. Compliance

2. Productivity

Transaction data: Testdata database tables

J. Mack Robinson College of Business / Georgia State University

1. Accounts: Accounts in general ledger

2. GeneralLedger: GL transactions

3. Incentive: Terms of vendor incentives

4. IncentiveCode: Incentive codes by type

5. Invoice: Vendor invoice

6. Purchase: Details of purchases

7. SKU: Information about stock keeping units (SKUs)

8. StoreSales: Sales by period, 4 weeks by week

Transaction data: ProgLibrary database tables

J. Mack Robinson College of Business / Georgia State University

1. LibraryTransaction: Program library transactions

2. StageCode: Definition of stageCodes

Four-column reporting template

J. Mack Robinson College of Business / Georgia State University

1. Audit objective

2. Audit procedure

3. Results from executing queries

4. Reporting

• Findings from querying

• Recommendations, if any

• Data limitations, if any

J. Mack Robinson College of Business / Georgia State University

Organofood audit report

Audit objective

Audit procedure

Results from execution of

queries Exceptions

Statement of the audit objective

Explanation of the audit procedure for implementing the audit objective in terms of the data attributes

1. Names of queries

2. Statement of query results

3. Explana-tion of results

1. Findings from querying

2. Recommendations, if any

3. Data limitations, if any

Prerequisite skills and knowledge

J. Mack Robinson College of Business / Georgia State University

1. Querying proficiency

2. Familiarity with system development

3. Some experience developing audit objectives and procedures

Analysis tool choices

J. Mack Robinson College of Business / Georgia State University

• Microsoft Access QBE

• Audit software, e.g., ACL, IDEA

• SQL

Readiness questions: Sergeant Pabletti epiphany

J. Mack Robinson College of Business / Georgia State University

Saving Sergeant Pabletti: Video game played by 80,000 Army recruits each year

1. Recruits inept in first play of game

2. With learning, recruits save the sergeant

3. Vicarious experience for the twitch generation on the value of learning and thinking

4. Objective: Afford learners in IS audit a Saving Sergeant Pabletti epiphany about the value of thinking deeply about the situation

Organofood readiness question

J. Mack Robinson College of Business / Georgia State University

The data in OrganofoodProgLibrary.mdb permit verifying:

1. Completeness of requirements

2. Effectiveness of user participation

3. Completeness of quality assurance

4. The adequacy of the SDLC method

5. Separation of duties in development

Learning objectives

J. Mack Robinson College of Business / Georgia State University

1. Understand up the business situation

2. Perform audit steps

1. Develop audit objectives*

2. Design audit procedures

3. Execute audit procedures with querying

4. Communicate objectives, procedures, and results in a report

* Could be provided for AIS classes

References for situation model building

J. Mack Robinson College of Business / Georgia State University

• Barsalou. 1999. Language comprehension: Archival memory or preparation for situated action? Discourse Processes 28 (1): 61-80

• Zwaan and Radvansky. 1998. Situation models in language comprehension and memory. Psychological Bulletin 123 (2): 162-185

The task: Organofood audit

J. Mack Robinson College of Business / Georgia State University

Dysfunctional conditions to find:

1. Sales data not extrapolating to achieving volume discounts

2. Volume discounts booked at wrong percent (too high)

3. Developers performing incompatible duties for some components

4. eXtreme Programming (XP) more productive than a more traditional SDLC approach

Why are these conditions hard to find?

J. Mack Robinson College of Business / Georgia State University

1. Developing audit objectives requires integrating the concepts of system development objectives:

• Satisfying user requirements

• Being on time and on budget

2. The querying is complicated, requiring sequences of queries, cf. Hendrawirawan et al. 2007

Learner reaction to Organofood

J. Mack Robinson College of Business / Georgia State University

1. Liked readiness questions for calibrating understanding of the situation

2. Frustrated initially but eventually pleased with developing concept of “systematic and rational”

3. Appreciated opportunity to test their sense-making ability

4. Interested in how to develop better audit objectives in future audits

What is Organofood’s contribution?

J. Mack Robinson College of Business / Georgia State University

Give students practice in:

• Making sense of an authentic audit situation from conversation

• Developing system development audit objectives

• Querying and analyzing transaction data in an authentic audit situation

• Interpreting query results

Access

J. Mack Robinson College of Business / Georgia State University

Web staging of the simulation including a link to the database:

http://www2.gsu.edu/~wwwsys/pro/project/Organofood/site/Organofood.htm

Use name = ac863 and password = Qd0319