Sampling and small populations Ljubljana, 12-13 October 2009

1

Ljubljana, 12-13 October 2009

Homologues Group Meeting

Slovenia, October 2009

Republika SlovenijaEuropean Union

Sampling and small populations Ljubljana, 12-13 October 2009

Sampling and small populations Ljubljana, 12-13 October 2009

Henk W. H. Geilen RE RA CISA Senior auditmanager

Dutch Audit Authority

Henk W. H. Geilen RE RA CISA Senior auditmanager

Dutch Audit Authority

2





OutlineOutline

1. Introduction

2. Why Sampling

3. What is Sampling

4. “Small populations”

5. Conclusion

3





Why sampling ?• A true Story

• A long time ago……….

4





5





What’s the problem ?• Situation A (no errors)

• What to do?

• Stop?

• Seen Enough?

6





What is Sampling (2)•Situation B (errors found)

•What to do?

•Take more items?

•Correcting only the errors found?

•……

7





So…..•You never know whether you have seen enough items

•So did you do “enough” work?

• “Enough” means:

•Not to few

•Not to much

8





Mathematics ?•Total number of balls

•Number of red balls

•Chance : Red /total

•Number of draws

• If you put the balls back after drawing

•Multiply chances for each draw

9





Mathematics/Formula•Red/(Red + White) * Red/( Red + White)…

•Number of draws: n

•Red / (Red +White) = Red / Total = p

•Chance: ß

•ß = pn

10





So What ?•Audit

•We know “total”

•We don’t know “error” p

•We don’t want to know “Chance” ß

•…..

11





Audit•The “object” is good

•We mean the “object" is good ENOUGH

•So there can be a little error

•Let’s call it MATERIALITY

•Hé isn’t that p

12





Chance•100% assurance

• Is this possible ?

• Is this necessary ?

•The auditor defines his risk

•This means: the chance that he accepts the population while the error is higher than acceptable

13





Where are we ?• We have a formula• ß = pn

• Let’s say ß = 0,05• So the risk is 5% or the Probability = 95 %• Say the materiality = 1 % (0,01)• Then we can calculate n = the number of samples• 0,05 = (1 - 0,01)n

• N = 300

14





This means• If you want to accept 1 percent errors (max)• You sample 300 times• You find zero errors• The risk is (less than) 5 %• or• If you take a sample of 300 items and find 0 errors you know 95 % sure that the maximum error in the population is 1 percent.

15





Playing with n•Different materiality:

• f.e. 2 percent n = 300/ 2 = 150

• f.e. 0,5 percent n = 300 / 0,5 =600

•Different risk:

•10 percent : n * p = 231

•15 percent : n * p = 190

16





Based on 0 errors•Risk 5 percent

•1 error : n * p = 475

•2 errors : n * p = 630

•3 errors : n * p = 776

•4 errors : n * p = 916

•5 errors : n * p = 1052

17





Risk•ß (Beta) Risk :

The risk that you accept the population on the basis that the error is less or equal than the materiality

•α (Alpha) Risk :

The risk that you don’t accept the population while in fact there is no error

18





What is important•Random / A-select

•Meaning : each element has an equal change of being sampled

•The sampled set has the same characteristics as the population

19





First ConclusionsSampling :

• A method to calculate how much work to do

Parameters :

• Materiality

• Confidence level

• Not size of the population

20





Back to small populations• The questions are/stay:

• How many items (operations) to audit

• What items (operations) to audit

• How to audit an item (operation)

• Non statistical sample

21





How can we do it (how much)?• Step 1: Audit all “big” items (big means >

materiality)

• Step 2: Calculate sample size for other items based on results system audit

• This defines the confidence level

22





In detailTotal population (all operations)

Minus Big operationsIs Population to SampleMultiply Confidence LevelIs “temp result”Devide (Population to sample) multiply Materiality

Is Sample Size

23





Assurance level from systems audit

Reliability Confidence Level

Factor

Only minor improvements

High 60 % 0,92

Some improvements

Average 70 % 1,21

Substantial improvements

Average 80 % 1,61

Does not work Low 90 % 2,31

24





Example141.596.219 (393 operations)

Minus 15.312.795 ( 4 operations)Is 126.283.424Multiply 1,21 (70 %)Is 152.802.943Divide 2.525.668 (0.02 *126.283.424 )

Is 60 Sample Size (plus 4)

25





Which items (1)• Random• Verify• Has sample same characteristics as

population• Average..• MA, Region etc• Looks like risk model Commission

26





What items (2)• Not the same characteristics =

• Draw again

• After a few draws ….

• Expand sample

27





After the auditTotal error fractions

Divide sample sizeIs average error fractionMultiply population to sampleIs projected errorPlus error in big operationsIs Total error

28





Evaluation• Calculate materiality

• Compare with total error

• Example

• 0,02 * 141.596.219 = 2.831.924

• Vs 5.096.546

29





What can you influence ?

141.596.219 (393 operations)Minus 15.312.795 ( 4 operations)

Is 126.283.424Multiply 1,21 (70 %)Is 152.802.943Divide 2.525.668 (0.02 *126.283.424 )

Is 60 Sample Size (plus 4)

30





Reminder• What was the goal?

• Sample?

• Audit?

• Audit Opinion?

31





Conclusions• No miracle solutions (Mathematic not Magic)• basic principles equal to statistic sampling• definition of the audit objective• definition of the population• definition of the characteristics to test• define confidence level, materiality• random selection

32





More conclusions• non-statistical sample size rather large to support

audit judgment• consider increasing tests of controls to determine

confidence level for sampling• difficult to determine precision of error projection• representativeness of the sample needs to be

evaluated

33





Thank you for your attention!Thank you for your attention!

Henk GeilenHenk Geilen

[email protected]@rad.nl+31 6 3100 7794+31 6 3100 7794

Sampling and small populations Ljubljana, 12-13 October 2009

Documents

Transcript of Sampling and small populations Ljubljana, 12-13 October 2009