1

Formative / Summative• Should an assessment do both?• Can an assessment do both?

2

Who does PTs ?• Germany – Aachen, Berlin, Bochum, Koln, Munster, Witten• Netherlands – Maastricht, Njimigen, Utrecht, Groningen, Leiden.• Belgium – Ghent• UK – Manchester, Peninsula,Keele• Finland – Tampere, Helsinki• Canada – McMaster.• New Zealand – Otago• USA – Kansas• South Africa – Pretoria• Ecuador – Quito• Brazil – San Paulo

3

Logistics - Delivery• Delivery Methods• Paper• All together?• Venues• Delivery of Results

4

Logistics - Delivery• On Line ?• Security• Equivalent tests

5

Results• How, when and where.• Automate as much as you can• Students want them tomorrow• If summative then performance must be reviewed

before release of results.

6

Progress Decisions• For summative how tight can you make them• Based on relative decisions• Is that fair?

2 4 6 80

10

20

30

40

50

60

70

80%Correct

Tests

1

6

5

4

3

2

R

‘78

‘79 ‘81‘80

‘83‘82 ‘85‘84 ‘86 ‘87 ‘88

Progress Test Results

Universiteit Maastricht

scoring

mean

- ½ sd

- 1 sd

D S SU

mean

-0.5 s.d.

-1 s.d.

9

Results

Year 2 Distribution of Scores

0

2

4

6

8

10

12

14

16

0 5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

100

Score

Fre

quen

cy

Universiteit Maastricht

scoring

mean

- ½ sd

- 1 sd

B S SU

15th percentile.

5th percentile

E

95th percentile

Scoring

• Bottom 5 % - Unsatisfactory grade

• Next 10 % - Borderline

• Remainder Satisfactory, except top 5% graded Excellent

But !This is a longitudinal assessment and

individual test scores aggregate. Decisions are made on aggregate scores NOT individual tests

Aggregation

Test 3 Aggregate Test 4 ScoreNew Aggregate

Excellent Satisfactory

Doubtful Satisfactory Satisfactory

Borderline Doubtful

Unsatisfactory Unsatisfactory

Results

• Students receive results electronically

• Atlas

Student Results

Progress decisions

• End of year decisions

• Aggregate satisfactory progress

• Aggregate unsatisfactory do not progress

• ?

• Decision rules, but often reviewing profiles

Progress Test – Z scores

90000716

S SS

S

S

S

SS

S

D

-3

-2

-1

0

1

2

T2 T3 T4 T5 T6 T7 T8 T9 T10 T11

Progress Test – Z scores

u

DS S

D

U

D D

U

UU-3

-2

-1

0

1

2

T2 T3 T4 T5 T6 T7 T8 T9 T10 T11

Progress Test – Z slope

-3

-2

-1

0

1

2

3

PT6 PT7 PT8 PT9 PT10 PT11 PT12 PT13 PT14 PT15

Progress Test

0

5

10

15

20

25

30

35

40

PT1 PT2 PT3 PT4 PT5 PT6 PT7 PT8 PT9 PT10 PT11 PT12 PT13 PT14 PT15

Per

cent

age

Progress Test Repeat

0

5

10

15

20

25

30

35

40

PT

1

PT

2

PT

3

PT

4

PT

5

PT

6

PT

7

PT

8

PT

9

PT

10

PT

11

PT

12

PT

13

PT

14

PT

15

Pe

rce

nta

ge