1 1

Emerging Adoption of E-Assessment in the Professions Emerging Adoption of E-Assessment in the Professions

Garrett Sherry

Country Manager - Ireland

2 2

Emerging Adoption of E-Assessment in the Professions Emerging Adoption of E-Assessment in the Professions

• Thomson Prometric – Body of Experience• External Factors Driving Adoption• Changing Business Needs • Emerging Developments • A Few Cases In Point• Review & ‘Our’ View

3 3

Thomson Prometric – A Body of Experience Thomson Prometric – A Body of Experience

• 1990 1st Computer Based Exam for Novell

• Global Leader in Technology Enabled Assessment Services

• 600 Clients, 132 Countries, 8 Million Exams every year

• Practical, Paper & Pencil, Internet & Computer Based Solutions

• Test Development, Delivery and Data Management

• Professional, Academic, Government, Corporate & IT

• Ireland - 60 Test Locations, 200,000 Exams

- Customers; Department of Transport, Medical Council of Ireland,

CITB NI, Microsoft, Cisco, Dell, Oracle…

4 4

External Factors External Factors

• Changes in Regulatory & Legal Environment

• Rapidly Evolving Curriculum

• Candidate ‘Test-on-Demand’ – The ‘Now’ Generation

• Globalisation of Programmes

• Competition for Candidates

5 5

Changing Business Needs Changing Business Needs

• Develop / Change Programmes Quickly

• Increased Availability & Test Windows– Quicker Results

• Improved Security, Integrity & Easier Administration

• More Relevant Exam Experience

• Improve ROI – Develop New Revenue Streams

6 6

Independent Research on UK Attitudes to e-AssessmentIndependent Research on UK Attitudes to e-Assessment

• Drivers & Barriers to Adoption of e-Assessment

• Attitudes to QCA Proposals for e-Assessment

• Acritas Research Commissioned to Independently Research 116 UK Awarding Bodies

• Qualitative - Face to Face Interviews

• Quantitative Study – Confidential Telephone Interviews

7 7

Qualitative FindingsQualitative Findings

• A Highly Receptive Environment – Innovative Working Examples

• A need to Develop Sophisticated, Robust and More Scalable

e-Assessment Examples

• The Needs and Wants of Learners Should be Central to Implementation

• Business Continuity is vital to e-Assessment

• E-Assessment should include Vocational and Professional Qualifications – Not Just Schools

8 8

Quantitative FindingsQuantitative Findings

• 26% Currently Deliver e-Assessment

• Of the 74% not delivering e-Assessment, 46% are developing / piloting

• 83% Believe e-Assessment More or Equally Robust as P&P

• 60% Believe e-Assessment More Cost Effective than P&P

• Client Satisfaction and On-Demand Testing are Most Important Factors

• 78% Would Prefer Multiple Technologies Conform to Agreed Standards

9 9

What are the three main advantages of using e-Assessment, in your opinion, overall? (unprompted) What are the three main advantages of using e-Assessment, in your opinion, overall? (unprompted)

0%

10%

20%

30%

40%

50%

60%

70%

80%

Overall Currently delivering Currently not delivering

10 10

What drives uptake of e-Assessment?What drives uptake of e-Assessment?

0%

10%

20%

30%

40%

Costs/

savin

gs

Exter

nal/Q

CA/Gov

ernm

ent p

olicy

/agen

da/ta

r...

Speed

of r

esul

ts/fe

edba

ck

Geo

graph

ic sc

ope/a

cces

sibilit

y

Conve

nience

/tim

e/av

ailab

ility/o

n de

man

d

Technolo

gy im

prove

ment

s

E' is c

urre

nt fas

hion

/tren

d/th

e fu

ture

Easy

to m

anage

by A

B's

Flexib

ility

(uns

pecif

ied)

Stand

ardisa

tion/

cons

isten

cy in

mar

king

Deman

d led

Overall Currently delivering Currently not delivering

11 11

Case StudiesCase Studies

12 12

Securities and Investment InstituteSecurities and Investment Institute

Customer Issues & Needs

• UK Awarding Body, 40,000 Exams Per Year & Rising• Paper & Pencil Delivery• Complex & Costly Administration• Weeks to Get Results• Candidate frustration - On-Demand Needs• Increasing International Focus• Revenue Growth

13 13

Securities and Investment InstituteSecurities and Investment Institute

Conversion to CBT

• Expanded delivery – 60 Locations, 30 Countries • On-Demand Continuous Testing• Immediate Results – Re-Test in 3 days • Administration Efficiencies • On-line Self Service Booking• Enhanced Candidate Experience• Flexibility to Change and Add Programs Quickly• Expanded Revenue Opportunities

14 14

Medical Council of Ireland Medical Council of Ireland

Temporary Registration Assessment Scheme (TRAS)- Customer Issues & Needs

• Paper & Pencil • Fixed Exam Delivery Dates• Limited Overseas Delivery• Increasingly Complex Administration• Difficult & Costly to Expand Delivery (especially overseas)• Cumbersome to Change & Update Exam

15 15

Medical Council of Ireland Medical Council of Ireland

Temporary Registration Assessment Scheme (TRAS)- Conversion to CBT

• Available On-Demand - Test When Ready • Flexible Global Delivery – 4 Countries, 15 Locations• Simplified Scheduling & Registration• Control of Eligibility Criteria• Robust High Specification MCQ Exam• Consistency of Exam Experience• High Level of Security and Integrity• Ability to Change & Update Exam Easily

16 16

American Institute of Certified Public Accountants (AICPA)American Institute of Certified Public Accountants (AICPA)

The Uniform CPA Examination- Customer Issues & Needs

• Paper & Pencil – Twice a Year• Limited Locations – Large and Costly Auditoriums• First Time Candidates – Two Day Exam• Very Stressful Environment for Candidates and Invigilators• Limited Ability to Simulate Real-World Skills• Need for a More Flexible, Seamless Candidate Experience to

Help Attract New Talent• A 21st Century Exam for 21st Century Practices & Demands

17 17

American Institute of Certified Public Accountants (AICPA)American Institute of Certified Public Accountants (AICPA)

The Uniform CPA Examination- Conversion to CBT

• Partnership Approach to Conversion (Steering Group)• Flexible Candidate Options – Whole Exam or Sections• Year Round Delivery – Test When Ready - 300 Locations • Fit for Purpose Test Centre Network – Secure, Comfortable• Innovative Test Items, Simulations and Tools• Greater Public Confidence in Competency and Ethical

Proficiency of CPA’s• A 21st Century Exam

18 18

19 19

20 20

21 21

22 22

23 23

24 24

Emerging Adoption of E-Assessment in the Professions Emerging Adoption of E-Assessment in the Professions

• Increased Availability & Test Windows– Quicker Results

– The ‘Now’ Generation• Improved Security, Integrity & Easier Administration• Increasing Competition for Candidates • More Relevant Exam Experience• Consistent Global Administration• Improved ROI / New revenue streams / Expansion of programs • Outsourcing Test Delivery & Administration • Concentrate on Developing Core Programmes• Improved Security & Integrity of Testing Process

25 25

Emerging Adoption of E-Assessment in the Professions Emerging Adoption of E-Assessment in the Professions

Thomson Prometric ‘Our View’

Professional bodies and other responsible organisations in Ireland should embrace the advantages offered by technology enabled learning and assessment to ensure:

That those who look after our sick are properly qualified

Those who look after our money have an appropriate to licence to practice and,

That education for the professions better reflects the realities of today’s business and regulatory environment