The Impact of English Language Testing on Medical

Registration Outcomes in Australia -

Evidence and Outcomes 2005-2011

Lesleyanne Hawthorne (Professor: International Health Workforce)

Anna To (Research Fellow)

AUSTRALIAN HEALTH WORKFORCE INSTITUTE

Faculty of Medicine, Dentistry and Health Sciences

IAMRA 2012 Ottawa

2-5 October 2012

Acknowledgements

Australian Health Practitioner Regulation Agency:

(2012)

English Language Skill Registration Standards Project

L Hawthorne & A To

Health Workforce Australia:

(2011)

Health Workforce Migration to Australia – Policy Trends

and Outcomes 2004-2010

L Hawthorne

Centre for Adult Education (Victoria)

(2010-2012)

Provision of Occupational English Test database 2005-2011 for the

above studies

Level of Reliance on Migrant Professionals by Field

Australia (2001 & 2006) and Canada (2001)

Occupation 2006 % Overseas-Born

In Australia

2001 % Overseas-Born

In Australia (cf Canada)

Engineering 52% 48% (50%) Computing 57% 48% (51%) Medicine 45% 46% (35%) Science 37% (36%) Commerce/ business 40% 36% (27%) Architecture 36% (49%) Accountancy 44% 36% (35%) Arts/ humanities 31% (24%) Nursing 25% 24% (23%) Teaching 25% 20% (15%) Source: 2001 and 2006 Census data analysis, Australia and Canada; The Impact of Economic Selection Policy on Labour Market Outcomes for Degree-Qualified Migrants in Canada and Australia, L Hawthorne, Institute for Research on Public Policy, Vol 14 No 5, 2008, Ottawa, 50pp

Migration pathways:

1. New Zealand flows (Trans-Tasman agreement)

2. International students (study-migration pathway)

3. Temporary labour migration (457 visa)

4. Permanent skilled migration (General Skilled Migration

Program)

5. Dependents of GSM migrants

6. Family and Humanitarian category migrants

Which English testing strategies? – to operate in an

increasingly complex registration environment

National Registration Context –

Sources of Medical Migration to Australia

Recent Trends in Medical Migration

2001-2012: Acceleration and diversification

Transition: Temporary migration now dominant (eg medicine – ‘area

of need’) = 4/5 of recent medical migrants/ immediate employment

Recent scale: Around 50,000 skilled arrivals every 5 years

2001-06 arrivals: Permanent arrivals double or triple from previous 5

years (most fields)

Nursing = 10,650 (compared to 3,100 in 1996-2000)

Medicine = 7,596 (compared to in 4,392 from 1996-2000)

Dentistry = 1,125 (compared to 540 in 1996-2000)

Pharmacy = 749 (BUT additional 2,080 selected as permanent skilled migrants

2004-05 to 2009-10)

Physiotherapy = 469 (additional 550 selected under GSM category alone 2004-05

to 2009-10)

Source: Analysis of ABS specialised matrices (2006 Census)

Top 10 Source Countries for Migrant Health

Professionals, All Fields (2005-06 to 2009-10)

Top 10 Permanent Source Countries: General Skilled Migration PA’s 2005-06 to 2009-10 (Total All Sources =13,880)

Top 10 Temporary Source Countries: 457 Long-Stay Business Visa PA’s 2005-06 to 2009-10 (Total All Sources =34,870)

1. UK: 4,120 2. India: 1,510 3. Malaysia: 1,300 4. China: 970 5. Philippines: 510 6. South Africa: 500 7. Republic of Korea: 480 8. Egypt: 420 9. Singapore: 390 10. Ireland: 350

1. UK: 9,350 2. India: 6,420 3. Philippines: 1,850 4. South Africa: 1,770 5. Malaysia: 1,570 6. Ireland: 1,560 7. China: 1,380 8. Zimbabwe: 1,180 9. Canada: 950 10. United States: 830

Source: L Hawthorne (2011), Health Workforce Migration to Australia – Trends and Outcomes 2004-2010, Health Workforce Australia.

Scale of Future Registration Demand?

Immigration Occupational Ceilings (2012-13)

Total:

Up to 29,880 migrant health professionals to be admitted

Select field:

Nurses: 15,660

Doctors: 4,560

Pharmacists: 1,380

Medical imaging professionals: 1,080

Dentists: 720

Physiotherapists: 840

Occupational therapists: 720

Optometrists and orthoptists: 360

Podiatrists: 300 (etc)

Additional health professional migration:

Uncounted family, humanitarian and spouse arrivals

Pre-Migration Screening of Economic Category:

Permanent (1999+) + Temporary Migrants

(Recent)

Re Economic =

MANDATORY

Family = X

Humanitarian = X

English Testing (IELTS/OET) = MANDATORY

Access to Medical Registration in Australia:

Australian Medical Council Examinations

Impact of AMC assessment (1978-10 outcomes):

Australian Medical Council MCQ Exam:

82% of candidates pass (most within 2 attempts)

AMC Structured Clinical Exam:

85% of candidates pass (as above)

Overall completion outcomes:

42.91% of commencing candidates (only 61% of IMGs

passing the MCQ proceeded to the Clinical Examination)

Impact of English language assessment:

Two tests: IELTS (Band 7) or OET (Grade B)

Occupational English Test = 52% pass rate by 2011 on

one or more attempts

IELTS?

English Language Skills Registration Standards

Specified purpose of English testing:

“… the English language test is basic competency to speak, to

listen, to write and to read. It does not deal with cultural

awareness, and it does not deal with issues about the use of

language in a medical cultural setting. That is supposed to be part

of the orientation that people get in the work setting when they

start work. It is supposed to orientate them to the cultural

situation, the workplace, and the particular needs of that context.”

J Flynn (Chair, Medical Board of Australia)

House of Representatives Standing Committee on Health and Ageing, Lost in

the Labyrinth – Report on the Inquiry into Registration Processes and Support

for Overseas-Trained Doctors, Canberra, March 2012

Key English Issues Raised by the 2010-12 House of

Representatives Inquiry into Overseas Trained

Doctors

1. The standards required for professional registration

2. The scope of tests (to ensure practitioners possess the

communication skills required for clinical practice?)

3. Tests’ stability, reliability and predictive capacity

4. The length of validity of test results

5. The requirement to pass all four sub-tests at a single

sitting)

6. The quality of rater training (perceived variability)

7. The dearth of feedback provided to candidates (to

support improved test performance)

8. The impact of ‘invisible’ language tests (such as the

Pre-Employment Structured Clinical Interview and

specialist college assessment)

Policy Challenges –

For Example Standards Required by Field?

2012:

48 Australian regulatory bodies set IELTS standards for

registration

Range = Band 6 to Band 8

Research base – registration for the health professions:

Lacking or slight

Impacts on registration (all fields):

Profound

Rationale?

Eg in health highest global levels required for pharmacy

Language testing expert:

“IELTS 7 gets specified for medicine, then all other fields

feel they should adopt it as the gold measure.”

Perspectives:

Global Key Informants (Seven Countries)

Specification of which tests/ standards:

Typically ‘inherited’

Rationale:

Largely unknown (eg ‘my predecessor would know’…. ‘lost in

the mists of time’)

Operational drivers

Evidence base:

Uncertain

Reliance on ‘expert advice’: Which instrument/s?

Fitness for purpose?

Validity, reliability, predictive capacity?

Specific requirements?

Occupational English Test Database: Analysed by

Key Variables January 2005 to December 2011

Total Candidates

Field 2005 2006 2007 2008 2009 2010 2011 Total

Medicine 700 1262 1438 1180 1366 1214 1079 8239

Nursing 696 551 943 2339 2573 1917 1550 10569

Dentistry 195 653 1119 1350 1562 1293 1234 7406

Physiotherapy 13 25 29 43 90 112 116 428

Pharmacy 128 214 302 345 402 361 194 1946

Allied health

(other) 28 23 37 40 77 63 68 336

Grand Total 1760 2728 3868 5297 6070 4960 4241 28924

Proportion of Candidates Passing OET by Field and

Gender (2011)

60%

17%

66%

36%

46%

52%

40%

44%

16%

55%

22%

32%

17%

42%

0%

10%

20%

30%

40%

50%

60%

70%

Can

did

ate

pass r

ate

(%

)

Female

Male

Proportion of OET Candidates Passing:

Medicine Compared to Other Major Fields

(2005, 2010, 2011)

53%

20%

40% 38% 39%

46%

37%

43%

19%

47%

34%

28%

32% 34%

52%

17%

62%

32%

38% 40% 41%

0%

10%

20%

30%

40%

50%

60%

70%

Can

did

ate

pass r

ate

(%

)

2005

2010

2011

Requirement to Pass All 4 Sub-Tests at a

Single Sitting

Introduction: 2010+

Impacts: Very significant

Source of recommendation: IELTS (operational drivers)

Evidence base?

Profound disagreement this is warranted (OET opposed)

Research (slow language attrition occurs at higher levels/ no

test coherence imperatives)

Policy outliers:

Nursing Council of NZ (successive attempts)

Candidates: Up to 20 reported attempts

Language testing = Global big business

Impact of the Requirement to Pass All Four Sub-

Tests at a Single Sitting, by Select Source Country

(2011 Outcomes)

Field Country of

Origin

Candidate

Pass Rate Number of

Candidates

1989-1995 2011 Differential 1991-

1995 2011

Medicine

China 59% 40% -19% 191 55

India 91% 63% -28% 200 75

Philippines 78% 36% -42% 93 11

Egypt 84% 49% -35% 126 57

Russia 77% 51% -26% 133 35

Nursing

Philippines 41% 11% -30% 413 193

India 55% 7% -48% 51 121

Hong Kong 50% 44% -6% 96 9

Fiji 70% 0% -70% 37 10

OET Sub-Test Pass Rates for Medical Candidates

(2005 and 2010): Major Challenge to Pass 4 Tests at

Single Sitting

63% 66%

62%

67% 64%

67%

70% 69%

65% 68%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Speaking Listening Reading Writing Medicine Total

Su

b-t

est

pass r

ate

(%

)

2005

2010

Candidate OET Pass Rates for Medicine by Top 10

Countries of Training (2005, 2010, 2011)

68%

58% 61%

35%

45%

53%

62%

49%

62%

53%

71%

54%

67%

47%

39%

45% 48%

43%

27%

36%

44% 45% 46%

51%

63% 59%

54% 54% 53% 52% 51% 51%

49% 49% 48%

40%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Can

did

ate

pass r

ate

(%

)

2005

2010

2011

Impact of Requirement to Pass All Four Sub-Tests at

One Sitting on Pass Rates (All Fields, 2005 and 2010)

43%

100%

53%

46%

31% 35%

46% 53%

26%

44% 49%

38% 37%

66%

53%

44% 43% 43% 43% 43% 40% 39% 33%

29% 27% 24%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Can

did

ate

Pass R

ate

(%

)

Country of Training

2005

2010

Only countries with 50 or more candidates in 2010 are displayed * Denotes instances where there were fewer than 10 candidates in 2005

Impact of Requirement to Pass All Four Sub-Tests at a

Single Sitting on Candidate Pass Rates by Select Field:

Nursing (2005 and 2010)

100%

0%

36%

100%

8%

34% 33%

20% 20% 25%

58%

42%

24% 23% 22% 21% 20% 17% 17%

13%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

United Kingdom *

Germany * Singapore Korea, South *

China Philippines Nepal * All other India Japan*

Nu

rsin

g ca

nd

idat

e p

ass

rate

(%

)

Proportion of nursing candidates passing OET by select country of training, 2005 and 2010

2005

2010

Average Number of Test Sittings for IMGs by

Top 10 Source Countries (2005, 2010, 2011)

1.4

1.3

1.4 1.4

1.2

1.3

1.4

1.5

1.4 1.4

1.3

1.4

1.6 1.6 1.6

1.4

1.7

1.5 1.5 1.5

1.8

1.5 1.4

1.4

2.2

1.9 1.8

1.8 1.8 1.8 1.7

1.6 1.6 1.6 1.5

1.4

0.0

0.5

1.0

1.5

2.0

2.5

Avera

ge n

um

ber

of

test

sit

tin

gs

2005

2010

2011

Test Exemptions?

Case Study 1 – Native English Speakers

Limited research base, which demonstrates:

Native speakers may perform poorly (reading and writing)

Many would score IELTS 5-IELTS 6.5

Many would not be deemed ready for tertiary study in terms of

English

Many would not pass all four sub-tests at a sitting

Results tend to correlate to education level and field status (eg

barristers out-perform teachers who out-perform year 12

students)

Case study:

Nurse OET outcomes to 2011 - UK and South Africa nurses

Native English Speaker Performance on the OET

(All Major Fields)

43%

100%

53%

46%

31% 35%

46%

53%

26%

44% 49%

38% 37%

66%

53%

44% 43% 43% 43% 43% 40% 39%

33% 29% 27%

24%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Proportion of candidates passing OET by select countries of training for all fields, 2005 and 2010

2005

2010

Only countries with 50 or more candidates in 2010 are displayed * Denotes instances where there were fewer than 10 candidates in 2005

Scale of Potential International Student Demand for

Test Exemptions (1996-2009 Enrolments by Select

Field)

Field 1996 2000 2002 2004 2008 2009

Dental science 98 124 155 227 341 387

Medicine 963 1117 1287 1505 2665 2772

Nursing (basic) 762 839 790 1623 5451 6124

Nursing (post-basic) 545 2336 3591 3109 2631 2566

Physiotherapy 79 173 197 239 392 365

Case Study 2: Test Exemptions for International

Students Qualified in Australia?

Issues:

Institutional dependence on export education: Revenue

stream

Academic entry and progression standards: Institution-

controlled

Nursing: Primary migration pathway

Level of English improvement in life of course? Uncertain

Rate of improvement: Typically 0.5 IELTS Band in 3 month

intensive English program

IELTS Research Reports (university students):

Modest gains: In 3-5 years of Australian study

Variability of outcomes: May improve/ plateau/ even decline

UK/ Ireland position:

No defensible evidence base for defining English testing

exemptions for registration (reportedly accepted by applicants)

Current Global Reviews

1. PLAB 2012 (UK): Designed to ensure the test is ‘effective, objective, fair, and non-

discriminatory’ in assessing that IMGs have ‘the knowledge and

skills to work as doctors’

Support standards required by longitudinal research on IMGs

relative to other groups in UK practice

2. NCSBN (US): (2005) Updated TOEFL and IELTS score recommendations to

60 Boards

(2009) Correlated TOEFL standards to TOEFGL iBT through 22

member expert panel of stakeholders

3. Medical Technicians 2009 (Canada): Definition of standard setting protocol for fields, plus attempted

benchmarking standards across 3 tests (IELTS, TOEFL, MELA)

Called for longitudinal research to track candidates and

establish external validity as well as advise on policy changes

Current additional Canadian work: Pharmacy, Physiotherapy

Select Australian Policy Challenges

1. Impact of English testing: On scale and sources of workforce supply

On different health fields

2. Issues raised in House of Representatives Inquiry into

OTDs (2011-12): Procedural fairness? Policy consistency? Transparency?

Appropriateness?

3. Select policies – justified? rationale? Special requirements (eg 4 tests passed at one sitting)

2 year validity period

One size fits all (common standards for all fields)

Few accepted tests (2)

Reliability of ‘high stakes’ instruments (in the context of test design and

outcomes) – etc

Major life stakes = English testing the major barrier to

professional registration