Testing times: can wedetect the doctor indifficulty?Alice Stewart, Department of Neonatology, Monash Medical Centre, Melbourne, Australia

Mitchell C, Bhat S, Herbert A, Baker P.

Workplace-based assessments of junior

doctors: do scores predict training

difficulties? Med Educ 2011;

45:1190–1198.

Workplace-based assessments(WPBAs) are now a mainstay ofcompetency and training out-come assessment in postgraduatemedical training. This is a uni-versal phenomenon, with trainingprogrammes in the UK, Europe,North America and Australiameasuring performance withWPBAs. However, there is noreference standard test by whichto measure competency, anddespite their widespread use forthis purpose, it is not known howWPBA results are associated withthe presence or the lack of com-petence. In this study, Mitchelland colleagues set out to evalu-ate the ability of WPBA to detect

foundation programme (FP)doctors with identified trainingdifficulties.

Using a retrospective observa-tional approach, the authorsextracted and anonymisede-portfolio WPBA data for all FPtrainees in a single UK deaneryover 11 placement periodsbetween 2005 and 2009. Theycollected data on 75 580assessments in 1646 trainees,and calculated the mean scoreachieved by each trainee on fourdifferent types of WPBA: mini-peer assessment tool (mini-PAT);mini-clinical evaluation exercise(mini-CEX); direct observation ofprocedural skills (DOPS); andcase-based discussion (CbD). Inthis period, 92 trainees hadbeen identified as ‘in difficulty’by the deanery. This was defined

as someone for whom the edu-cational supervisor–traineeinteraction was insufficient toresolve an issue of professionalcompetence, requiring notifica-tion of the deanery. Difficultywas classified by the deaneryaccording to one or moresubtypes: conduct; health; andperformance.

For two types of WPBA, the mini-CEX and CbD, there was a statis-tically significant associationbetween mean score and beingidentified as being ‘in difficulty’.However, when it came to pre-dicting the trainee in difficulty, theWPBA scores were only just betterthan ‘tossing a coin’. On subanaly-sis, performance-related (but nothealth- or conduct-related)difficulties were associated withsignificantly lower mean scores

It is not knownhow WPBAresults areassociated withthe presence orthe lack ofcompetence

Digest

� Blackwell Publishing Ltd 2012. THE CLINICAL TEACHER 2012; 9: 265–268 265

on mini-CEX and CbD, but thesescores in turn were only weaklypredictive of ‘in difficulty’ status.

Mitchell and colleagues concludedthat WPBA is not a valid methodof screening for trainees in diffi-culty. One of the limitations ofthis study is the reliability of thereference standard: identificationof the trainee in difficulty byeducational supervisors. Becauseof its subjective nature, it is likelythat the true proportion of train-ees in difficulty is underestimated(although the number is stilllikely to be small). In particular,those with health-related diffi-culties may fall under the radar.

Fox and colleagues found that afear of stigma and of academicfailure is a significant barrier todisclosure and finding support forFP trainees with health problems.1

The WPBAs themselves are notwithout difficulty. Their validity isinfluenced by the observer–trai-nee relationship, and a traineealready aware of their ‘in diffi-culty’ status may selectivelychoose cases and observers inorder to maximise their WPBAscores.

Mitchell and colleagues offer aword of caution. Despite real-lifeexamples of WPBAs being pilotedin summative assessment, there is

insufficient evidence that theycan be relied upon as certifyingcompetence. Instead, WPBAsshould continue to be used bothin formative assessment and inunison with other assessments togive a holistic measure of profes-sional competence.

REFERENCE

1. Fox FE, Doran NJ, Rodham KJ, Taylor

GJ, Harris MF, O’Connor M. Junior

doctors’ experiences of personal

illness: a qualitative study. Med Educ

2011;45:1251–1261.

doi: 10.1111/j.1743-498X.2012.00588.x

Supporting the traineedoctor with personalillness: is there a cure?Alice Stewart, Department of Neonatology, Monash Medical Centre, Melbourne, Victoria,Australia

Fox FE, Doran NJ, Rodham KJ, Taylor GJ,

Harris MF, O’Connor M. Junior doctors’

experiences of personal illness: a

qualitative study. Med Educ

2011;45:1251–1261.

A research group at the Universityof Bath has investigated theexperience of personal illnessamongst trainee doctors. Theprevalence and burden of illnessin this population is poorlyunderstood. Although workingconditions and training demandsare known to have a negativeimpact on health, there is littleunderstanding of the experiencesand support needs of traineedoctors who are unwell. In re-sponse, Fox and colleaguesexamined this issue in foundation

programme (FP) trainees in adeanery in south-west England.

This qualitative study was spon-sored by the Severn Deanery, butto ensure anonymity, data col-lection and analysis were inde-pendently performed byUniversity of Bath researchers. Anemail was sent to all FP traineesin the deanery, inviting them toparticipate if they had ‘experi-enced significant illness’ duringtheir medical training. Ten doc-tors responded and eight con-sented to participate. Theyreported a range of health condi-tions: medical, mental, malig-nancy and surgical. A semi-structured interview with open-

ended questions was conductedwith each participant by the leadresearcher. The audiotaped inter-views were transcribed, anony-mised and coded. The followingmaster themes were identified:illness experience; crossing theline; medical culture; stigma;disclosure; and support.

The illness experience challengedthe trainees’ perceptions of theirsocietal role and assumed invul-nerability to illness. The roletransition from ‘well’ doctor to‘unwell’ patient was conceptua-lised as crossing the line. Someparticipants reported increasedempathy for their patients as aresult of their own illness

The illnessexperience

challenged thetrainees’

perceptions oftheir societal

role

266 � Blackwell Publishing Ltd 2012. THE CLINICAL TEACHER 2012; 9: 265–268