Gill Clarke, University of Bristol and Stuart Powell, University of Hertfordshire

Quality and Standards

of Postgraduate

Research Degrees – 2009

First published in 2009 by the UK Council for Graduate Education

© UK Council for Graduate Education

ISBN 978-0-9543915-7-7

3

Foreword 4

Executive Summary 6

1. Introduction 7

National Reports and Initiatives in the UKInfluence of Europe

2. Some Contextual Factors 9

Rising demand and competition for doctoral candidatesocus on Quality of provisionCompletion RatesPart-Time StudentsResponding to the Needs of Industry and Commerce – TransferableSkillsConcern for Quality

3. What is ‘Quality’ in Postgraduate Research Education 17

Definitions of qualityPurposes of the doctoral qualificationNon-traditional doctoratesThe doctoral programmeExamining the doctorate

4. The Assessment of Quality 28

5. International Dimensions 29

6. Conclusions 29

References 31

UKCGE Published Titles 33

Contents

The original paper on the quality and standards ofpostgraduate research degrees published in 1996was only the second in the Council’s series ofpublications. It was prepared by a Working Groupestablished by the Council and Chaired byProfessor Michael Harloe, University of Essex. Theintention of the report was to assist institutions todebate quality issues that were seen as central tothe development of postgraduate education andtraining in the UK.

The Council has recently begun to update itsreports as circumstances have required. In anumber of cases this has involved conducting asurvey to gather new data on a continuing issue,for example the use made by member institutionsof the award of PhD by Published Work, as a meansto providing a view on developments in the field.However the original report on Quality andStandards was not based on a survey but ratherrepresented the views and discussions of theworking group. In revisiting this topic, we thereforedecided not to try to ‘revise’ the earlier work – itstands as a useful statement of the group’sdeliberations and we would not wish to try to

recreate those deliberations or indeed ‘update’them in any way. Rather the Council asked twoexperts in this area (Professor Stuart Powell of theUniversity of Hertfordshire and Gill Clarke of theUniversity of Bristol and formerly the QAA) to offersome reflections on the current situation withregard to quality issues. There are some newconsiderations – significantly the revised section 1of the QAA Code of practice for the assurance ofacademic quality and standards in highereducation: Postgraduate research programmes(‘the QAA Code’) – that were simply not in place in1996. Where relevant the authors of this newreport have made use of points made in theoriginal publication as a reference point for currentprinciples and practices.

Professor Malcolm McCraeFebruary 2009Chair, UK Council for Graduate Education

4

Foreword

The UK Council for Graduate Education (UKCGE) was established in 1994 to

promote the interests of graduate education across all disciplines in higher

education institutions in the UK. It currently has some 125 institutional

members. This report on Quality and Standards of Postgraduate Degrees –

2009 is one of a series of publications from the Council investigating key

issues in graduate education.

Quality and Standards of Postgraduate Research Degrees

5

The UK Council for Graduate Education is an organisation established to

promote the interests of graduate education in all disciplines in higher

education institutions. The Council was established in 1994 and has over

125 institutional members.

For further information about the UK Council andits activities, please contact:

The Principal OfficerUK Council for Graduate EducationLichfield CentreThe FriaryLichfieldStaffordshireWS13 6QG

T: 01543 308602F: 01543 308604E: ukcge@ukcge.ac.ukwww.ukcge.ac.uk

A list of the Council’s publications is given at theback of this book.

As with the original ‘Quality and Standards’ Report(UKCGE, 1996), this paper seeks to identify thecurrent context within which doctoral leveleducation is being undertaken in the UK andreview the kinds of developments that are takingplace that affect the quality of that education. Thepaper revisits some of the thematic issues thatwere referred to in 1996, namely the purposes of adoctoral education, the development of so-called‘non-traditional' doctorates, assessmentmechanisms and international aspects of doctoraleducation. It also addresses some issues anddimensions that were not referred to in 1996 (i.e.issues that had not come to the forefront ofconcerns at that time), namely:

� the impact on UK research degree programmesof wider European developments, including theBologna Declaration (third cycle qualifications)and related agreements such as the Frameworkfor Qualifications of the European HigherEducation Area and the Salzburg Principles1

(and see 1.2 below)� the overall requirements of Research Councils

and the development of their Joint SkillsStatement2

� the establishment of the Arts and HumanitiesResearch Council

� the increased focus on the quality assuranceand management of research degreeprogrammes, including the annual HEFCEsurvey of research degree completion rates3

� recognition of the true resource implicationsfor institutions in providing research education,while recognizing their centrality to researchquality4

The paper is intended to promote furtherdiscussion across the sector regarding researchdegree education. It is not intended as asupplement to the revised QAA Code of Practice(QAA, 2004), nor as a critique of that Code; rather itrepresents the views of the authors about howresearch education in the UK has developed in thelast five to seven years, in the context of variousreference points, of which clearly a significantelement is the Code and its implementation.

The views expressed in this paper are those of theauthors and are not necessarily those of the UKCouncil for Graduate Education or any otherorganisation or institution.

6

Executive Summary

1 A Framework for Qualifications of the European Higher Education Area, Ministry of Science, Technology and Innovation,Denmark, February 2005, ISBN: 87-91469-54-6 and ISBN (Internet): 87-91469-53-8 and outcomes of the Bologna Seminar:Doctoral programmes for the European knowledge society, Salzburg, 3-5 February, 2005, General rapporteur’s report:http://www.eua.be/eua/jsp/en/upload/Salzburg_Report_final.1129817011146.pdf

2 UK Research Councils, Joint Statement of Skills Training Requirements of Research Postgraduates (2001):http://www.grad.ac.uk/cms/ShowPage/Home_page/Policy/National_policy/Research_Councils_training_requirements/p!eaLXeFl#Joint%20Statement%20of%20Skills%20Training%20Requirements%20of%20Research%20Postgraduates%20(2001)

3 HEFCE Research Degree qualification rates, first annual publication, Issues paper, October 2007/29 (web only):http://www.hefce.ac.uk/pubs/hefce/2007/07_29/

4 Costs of training and supervising postgraduate research students – a report to HEFCE by JM Consulting Ltd, February 2005:http://www.hefce.ac.uk/pubs/RDreports/2005/rd01_05/rd01_05.doc

7

1. Introduction

1.1 National Reports and Initiatives in the UK

The 1996 ‘Quality and Standards’ Report notedthat ‘doctoral education was frequently regarded asa cottage industry, a prestigious yet somehow fringeactivity in higher education’ (UKCGE, 1996). Whilepostgraduate research may still be a minor part ofthe HE sector in terms of student numbers, it isnevertheless a significant part of that sector, andparticularly the output of the sector in relation tothe UK economy and international research effort.In this current paper we explore whether the moveover recent years to higher numbers of researchstudents in UK universities has affected the qualityand focus of research degrees.

The publication in 1996 of the UKCGE papercoincided with the Harris Report which proved amilestone in the debate about quality issues inresearch degree education. The Harris Reportrecommended the conditions under which qualitycould be delivered at PhD level (Harris, 1996).There are several initiatives and actions that havefollowed subsequent to Harris, for example:

� the Higher Education Careers Services Unit(CSU) directory of research and postgraduatequalifications (the Prospects Directory)

� the Review of Research undertaken by theHEFCE (HEFCE, 2001) which indicated the needto examine the research training provided byuniversities in order to produce thresholds ofprovision and good practice guidelines,eventually to be linked to funding. This Review,together with the two HEFCE ThresholdStandards consultations5 (HEFCE 2003)contributed directly to the revised Quality

Assurance Agency (QAA) Code of Practice:Section 1 Postgraduate Research Programmes(QAA 2004), to the extent that some of theHEFCE threshold standards are embedded inthe revised Code

� the QAA Framework for Higher EducationQualifications (FHEQ), January 2001, revisedSeptember 2008 (including as it doesdefinitions of doctoral level outcomes in theform of a specific ‘qualification descriptor’)

� HEFCE’s modifications to its funding model forPhD students (effectively ceasing to fundstudents located in departments with an RAErating of less than 4)

� the Economic and Social Research Council(ESRC) requirements regarding thedevelopment and monitoring of supervisorcapability (e.g. ESRC, 2001)

� the Research Councils and AHRB jointstatement regarding skills requirements forresearch students which is now included asAppendix 3 in the 2004 QAA Code)

� the introduction by the ESRC of a 1+3 model forthe PhD which has begun to spread to otherareas, and the broadening of Research Councilfunding parameters generally to encourageincreasing interdisciplinarity and ‘themed’research projects

� the development of Doctoral Training Centresby EPSRC, BBSRC and proposed by ESRC (see2.1 below)

1.2 Influence of Europe

The Bologna declaration and the various initiativesthat have followed from it have created a newfocus on doctoral education as well as on otheraspects of the postgraduate agenda. The Berlincommuniqué of 20036 added a third cycle, which

5 Improving standards in postgraduate degree programmes: information consultation, HEFCE, 2003/01;http://www.hefce.ac.uk/pubs/hefce/2003/03_01.htm, and Improving standards in postgraduate degree programmes, Formal consultation, HEFCE 2003/23;http://www.hefce.ac.uk/pubs/hefce/2003/03_23.htmhttp://www.hefce.ac.uk/pubs/hefce/2003/03_23.htm

6 The European Higher Education Area – achieving the goals: Communiqué of the Conference of European MinistersResponsible for Higher Education, Bergen, 19-20 May 2005http://www.bologna-bergen2005.no/Docs/00-Main_doc/050520_Bergen_Communique.pdf

included ‘doctoral education’, to the process thatwas started in Bologna. This addition was intendedto promote links between the European ResearchArea and the European Higher Education Area. Theshared ‘Dublin’ descriptors7 for academic awardsinclude doctoral level. The impact of theseinitiatives is explored in more detail later in thispaper.

Across Europe there are different registrationpatterns that link masters and doctoralprogramme levels. The most common of these arethe 1+3 or 1+4 but variations such as 1+2+3 and2+2+4 exist and clearly there are differences notjust in terms of the pattern of time spent at thedifferent levels but also of the sum totals of timespent. These different patterns and time frameshave their origins, to a greater or lesser extent, inthe attitudes held towards what is required of thelearner if they are to ‘step-up’ to the next (researchdegree) level. Whereas in some subjects in the UKit is the norm for students to move straight from afirst degree to a doctorate (e.g. biological sciences,law, psychology), it is unusual for a first degreegraduate in Europe outside the UK to register for adoctoral qualification. Typically, such graduatesfollow the bachelors → masters → doctorateroute. This then is potentially difficult for UK firstdegree graduates wishing to study for a doctoratein other parts of Europe. The equivalence ofMaster’s programmes of all types is also relevantwith, for example, credit for Master’s degrees incontinental Europe based on academic years, andin the UK on calendar years. However, ProfessorWendy Davies’s research on behalf of the UUKEurope Unit shows that there is also great diversityin the structures, length and credit weighting ofMaster’s degrees in parts of continental Europe.

Her study of four European countries (UUK 2008)8

indicates the lack of consistency does not just liebetween UK and the rest of Europe. In the authors’view, this shows the need for explicit, clearpublicity and definitions of all Master’sprogrammes offered by UK institutions, linked withthe FHEQ Master’s qualification descriptor andstating credit (UK and ECTS equivalent) at differentlevels. This may help graduates of such degrees todemonstrate equivalence and therefore facilitateentry to employment or doctoral programmes inwider Europe.

The final recommendations of a EuropeanUniversities Association (EUA) Bologna seminarheld in Nice in 20069 helpfully (for the UK inparticular) sought to maintain flexibility forinstitutions in admissions to doctoral degrees, asfollows:

“2.3 Ensuring access and admissionIn a fast-changing environment, it is essential tomaintain flexibility in admissions to doctoralprogrammes, and full institutional autonomy:diversity of institutional missions and context, andthe growing importance of lifelong learning, meanthat there are good reasons for different entryrequirements in institutions and programmesprovided fairness, transparency and objectivity isensured; The Bologna commitment that the second cycle givesaccess (= right to be considered for admission) to thethird cycle should be maintained, but access to thethird cycle should not be restricted to this route.”

This recommendation was subsequently formallyaccepted through the Bologna process.

8

7 Shared ‘Dublin’ descriptors for Short Cycle, First Cycle, Second Cycle and Third Cycle, a report from a Joint Quality Initiativeinformal group, 18 October 2004: http://www.jointquality.nl/content/descriptors/CompletesetDublinDescriptors.doc

8 'Mastering Diversity': UK HE Europe Unit report on current arrangements for Master’s programmes in France, the Netherlands,Germany and Bulgaria: http://www.europeunit.ac.uk/sites/europe_unit2/resources/E-08-02.pdf

9 EUA Bologna Seminar: Doctoral programmes in Europe, 7-9 December, Université de Nice, France; details of seminaroutcomes are at:http://www.eua.be/research/doctoral-programmes/doctoral-programmes-in-the-bologna-process/doctorates-seminar/

9

In addition, if we are concerned with the quality ofthe student experience then we need to takeaccount of what research students bring to theircurrent studies by way of prior experience and inthis there is a need to recognise how a programmeof studies at one level can be seen to prepare thestudent for the next. Here we need to acknowledgeboth direct and indirect preparation. When aninstitution seeks to appraise the quality of itspostgraduate research provision then self-evidentlyit needs to make judgments about what studentsneed to learn and how best they may be taught andsupervised in terms of the characteristics of thosestudents – and those characteristics include priorknowledge and experience as well as culturalbackground and so on. Where prior knowledge andexperience vary widely then clearly the onus is onprovision to be flexible and responsive to individualneeds. In short, any institutional research degreetraining framework needs to be related torecruitment policies and admission criteria; forexample, what counts as effective induction clearlyrelates to student characteristics as well as to theprovision of resources by the institution.

All of the above has stimulated debate across theEuropean Union. Green and Powell (2005) notethat in France, for example, the Government hasintroduced the Ecoles Doctorales to act as the focusfor PhD work, concentrating significant resourceson programmes in such écoles (Ministère de laRecherche, 2001). Green and Powell also note thatsimilar reviews are taking place in the US andCanada (citing for example Walker, 2001).

2. Some Contextual Factors

2.1 Rising demand

In 1996 the UKCGE paper (UKCGE 1996) noted therising demand for doctoral study in the UK. If wego back a stage further, it is clear that the growthin the postgraduate population has created ahigher education environment that is radicallydifferent from that of the 1960’s when the Robbins

Report in 1961/62 recorded 19,400 full-time and6,300 part-time students. Green and Powell (2005)note that by 1994/95 there were 128,300 full-timeand 187,100 part-time postgraduate students andthat the relative position of postgraduate studentnumbers had similarly changed during this period.They cite HESA (HESA, 2001) data as indicating thatin ‘1979, 13% of the total student population werepostgraduate (100,900 postgraduate in a totalpopulation of 787,000) yet in 1994/95 thecomparable figure was 21% (315,400 out of apopulation of 1,528,600). By the end of themillennium, the total population of postgraduateswas 151,330 full-time and 257,290 part-time, (Greenand Powell, 2005). Similarly, Green and Powell (2005)note changes in mode of study with a growth in thenumber of part-time students, equalisation of thegender balance and a significant growth in thenumbers of overseas students. In this latter respectthere were 25,100 international postgraduatestudents in HEIs in the UK in 1992/93, 8% from theEU, 92% from the rest of the world. By 1997/98 thisfigure had risen to 81,000, of which now 33% werefrom the EU and 67% from the rest of the world.

While the demand noted in 1996 has continued torise in the intervening period, a significant part ofthat ‘growth’ has perhaps been in doctoral studyoutside of traditional PhD programmes. Certainly,the UKCGE Reports into professional doctorates(2002 and 2005) show a marked increase in therange of nomenclature that is employed to denotethese doctorates and it seems reasonable toextrapolate that numbers of students studying forthese ‘new’ awards are increasing. In part, thisincrease in the number of different named awardsat doctoral level is a result of new areas of studyopening up at doctoral level (Powell and Long, 2005).

Preliminary analysis of the responses to the QAAdoctoral discussion paper (QAA, 2007, web-published only), offers empirical evidence that thenumber of students registered for professionaldoctorates has increased in the last five years, andalso that there is considerable diversity in structureand content, with clinically-based professional

10

doctorates offering opportunities for inclusion ofclinical practice through placements and otheractivities. The authors are hoping to conductfurther work to provide some statistics on thenumbers of students currently registered ondifferent forms of doctorate (includingprofessional doctorates and PhDs involving asignificant amount of formal training).

In short, there seems to have been an increase ininterest in professional and practice-baseddoctorates as well as the introduction of so-called‘New Route’ PhDs, intended mainly forinternational students. Therefore, while thenumber of doctoral students overall has increased,it is also the case that the range of kinds of doctoralprogrammes, and the range of kinds of studentengaged in them, has broadened.

During 2008, funding was made available by atleast one funding council for doctoral trainingcentres (DTCs) in science and engineering, througha competitive bidding process. Although at anearly stage, these developments appear to beintroducing a form of doctorate that sitssomewhere between the traditional PhD and theprofessional doctorate. Candidates will beregistered on ‘integrated’ four-year PhDs thatinclude elements of compulsory training.

2.2 Focus on Quality of provision

The earlier paper (UKCGE 1996) also made referenceto a growing recognition by Government thatpostgraduate education is a ‘valuable contributor tothe balance of payments and, more generally, to theUK's international standing’ (page 7). The importanceof research graduates to the general economy hadalso been publicly recognized at the UK GRADProfiting from Postgraduate Talent conference inSeptember 2004, where Raffaele Liberali, Director forthe Human Factor, Mobility and Marie Curie activitiesat the Directorate-General for Research, EuropeanCommission and a UK government representativefor higher education, provided evidence for researcheffort as a major contributor to GDP, highlighting

increasing demand for European Frameworkfunding, improved co-operation between academiaand industry, and greater recognition for research asa profession (UK GRAD, 2004).

More recently, a substantive research projectconducted by the UK Higher Education InternationalUnit – “The UK’s Competitive Advantage: The Marketfor International Research Students” (UUK, 2008),noted that the UK has 15% of the internationalmarket for research students, and that 42% of allpostgraduate research students in the UK areinternational students. The benefits to the UK of thissituation are outlined in the UUK publication,alongside some less reassuring statistics that showother countries’ success in recruitment are likely tomake the UK’s current position difficult to maintain,even though the quality of our graduates is not inquestion. One of the principal concerns is the cost ofstudying in Britain – both in relation to tuition andbench fees and the cost of living.

The UKCGE paper continued to note theGovernment stress on the ‘need to provide anappropriate quality of service so as to ensure thatvalue for money is achieved and that the UK's enviablereputation in this area is not damaged’ (page 7). Therecan be little doubt – as shown by the two HEFCEThreshold Standards consultations on postgraduateresearch programmes and the subsequent revisionof the QAA Code of Practice in relation to thoseprogrammes – that the quality of the doctoraleducation on offer in UK universities has comeunder increasing scrutiny. This increase is in line witha greater emphasis across the sector on qualityissues. Indeed, the 1996 Report tracked thedevelopment of the notion of doctoral educationfrom an essentially ‘private relationship between asupervisor and his or her research student’ (page 7) toone where institutional expectations werebecoming paramount.

2.3 Completion Rates

UKCGE (1996) described a situation wherein the lackof institutional engagement with the detail of what

11

went on in the supervision of research students ledto a potential consequence of ‘poor completiontimes, particularly in the arts, humanities and socialsciences’ (page 7). However, as far as the authors areaware, there is no direct evidence of any causal linkbetween ‘poor supervision’ and ‘poor completiontimes’; indeed the QAA Special Review noted that‘in most institutions arrangements for supervision areappropriate and satisfactory’ (QAA, 2007, page 7). Italso noted examples of good practice in supervisionin paragraphs 41, 42, and 43. Clearly, we are notclaiming that there are no examples of poor practicein research degree supervision in the UK but ratherthat the student-supervisor relationship is critical toensuring a research degree candidate knows whatis expected of him/her and has access to thenecessary resources to support the intellectualeffort and creativity needed for successfulcompletion.

The current situation is that after some years ofattention from the Research Councils to the issueof comparative completion rates HEFCE arebeginning to address the issue directly and for allstudents by looking at institutional completion, or‘qualification’ rates. After approximately a two yearlead-in period during which institutions wereprovided with direct access to HESA data on theirresearch degree completion rates and invited toupdate and correct them, in October 2007, HEFCEpublished UK research degree qualification ratesfor full-time students for the first time. Two datasets were produced, one for UK/EU graduates, theother for international graduates (HEFCE, 2007,web published only). These data can be found athttp://www.hefce.ac.uk/pubs/hefce/2007/07_29/

Institutions that responded to the data verificationexercises proposed that the publishedqualification rates should exclude students whohave taken periods of suspension or extension.HEFCE adjusted for this by basing theirqualification rate calculations on a seven-year

completion period which would take account ofthe majority of full-time students who suspend orextend their studies but go on to qualify. They alsonoted that there should not be a significantlyhigher proportion of students suspending orextending their studies at one institutioncompared with another.

Again in response to feedback from institutionsduring the data verification exercise HEFCEincluded context statistics showing the number ofstudents who were still actively pursuing theirresearch degree programmes and the numberwho transfer to another institution. For example,the numbers of students at each institutionstudying science-based subjects10 and (for homestudents) receiving Research Council support wereshown as context statistics to help the reader tointerpret the qualification rate and benchmark.

In addition, HEFCE introduced benchmarks in orderto avoid comparisons between institutions whosestudent profiles were so different that they couldand should not be compared directly. Thebenchmarks were used to calculate for eachinstitution what the qualification rate would be if itreflected the sector average after taking intoaccount the impact of variations in subject mix and(for home students only) the proportion of studentsreceiving funding support from a Research Council– the two most significant remaining causes ofvariation in the qualification rates.

It is HEFCE’s intention to publish detailedqualification rates annually, gradually building upinstitutional profiles. At the time of going to press,the 2008 rates have not yet been published.

2.4 Part-time research students

UKCGE (1996) noted that relatively little attentionhas been given to the growing number of part-time and/or self-financing research students,

10 For this purpose, science-based programmes are defined as medicine, veterinary science, subjects allied to medicine,biological sciences, physical sciences, engineering, mathematics, and agriculture.

12

especially since part-time students are typicallynot funded by the research councils or other grant-awarding bodies. The direct attention given byHEFCE to completion/qualification rates and theover-arching scope of the QAA Code changes thisobservation though many would still argue thatdespite the introduction of graduate schools andthe like the issues facing part-time students arenot necessarily addressed by institutions asdirectly as those for full-time students.

It is perhaps worth noting here that so-called‘Roberts’ funding may have had a beneficial effecton provision for part-time research students sincethe 1996 UKCGE paper. Although this funding isintended primarily for Research Council fundedstudents, RCUK has said that developmentopportunities funded through the Roberts incomecan be opened up to non Research Council fundedstudents, where there is enough capacity. The spin-off from this is that such developmentopportunities could potentially benefit all studentsincluding those studying part-time.

2.5 Responding to the needs of industry andcommerce – transferable skills

In May 1993 the Government produced a WhitePaper entitled 'Realising our potential: a strategy forscience, engineering and technology'. This WhitePaper contained a number of recommendationsrelating to the doctoral qualifications existing in theUK at the time. In particular, it encourageduniversities to respond to the perceived needs ofindustry and commerce. Part of this response wasto be an increased emphasis on research methodsand transferable skills during the early stages ofresearch projects. Again, this trend, which had, asseen above, begun by the time of the 1996 UKCGEReport, has continued through the interveningperiod. Perhaps most notably the ResearchCouncils and Arts and Humanities Research Board(AHRB, now the Arts and Humanities ResearchCouncil) issued a Joint Statement with regard toskills training requirements for researchpostgraduates (Joint Statement of Skills Training

Requirements of Research Postgraduates,(Research Councils, 2001)).

In short, the emphasis on a programme of doctoralstudy as being broader than the completion of aresearch project and including the student having tolearn both subject specific research skills and thoseof a more generic kind, has increased from 1996through to the present. The argument for moreattention to be given to generic skills training was akey feature of the HEFCE consultations on ‘ImprovingPractice in Research Degree Programmes’.

It may well be that in all of this there has beeninfluence from the USA where skills training hasformed a significant part of doctoral education forsome time (Nerad, 2007). This influence has beenmanifest in initiatives such as the so-called ‘NewRoute PhD’. It is also perhaps worth noting here thatthis influence has been variable in that the systemsare not parallel and the adoption of ideas has notbeen universal. Many countries round the world arere-assessing their provision of doctoral educationand in particular the implications of the extensionof doctoral level study into ‘new’ areas ofprofessional work. Tensions arise when new areas ofstudy raise issues about the appropriateness ofdifferent forms of study and, perhaps moreimportantly, new forms of examination (notablywhere the professional practice of the candidatebecomes the focus for academic development andhence of subsequent assessment). See Powell andGreen (2007) for examples of such re-assessing.

It may also be worth noting here that the term‘professional doctorates’ may not mean exactly thesame in the USA as in the UK and elsewhere inEurope. In a ‘Review of Professional Doctorates’undertaken by the National QualificationsAuthority of Ireland, the characteristics ofprogrammes of study with the title ‘professionaldoctorates’ in the USA are said to include “asubstantial element of taught coursework, and atendency towards shorter dissertations and longerperiods of supervised professional practice”.(National Qualifications Authority of Ireland, 2006.)

13

This publication goes on to say that “Most of theseawards have been classified as ‘1st ProfessionalDegrees’.” and that “The ‘1st professional degree’is a first degree, not a graduate degree even thoughit incorporates the word ‘doctor’ in the title”11.

The White Paper was followed by a consultativedocument published by the Office of Science andTechnology in February 1994 entitled 'A newstructure for postgraduate research trainingsupported by the Research Councils' (Office ofScience and Technology, 1994). In the UK however,the QAA definition of research based doctoratesincludes Professional Doctorates and these arelisted in the FHEQ as research degrees.

2.6 Concern for Quality

2.6.1 Enhancing quality (1) positive developments

The UKCGE 1996 ‘Quality and Standards’ Reportnoted that ‘comparative data on the relativeperformances of doctoral students is currentlypatchy’ (page 11). To a large extent this situationhas not changed, though the revised QAA Code ofPractice and HEFCE’s monitoring of qualification(completion) rates (see above) and their use in theQAA institutional audit process may bring about anew transparency in relation to standards across arange of criteria (effectively those covered by theprecepts in the Code). In the covering paper to thefirst set of qualification rates tables published,HEFCE makes it explicit that the compilation andpublication of doctoral completion rates is linkedwith the quality assurance of research degrees: “Inkeeping with our commitment to support excellencein the national research base, HEFCE requires theresearch degree programmes that it supportsthrough its grant to meet minimum standards set outin the revised Quality Assurance Agency (QAA) Codeof Practice” (HEFCE, 2007).

Of course, the Research Councils generatesubmission statistics as they did in 1996 but theseare limited in terms of representing performancemeasure that go beyond how long it takes for theaverage student to complete their studies.

It is worth noting here that many UK HEIs haveintroduced, or are introducing, on-line systems thatboth facilitate tracking of research studentprogress and encourage candidates to reflect ontheir own development and the progress of theirresearch degree programmes. The introduction ofsuch systems, therefore, brings increasedadministrative efficiency but also raises thepossibility of enhancing student self monitoring.

Until 2005/06 assessment of the quality of currentdoctoral learning experiences was based primarilyon the outcome of institutional audits undertakenby QAA and its predecessors (mainly HEQC)though clearly other direct and indirect measureswere also available, for example: the proportion ofstudents gaining doctoral awards and thecontributions made by research students withinthe RAE exercises.

It is perhaps worth reminding ourselves here of thepositive developments revealed by HEQC (1994 and1996) audits. Those listed below were included in the1996 UKCGE ‘Quality and Standards’ Report (page 11):

� increased use being made of supervisory teams� development of institutional induction

programmes for research students� development of more formal frameworks for

the management of graduate level activities� improved monitoring of research student

performance� improvements in the opportunities for research

students to make their views known� the development of operational codes of practice

11 First professional degrees are classified by the National Centre for Educational Statistics (NCES) as those awarded aftercompletion of academic requirements to begin practice in the following professions: chiropractic (D.C.); dentistry (DDS orDMD); law (LLB or JD); medicine (MD); optometry (OD); osteopathic medicine (DO), pharmacy (Pharm.D); podiatry(DPM,DP,PodD); theology (MDiv, MHL,BD or Ordination) or veterinary medicine DVM). The classification of these kinds ofdegrees is under review within the NCES. See http://nces.ed.gov/ipeds/trp15a.asp

14

In terms of developments in these respects since1996, it is arguable that the QAA Code has addressedall of these ‘positive developments’ and made thema set of universal benchmarks rather than examplesof good practice not universally accepted. Certainly,the use of supervisory teams rather than singlesupervisors was one of the contentious issues thatarose during the HEFCE consultations regarding‘Improving Standards’ (HEFCE, 2003), but that waspartially resolved in the revised section of the QAACode of Practice published in 2004 (QAA, 2004,precept 12). Use of teams has been a feature of some,but not all, universities’ practice for some time andthe Code has brought it to a ‘normal’ status.However, these ‘positive developments’ may not inthemselves enhance the experience of the researchstudent. The elements listed above are parts of aframework that enable such enhancement – thequestion that this current report considers is theextent to which these developments (and more) canbe seen to assure that the experience of doctoralstudy is enhanced for all students. In the currentclimate of concern with quality we are in danger ofmaking sure systems are in place but not necessarilylooking at what the students actually receive. It maywell be then that the challenge facing institutions atthe moment is how to assure themselves that a)these desirable elements are in place and b) they aremaking a difference to the student experience andto successful completion.

2.6.2 Enhancing quality (2) – problems (asidentified in the UKCGE 1996 paper)

The 1996 UKCGE paper reported some ‘problems’in RDP provision as highlighted by HEQC; weshould note that these were given as examples –it was not the stated intention to provide anexhaustive list. These are paraphrased below inorder to give some historical perspective onquality issues that were pertinent in 1996.

(a) variable supervision arrangements,(b) lack of supervisor training,(c) inadequate training and monitoring of

students engaged in teaching;

(d) weaknesses in methods used to select andbrief external examiners;

(e) variable completion rates for postgraduatedoctoral students;

(f ) inadequate training for students in ‘researchtechniques and quantitative methods’;

(g) absence of effective rules and proceduresregarding appeals,

(h) variations in the mechanisms for transfer ofregistration from MPhil to PhD.

Current areas of consistency

As a result of initiatives such as the QAA Code ofPractice some consistency can now be expectedacross the sector in terms of (using the numbering asabove): (a) the need for supervisory teams (rather thansingle supervisors), (b) the training of supervisors, (d)rigorous procedures for selecting, appointing andbriefing examiners (internal and external) (f ) theexpectation that students will be offereddevelopment opportunities (supported by Robertsfunding), including research methods training, (g) theneed for transparent rules and procedures for Appealsand (h) clarity in the transfer from MPhil to PhDregistrations. It is also the case that the issue of (e)‘completion’ is now coming under increased scrutinywith the HEFCE survey of qualification rates(completion has been deemed too imprecise a dateand the actual conferring of the award by the relevantcommittee on behalf of the institution, or similar, hasbeen chosen as the census point).

Students who teach

The issue of students who teach is also addressedin the revised Code (within Precept 9). But here thematter is less clear-cut. Teaching by students isreferred to in the context of entitlements andresponsibilities. Thus it is not quite so clear thatwhat was perceived of in 1996 as ‘inadequatetraining’ is a clear focus of change in the currentclimate. However we should note that the QAACode does make it explicit that when researchstudents/doctoral candidates are asked toundertake teaching duties, they should receive

15

appropriate and adequate training for, and besupported in the role.

Selection and briefing of examiners

This leaves (d) selection and briefing of examiners.Certainly, selection is addressed in the revisedCode and briefing is covered in section (a) ofPrecept 23 “Other issues include how to establishthat the examiners have relevant qualifications andexperience and a clear understanding of the task”and also in (b) “Institutions will wish to consider waysof making sure that the examiners have theinformation and conditions they need to identify theareas to be explored at the viva.”. In spite of thisprecept it may be that variable expectations andvariable notions of the purposes and practices ofthe examination process remain issues largelyunresolved (see for example discussions in Tinklerand Jackson 2000 & 2004).

Certainly, Williams (2007) among others has notedthe differences in approach to examination atresearch degree level between disciplines andindeed between subject areas within disciplines.For example, Tinkler and Jackson (2000) note thatthe average length of vivas differs according todiscipline. While this may seem a fairly superficialdistinction, we suggest that it is indicative ofdifferent processes followed by examiners acrossdisciplines which in turn relate to differentconcerns about what is being examined and howthat examination can best be carried out.

Training in ‘Research Techniques andQuantitative Methods’

The other issue that perhaps remains unresolvedfrom 1996 is that of inadequate training in‘research techniques and quantitative methods’.This comment from 1996 is interesting for anumber of reasons. First, it is noteworthy that it isquantitative methods that were called intoquestion. This selection ignores the range ofqualitative methods and it is hard to know if this isa result of a perception, by the authors of the 1996

Report, that training in the latter was adequate orrather unworthy of mention. Certainly there are nopositive references to training in qualitativemethods. Second, it is interesting that the concernin 1996 was with research specific skills rather thanwith generic skills.

It might be argued that the growth in emphasis ongeneric skills over the past ten years has been at theexpense of an emphasis on subject specific orresearch specific skills. However, it is also the casethat many working in the sector have maintained afocus on research methods as part of a widertraining which includes generic issues. Indeed, theResearch Councils’ Joint Skills Statement as quotedin section 1 of the QAA Code of Practice states thatthey “would also want to re-emphasise their belief thattraining in research skills and techniques is the keyelement in the development of a research student, andthat PhD students are expected to make a substantial,original contribution to knowledge in their area,normally leading to published work. The developmentof wider employment-related skills should not detractfrom that core objective”. (QAA, 2004). Although‘Roberts’ Funding is explicitly given to institutionsfor generic skills training (i.e. skills in sections C to Gof the Joint Skills Statement) and not for the skillslisted in sections A or B, in practice it is often difficultto differentiate between research and generic skills.That is, there is a blurring of the boundariesbetween the two, which is normally of benefit to theresearch candidate – he/she is often learninggeneric skills as an integrated part of doing andlearning about research.

The increased emphasis on skills developmentmirrors perhaps the change in perception of thepurpose of the PhD itself. PhD training wascriticised as being too narrow and as producingthinkers and doers not sufficiently capable ofcommunicating and collaborating with peers fromdifferent disciplines (the 1993 Government WhitePaper ‘Realising our Potential’ had made this broadpoint). Subsequently the PhD has been interpretedby some (e.g. Felton, 2008) as a broad high-leveltraining within a cognate discipline with attention

to the ways in which that discipline impacts uponothers. As we have noted earlier, generic skills havebecome – to many – an integral part of thedoctoral process (including here the professionaldoctorates as well as the ‘traditional’ PhD).

Monitoring and assessing generic skills

While it is relatively straightforward to see howsubject-specific knowledge and skills can bemonitored and ultimately assessed in theproduction and examination of a thesis, it is lessclear how the learning of generic skills can besimilarly monitored and assessed (at least withinthe ‘traditional’ PhD programme of study). In short,to be judged as truly ‘generic’ a learnt skill wouldneed to be demonstrated by the student andassessed by an examiner in a range of situations –i.e. it is precisely the generalisability that needs tobe the focus of any assessment procedure. Boththe criteria for the award and the assessmentmethod would need to change in response to this‘new’ element in the learning process.

If an institution does not make changes of thiskind then the situation may arise wherein there isan insistence on the learning of generic skillswithin a formal programme of study with noadequate way of measuring the success of thelearning that goes on. On the other hand aninstitution that does make changes of the kinddescribed has to balance what is to be taken astruly important in the programme of study andhence in the assessment – is it to be the‘contribution to knowledge’ or the candidate’sability to communicate and collaborate in a rangeof (possibly unspecified) situations? It can ofcourse be both, but then an institution has to facethe possibility of a candidate making a significant‘contribution to knowledge’ whilst being judgednot sufficiently capable of communicating withothers outside the discipline about his/herfindings and thus denied the award (or given theaward conditional upon giving evidence tooverturn the judgment of insufficiency in genericskill ability).

On the other hand, there is a sense in which theoral examination that is almost without exceptionseen as a necessary part of research degreeexamination in the UK does, by definition, assesssome generic skills: the ability to communicateclearly, orally and in writing, critical thinking andanalytical skills, and the ability to present areasoned argument.

All of this is not to deny the place of generic skillslearning in doctoral study but to raise, from thequality perspective, the difficulties of managingthe assessment of that learning in a transparentway that is meaningful in the context of a doctoralaward. What these issues do perhaps require of thesector is to revisit and review the purpose andoutcomes of the doctorate itself. The qualityquestions we have raised above can only really beresolved when there is transparency about whatinstitutions are doing when they educate doctoralcandidates – in this sense assessment shouldfollow purpose and when purpose is unclear thenassessment will necessarily be impaired.

Reviewing the criteria for the award

The initial focus for any revisiting and reviewing ofthe purpose of the doctorate might be to examinethe criteria for the award. If one takes the view thatall elements of a learning programme need to bereviewed in any assessment of the award inquestion then, in the case of the PhD, the criteriafor the award would need to be amended toreflect this requirement. In short, the learning of anassessable skill would need to be built into thecriteria for gaining the award.

This becomes a particularly pertinent issue whenone considers the non-traditional routes to adoctoral award. If there is parity of level between,for example, the PhD and a professional doctorate,then that parity needs to be apparent in theassessment criteria for both types of award. This isnot to suggest that the criteria need to be thesame – indeed for these to be separate awardsthen necessarily the criteria need to spell out how

16

17

one is distinguished from the other – but theyneed to relate to each other in a way that indicatesthat the level of achievement is comparable.

In a pragmatic sense it may be that the two sets ofcriteria would share many of the same words, forexample about evidence regarding ‘contributionto knowledge’ ‘independent critical ability’,‘technical competence in the chosen field’, ‘theability to present and defend a thesis in a lucid andscholarly manner’, and ‘the production of materialworthy of peer-reviewed publication’. Yet thesecriteria would differ in the interpretation of thelocation and kind of contribution to be made bythe candidate in order to merit the award and inthis sense standard or level of award would remainconsistent though the focus within which it isachieved may differ.

From part of the initial analysis of responses to theQAA discussion paper on doctoral degrees, it isclear that many institutions use consistentassessment criteria for different types of doctorate.‘Publishability’ remains an important criterion,supported by different definitions of ‘originality’. Adetailed summary about responses relevant to thisdiscussion is available at: http://www.qaa.ac.uk/academicinfrastructure/doctoralProg/Q8Responses.pdf(Clarke, 2008)

In terms of the level of two doctoral awards ofdifferent kinds it is difficult to see how parity canbe assured if the criteria are wholly unrelated. Itmay be that it is preferable for an institution toconsider the criteria for all its doctoral awardswithin a common evaluative frame, as someinstitutions have done (Clarke, 2008) (cf doctoralqualification descriptors in the FHEQ). Parity will behard to sustain where the prime driver for thedevelopment of the criteria for one doctoral award(or more likely sub-set of doctoral awards) is pre-specified curriculum content rather than a(unifying) notion of ‘doctorateness’. Conversely, itmay be that the ability to have parity in thestandards of academic awards is more about

fitness for purpose in the subject (e.g. professionaldoctorates are more common in the socialsciences, engineering and, to a limited extent,clinical subjects, than in arts or science), than aboutwhether someone who has a PhD has or has notachieved a different academic level from someonewith a professional doctorate. We suggest that thisis a matter that institutions might wish to debateas they develop standards and criteria that meetthem across a range of doctoral awards.

In short, there is a need for a wider debate aboutthe purpose of the doctorate and hence whatcriteria should be applied in order for a candidateto gain a doctoral award.

3. What is ‘Quality’ in PostgraduateResearch Education?

3.1 Definitions of quality

In the 1996 ‘Quality and Standards’ paper UKCGEsuggested five definitions of quality, which can besummarised as follows:

(i) the highest possible standard (of academicachievement, as well as of support for andmanagement of research degrees)

(ii) fitness for purpose(iii) effectiveness in achieving institutional goals(iv) conformity to a specification or standard(v) meeting the stated or implied needs of

customers

Of course these are not mutually exclusivedefinitions and certainly (iii) and (iv) arecompatible with (and perhaps subsidiary to) theother three definitions.

(i) The highest possible standard

The 1996 Report makes the point that the firstdefinition above – the highest possible standard -may be the ‘traditional’ one but it is not generallywhat concerns HEIs in terms of their quality

18

assurance procedures. All might aspire to thehighest possible standards but what is takencurrently as the objective is to set an acceptablelevel of quality of provision for research students– a threshold standard. Further, it might be possibleto find ready agreement that this threshold shouldbe as high as possible so in one sense thisdefinition becomes a part of any workingdefinition (the highest possible standardacceptable for all). Certainly, it might be acceptableto most that, as noted in 1996 (page 13) any systemshould seek to raise in a progressive way thethreshold standard. This is an important point in asmuch as it indicates one aspect that we may argueshould be an integral of any quality system – thenotion of an inbuilt self-evaluative structure thatseeks feedback and accommodates it within acontinuing cycle of review and revision of bothprinciples and practices.

(ii) Fitness for purpose

The 1996 Report argues for this definition as boththe most commonly accepted (e.g. adopted byHEQC and HEFCE, page 13) and the one mostreadily equated with the authors’ notion of quality(page 14). However, fitness for purpose of theaward without some notion of the ‘quality’ of thatpurpose is a somewhat hollow notion. Certainly,one might expect that fitness for purpose wouldrequire a contextual framework involving somevalue statements about what is desirable and whatis not if it to be more than a rather blunt utilitariandevice. In our view therefore any qualityframework relating to doctoral study would needto address the purpose of the study itself and itslevel (why do we need people to be studying inthese research areas and why do they need tostudy as this particular (doctoral) level?) This, ofcourse, brings us full-circle to criteria for the award.

In responding to the QAA consultation on doctoraldegrees, institutions supported firm yet flexiblecriteria (FHEQ doctoral qualification descriptor) atnational level to set clear academic standards fordoctoral awards. Implicit in this is a need for

interpretation at subject level (see commentsabove about discipline-related imperatives)together with recognition of the importance ofconsistency and doctoral graduates that are ‘fit forpurpose’ for the next stage in their career, whetherthis be industry, academia, or one of the hundredsof other destinations of research degree graduates.

This setting of an academic standard for doctoralqualifications is underpinned by Section 1 of theQAA Code of Practice, which provides a frameworkwithin which institutions are working towards, andachieving, high quality provision, including ininstitutional arrangements and the researchenvironment, as well as in admissions, supervision,assessment, etc. Institutional frameworks areexplored further below.

(iii) Effectiveness in achieving institutionalgoals

This definition lies somewhere at the heart ofprocedures followed by some quality assurancemethodologies (e.g. QAA). There may be a sensethat what it is possible to recognize and to someextent measure is (a) the nature of what aninstitution sets out as its goals for researcheducation and (b) how clear it is that it meetsthem, including what evidence is available.However, as noted in the 1996 ‘Quality andStandards’ paper (page 13), this (i.e. institutionalgoals and quality assurance methods) cannot tellthe whole story. Certainly, it is important that aninstitution is clear about the goals it seeks to attainbut of course the goals themselves matter – at leastin our interpretation of ‘quality’. We diverge from theearlier Report in suggesting that goals with regardto doctoral education do differ across the sector –not just within disciplines but also according toinstitutional views of doctoral education and whatare reasonable routes to achieving a doctoral award.Therefore it seems fair to suggest that it is notenough (in our view of achieving a quality learningenvironment for doctoral students) to be effectivein achieving institutional goals – the goalsthemselves need to come under scrutiny. It is

19

perhaps worth noting here that the use of externalexaminers is in itself an attempt to give this kind ofscrutiny and ensure some notion of uniformity ofquality within disciplines and between HEIs.

In addition, and perhaps as importantly, the way inwhich an institution sets about developing its goalsfor doctoral study matters when it comes to makingjudgments about the quality of doctoral educationand frameworks within which this is offered. Onemight legitimately expect that an institution shouldwork through, in some systematic and consensualway, the arguments for particular kinds of processesand procedures to support doctoral student‘training’, i.e. research student education, on the onehand and ‘doing research’ on the other. In short, fora quality learning environment to exist in aninstitution there needs to be an argued through andthought out stance with regard to what can beexpected of students and what the institution’sresponsibilities are in enabling the student to meetthose expectations. Currently, many institutionsseek to make such expectations and responsibilitiestransparent via induction days or similar devices.

(iv) Conformity to a specification or standard

This definition does not exclude that given above in(iii). The difficulty with this definition lies in thepotential of a set specification, once set down, to beapplied inflexibly. Where a specification is laid downin a way deemed by institutions as immutable thensome individuals in those institutions may targettheir activities to meet that specification withoutthinking through the changing context withinwhich they are working and the implications ofmeeting the specification. Similarly, where thespecification is set out in numerical form theninstitutions may begin to operate on the basis ofmeeting targets in an overly simplistic way that maydeny opportunities for improving quality or preventstudents from achieving their full potential. Policiesand advice at institutional level need to be feasible,fully owned, and therefore understood by thoseworking within the institution. It may well be that(as implied in the 1996 ‘Quality and Standards’

paper) the temptation to simplify issues byproducing clear and measurable performanceindicators may lead to an inability to evaluate thequality of research education, or at least may onlyallow a narrow aspect of it to be judged in aninflexible way. The issue with any specific targetstandard, then, is how to ensure that meeting thestandard will in itself realize an increase in quality ofthe student experience. Where such realization isnot automatic then raising quality becomes amatter of an institution making use of thespecification to guide its practice – this latterapproach requires of the institution a morereflective modus operandi than the former. Suchreflection might from part of training and updatingprogrammes for research degree supervisors thatare set in line with QAA expectations.

An aspect of this principle might be that, to beeffective, any specification or standard needs to besufficiently flexible to address the needs of diversestudents, staff and subjects within an institution,and nationally, the needs of diverse institutionswith different missions, e.g. those with largenumbers of part-time students compared withthose that have large numbers of full-time,Research Council funded students. The frameworkset out through the precepts of Section 1 of theQAA Code was designed with the above diversityin mind and to enable any institution to create theconditions within which research education canflourish.

(v) Meeting the stated or implied needs ofcustomers

This is perhaps a deceptively straightforwarddefinition of quality in postgraduate researchdegrees. In the sense that students have needs inrelation to their study then meeting them wouldseem to relate directly to the quality of theexperience. But, as noted in the 1996 Report, it isnot so straightforward to define ‘customers’ in thisarea. The funders of research degree study –whether they be government agencies,independent charities or overseas bodies have

20

legitimate claims to be stakeholders in researchdegree study and hence in this sense ‘customers’.In short, what counts as a successful outcome maydiffer according to the needs of stake-holders.Therefore any notion of quality as applied in thisarea perhaps needs to accommodate the differingneeds of the various stakeholders (and, above all,to meet the rigorous academic standards set bythe assessment criteria used by institutions andexaminers of research degree candidates).

3.2 Purposes of the doctoral qualification

The discussion of different definitions of qualityleads inevitably to a need to examine the purposeof doctoral study itself and the purpose of thedoctoral qualification.

The purpose of doctoral study may be defined asthe finding out of more about the topic by thestudent and in so doing enhancing that student’sresearch skills and his/her personal knowledge ofthe substantive area as well as of knowledge in amore general sense. The purpose of the doctoralqualification on the other hand is lessstraightforward. Here the 1996 Report (pages 15 &16) considers the problem from the perspective ofthe ‘product’ of which the qualification is evidence.That product is interpreted as the independentresearcher him/herself (the doctoral candidate)and/or the contribution to knowledge made bythe doctoral submission (the thesis). In all of thiswe may conclude that perceptions of the purposeof qualification may differ according tostakeholder, whereas the purpose of study mayremain constant.

In one sense, the discussion above may seem to beabout the process and the product of researchdegree study, with the process relating to thelearning of the student and the product to theoutcomes in terms of personal development ofcandidates and the (universal) knowledge base. Inour own view however discussion may not be soreadily separated out. The issue runs to the heartof any debate about quality. The doctorate is

typically taken to represent the pinnacle ofacademic achievement (setting aside for amoment the so-called ‘higher doctorates’) and inthat sense its purpose is, crudely put, to demarcateor identify those who have attained the pinnacle.The doctorate is attained by means of an educativeprocess in which students learn via a process ofengagement with ‘teachers’ and submit the resultsof their learning for assessment by examiners.

The single aspect that makes the doctorate uniquein the educational system is that, for the doctoralqualification to be affirmed, the learning of thestudent must change the very nature of the ‘topic’that he/she has studied to gain such affirmation. Theevidence that candidates present for examinationis this ‘change’ and their ability to understand it toa level that indicates that they can go on to makefurther such changes in an independent andcreative way. In this sense the purpose of doctoralstudy is to lead a candidate to a state - the doctoralqualification - in which he/she makes thecontribution and successfully indicates the ability tocontinue in the independent way noted above.Stakeholders may graft their own purposes uponthis underlying raison d’être but we suggest thatthose concerned with quality need to focus on thelatter rather than be sidetracked by the former.

We note above that a single aspect makes thedoctorate unique: it involves a changing of the topicunder study, i.e. it makes a ‘contribution’ toknowledge. It follows then that all other aspects ofdoctoral study and qualification are not necessarilyunique but may share common features with otherlevels and aspects of university study andqualification. So for example, it seems to us that whilethe sector may use the term ‘supervision’, this is notradically dissimilar to ‘teaching’ at other levels in HE(excepting that a supervisor teaches towards the‘contribution’). Similarly, progress through aprogramme of study is often given the status of‘transfer’ from MPhil to PhD registration when inreality it is merely a matter of showing that thecandidate has worked towards a goal and has plansthat make further progress to a new goal reasonable.

21

Finally, it is arguable that the research degreeexamination in the UK has become an affair that isoften shrouded in mystery and with unclearpurposes and procedures. Certainly it is fair to sayperhaps that a significant number of candidatesare not aware of what is required of them in thedoctoral examination (see for example Pearce,2005), yet in reality it should simply be a matter ofjudging whether or not the candidate has met thecriteria for the award. Our point is that whentalking of quality there is a need to cross-referenceto what counts for quality in other parts of thehigher education system rather than seeingresearch degree study as somehow separate andnecessarily different. Certainly, it is distinctive (forthe single reason we give) but principles ofeffective pedagogy and assessment still apply.

3.3 ‘Non-traditional’ doctorates

The 1996 paper discussed the need to consider‘non-traditional’ doctorates within the samequality domain as the traditional PhD. To a greatextent this has been achieved in practice, althoughthere is some evidence that not all involved withdoctoral education are committed to parity ofesteem between professional doctorates and thePhD. However, the need for similar measures ofquality to be applied to all forms of doctoral studyhas perhaps become more pronounced over thelast ten years as these forms have increased in bothnumber and popularity (in terms of number ofstudents studying) – see for example the UKCGEreports on Professional Doctorates (2005) and thePhD by Published Work (2004). It may also be,however, that the issues of quality differ across thevarious forms of doctoral study. The forms referredto in 1996 were: (i) the PhD by Published Work, (ii)doctorates in the creative arts and (iii) so-called‘taught doctorates’.

3.3.1 PhD by published work

The 1996 paper points out correctly that, while itmay be the case that a candidate’s published workdoes not provide direct evidence of successful

training in research methods, there is typically anacceptance in institutional regulations where suchawards are on offer, that success in the process ofpublication itself provides evidence of the abilityto carry out research through to completion. Careis obviously required when assessing whether ornot that ability is of an independent kind - that isthe levels of collaboration and the kinds of inputmade by various individuals named ascollaborators need to be checked to ensure thatthe successful candidate can continue as anindependent researcher in a sustained andcreative way.

The oral examination clearly takes on a keysignificance in these respects. The 1996 paper spellsout the need for: (a) recognition of contributionsfrom others, (b) overall coherence of the workpresented and (c) the need for particular rigour interms of involvement of two external examinersappointed with no input with regard to theirselection from the candidate. While we have no realissue with these points (except perhaps the last one– input from the candidate) it does seem to us thatthe real issue again is the purpose of the study andthe qualification. If we return again to the making ofa contribution and the providing of evidence thatthe candidate can continue to make suchcontributions in an independent and proactive waythen the matter of quality of award is not diminishedfrom the ‘traditional’ PhD. The level can be seen asthe same; it is merely that the candidate has taken adifferent route to providing the evidence that he/shehas met the criteria for the award.

The real question in relation to quality is where thecriteria for the award of PhD by Published Work liein relation to its traditional counterpart. It seems tous that any deviation from the standard set out forthe traditional PhD would result in diminishedquality of award and thus of student experience. Ifthe level of the doctoral qualification refers to thestandard as set down in the Framework for HigherEducation Qualifications then, for quality to bemaintained across awards, it must apply equally tothe PhD and to the PhD by Published Work.

22

Our recommendation then would be thatinstitutions make reference to their own criteria forawards and where any discrepancies of level existbetween these two doctoral awards then suchdiscrepancies would need to be addressed. If thecriteria of the two awards refer to levels that are ona par, then issues to do with where the study wasundertaken and the period of time that was takento reach the point of submission become lessrelevant.

3.3.2 Doctorates in the creative arts

The 1996 Position The 1996 paper did not dwell on doctorates in thecreative arts, noting that a working group toexamine them had just been set up by UKCGE. It didhowever put forward a view that this form of awarddid not ‘fit neatly’ within the kind of doctoralframework involving training in carrying outresearch leading to the production of a piece oforiginal research. It argued that, while it might bepossible to chart a student’s progress in learningrelevant skills through the exhibition of works bythat student, there is ‘necessarily no viva voceexamination since there is no thesis to be examined,and in its absence there appears to be no basis inwhich to award a research degree, or to base anysystem of quality assurance’ (UKCGE, 1996, page 18).The authors are aware of institutions whereassessment regulations for research degrees havebeen revised such that this point is addressed: insummary, candidates have to submit a writtencommentary to accompany the artefact, analysingthe process and methods used and saying how thatartefact contributes to knowledge in the particulararea. Our impression is that such (or similar)assessment criteria are now common in institutionswhere creative arts doctorates are awarded.

Because of the inevitable constraints on space inwriting this current paper, we have followed the1996 original in addressing the issues surroundingdoctorates in the creative arts rather thanextending our discussion to include other notionsof ‘practice-based’ doctoral study.

Doctorates awarded solely on the basis ofcreative outputs There are problematic issues arising from viewsabout whether or not a doctorate can be awardedsolely on the basis of the production of a creativework(s) assessed by knowledgeable peers,experienced in the field, as worthy of note asexcellent and as contributing to knowledge in thatfield. Since the 1996 Report there has been a debateabout this issue with perhaps an emergingconsensus suggesting that, however much criticalacclaim can be agreed upon, the work(s) requires anintellectual contextualisation and criticalinterpretation if it is to be deemed worthy of adoctoral award. However, we should note that this isnot universally accepted, particularly perhaps withinthe domain of music (and the award of DMus).

Contribution to knowledge through practice There is a contrary view to the one put forward inthe 1996 Paper, which suggests that doctoral levelstudy may be accommodated within all intellectualdomains, including the arts, and therefore that aPhD award can be made in any such domain (seeGreen and Powell, 2005, Chapter 6, for a fulldiscussion). In this view the PhD is a generic awardmade for contribution to knowledge of whateverkind and within it may be a distinctive sub-groupof areas that are rooted in practice and where, moreimportantly, the contribution to knowledge may bemade through the practice itself. What is involvedhere then is a ‘creative product’ and a differentorientation to the process of researching. This kindof doctorate is not, therefore, merely a doctorateawarded in an area that includes an element ofpractice but rather one involving a kind of studywhere knowledge is advanced by means of thepractice itself.

The UKCGE survey of Professional Doctorates inthe UK (Powell and Long, 2005) indicated that alarge (and at the time of that publication everincreasing) number of professional doctorateshave developed in response to professionalpractitioner needs, for example in psychology andother health disciplines, in business, and in

23

engineering. This brings new challengesthroughout the range of concerns about doctoraleducation. Such challenges are perhapsexemplified by the use of portfolios for assessmentin areas such as engineering and the arts wheremany examiners used to examining PhDs expecta thesis or dissertation and are faced with aportfolio that looks somewhat different and in turnneeds addressing in a different way. We are notsaying here that portfolios do not represent anappropriate academic level but rather that the wayin which contribution is presented in them differsfrom that in the traditional thesis/dissertation andtherefore ways of evaluating that contributionneed to vary accordingly.

Quality assurance and purpose of the award In considering the assurance of the quality of theaward of a practice-based doctorate within thecreative arts it is necessary to revisit its purpose.There are three dimensions to purpose here.

(i) If the PhD is inseparably related to training inhow to do research then assessment of thecreative output should include a judgmentabout whether the candidate is able to conductindependent research. While a creative outputmay indicate that the creator has engaged insuch training (e.g. has investigated, designed andmade use of research techniques) such anindication will remain implicit and hence notamenable to assessment unless the candidatesets down an account of how the research goalswere achieved. In essence this is what is requiredwithin a traditional PhD submission where thecandidate sets down the outcomes of theresearch programme but also a discursiveaccount of how those outcomes were achieved.

(ii) If dissemination of findings is a necessaryrequisite of successfully undertaking researchand, in turn, of being awarded a higher degreefor doing it, then successful research at doctorallevel must result in a contribution toknowledge that is realised when it iscommunicated effectively to others in the field.

The point of debate in this dimension iswhether the contribution can becommunicated adequately without asupporting discursive account that describeshow the research that underpinned thecreative act(s) was designed and carried out.Arguably, the acid test in this dimension mightbe that others working in the field need to beable to learn from the creative output if it is tobe acceptable as a contribution to knowledge.

(iii)A doctoral examination typically takes the formof the candidate defending argument set outin the thesis. Here there is a distinction betweena creative work that is presented withoutexplanation or critique and one presented withan attendant critical analysis. Both might havesome legitimacy in their own right; however,while the former conveys meaning to itsaudience and has a potential to make asubjective impact on that audience, the latterconveys critical explanation about, and analysisof, that meaning and that impact. The questionthen arises for the examiners: can the merits ofthe creative work be measured against thecriteria for the award by examining the creativeworks alone?

In terms of ensuring the standard of the academicaward, we suggest that the latter rather than theformer that can be deemed an academic exerciseand thus worthy of an academic award. This is not,of course, to deny the intellectual effort required ina creative act but rather to focus of the needs of anyassessment process leading to an academic award.

There is a contrary view that we should note hereand which suggests that the intellectual positionand its defence can be embodied in the creativeoutput. This view is perhaps most prevalent in thearea of musical composition (see Green andPowell, 2005). The debate about how an artefactmay be understood in relation to any conceptsthat are being communicated (i.e. whether anartefact in itself can embody knowledge or ifunderstanding of it only comes about through its

24

contextualisation) is summarised in Green andPowell (2005, pages 105-111) and in Biggs (2002).

Definitions within the wider academic field Definitions of research in the arts may be still opento debate within the field but the wider academiccommunity (e.g. Higher Education FundingCouncil of England Research Assessment Exerciseand the UK Arts and Humanities Research Council(AHRC)) seem to have consolidated a view thatsuggests that, to qualify as research, artisticprojects should specify and be located within a‘research context’. More recently the topic isexamined in the AHRC Research Review – Practice-led Research in Art Design and Architecture (2007)(available on line athttp://www.ahrc.ac.uk/About/Policy/Documents/Practice-Led_Review_Nov07.pdf).

Further, HEFCE (1999) requires that the workshould be open to critical review and that itshould be possible to judge its impact andinfluence on the work of peers. AHRB requires thatthe work should specify and justify the particularresearch methods chosen. Of course, researchcontext, methods and impact are often notdefined or are ill-defined, nevertheless theimplication is clear – if there is to be adequatequality control over academic awards in this areathen for artistic endeavour to be ‘counted’ asresearch, and therefore be amenable toassessment for a research degree, it would needto involve critical reflection on both processes andproducts.

Summary of our view in relation to doctorateswithin the creative artsTo summarise our view therefore: it seemsreasonable to suggest that, if they are to haveparity with other doctoral degrees, practice-baseddoctorates within the creative arts should standscrutiny under the same standards of assessmentas any other kind of doctorate. The criteria used toattain the standard might differ in kind but itshould be possible to relate them to other doctoralcriteria in terms of level.

Successful candidates will be expected todemonstrate that they have learnt to place theirwork within the arena of other related work and tojudge its impact on related areas of creativeendeavour; they will be able to demonstrate thatthey have understood how they have appliedparticular methods (both conceptual andpractical) in the production of their creative worksand justify why choices of those methods weremade; they will be able to communicate both theirmethodologies and their resulting works in anexplanatory and analytical way to an audience ofcritical peers; they will be able to develop, set downin a written text and defend an intellectualposition relating to their creative works bysustained argument.

Further, the doctoral qualification demands thatthey be able to continue that contribution in anindependent and self-critical way. Finally, in asmuch as the award of a doctorate signifiesadmission to a community of scholars, then in ourview no area of intellectual endeavour such as thecreative arts should be excluded from this processon the grounds that assessment is difficult toconceptualise and organise and because it mayinvolve subjectivity.

3.3.2 Professional doctorates

A third area for debate noted in the 1996 Report isthe notion of a (wholly or largely) ‘taught’doctorate. The report noted an extension of taughtdoctorates to a wider range of professions hasbeen especially in the United States and slowlydeveloping in the UK, ‘for example in education,clinical psychology and engineering’ (UKCGE, 1996,page 18) The Report suggested that ‘steps shouldbe taken, for example by ‘giving them a title otherthan PhD, to indicate that they are forms ofprofessional advanced training rather than researchbased qualifications’ (UKCGE, 1996, page 18). Itseems to us that this is one area in whichunderstanding has moved on since 1996. The useof ‘taught doctorates’ synonymously with‘professional doctorates’ is not helpful where the

25

distinguishing features of this latter grouping ofdoctorates relate to focus of intellectual enquiryand kinds of contribution to knowledge ratherthan necessarily to mode of study. Whether or notthere are some elements of study that involveattending ‘taught’ sessions is not the issue. Rather,it is about the graduate’s attributes and abilities atthe end of the programme being at doctoral level(and therefore being in alignment with thedoctoral qualification descriptor). The term ‘taughtdoctorate’ is also unhelpful when trying to assureparity of esteem between professional doctoratesand the PhD, especially in a global context. Giventhe world-wide competitiveness for doctoralcandidates, we suggest that it is important for theUK to protect its doctoral ‘brand’ and that this isinhibited by referring to ‘taught doctorates’, whichcould also be viewed as a contradiction in terms.

In relation to a concern with the quality of thelearning that takes place and the kinds of outcomesthat may accrue what matters is the level of skillsand contribution that is attained by the successfulcandidate. For many in the sector a ‘taughtdoctorate’ is a contradiction in terms. As we havealready noted to attain doctoral level necessarilyinvolves making a contribution to an intellectualfield and showing evidence of being able tocontinue to make such a contribution and in thissense it is not possible to define a ‘syllabus’ that willlead to such a level in the same way that it ispossible in taught courses. Having said that, we havealso recognized that research degree students are,in one sense, taughtby supervisors and other senioracademic colleagues, and quite possibly throughattending lectures and seminars. They also learnfrom their peers. It is the nature of the learningoutcomes that separates out the traditional notionof ‘taught courses’ from ‘research degrees’ ratherthan the forms that learning may take.

3.4 The doctoral programme

The 1996 Paper considers two main issues underthe heading of ‘doctoral programme’ (i) researchmethods and generic skills and (ii) supervisor

training. Its main conclusions are that prescriptionin either of these areas is unlikely to enhancequality because of the diversity of needs in relationto both students and disciplines. While it iscertainly the case that doctoral study is a diverseoccupation and hence uniform requirements areunlikely to work, it may also be argued that thereare certain principles that may apply across theboard. Since 1996 the introduction of the revisedsection 1 of the QAA Code of Practice - ResearchDegree Programmes has to a large extent mappedout what may count as ‘effective practice’ inrelation to (i) giving students access to a range oflearning opportunities that encompass generic aswell as subject specific skills and (ii) givingsupervisors access to training that focuses uponthe requirements of the specific educative processat research degree level.

In our view the focus of concern when seeking toassure quality of learning within a programme ofstudy at research degree level needs to be on thespecifics of the learning required of the student toattain the expected level of academic achievementand subsequently on how that learning can bestbe achieved in terms of environment andpedagogy. It is erroneous to focus on generic skillswithout first defining what they might be, whythey might be valuable to individual researchersand how best they can be attained.

Subsequent to this comes the question of how theacquisition of those skills may best be assessed.This latter issue is often overlooked, yet in terms ofthe quality of the student’s learning experience, itis hard to justify an assertion of something’susefulness without indicating a way of measuringhow readily it has been learnt and how usable it isin the learner’s continuing experience. Similarly, tosay that supervisors should be trained is nebulouswithout an engagement with what it is that makesresearch degree supervision different from anyother kind of university teaching and thereforewhat it is that potential supervisors need to knowand to be able to do (QAA Code of Practice, Section1, Precepts 11 to 14).

26

3.5 Assessing the doctorate

The 1996 Report made four main suggestions withregard to the examination of doctoral candidates.First, that there is ‘little purpose in requiring theholding of a viva if it is clear to the examiners that thecandidate will pass’ (page 20). Second, that makinguse of external examiners from outside of the UKwould ensure ‘some sort of international equivalencein the standard required for the award of a researchdegree’ (pages 20 and 21). Third, that Codes ofPractice for the conduct of examinations should be‘actively encouraged’ (page 21) and similarlymechanisms for appeal against examinationoutcomes should be clearly available to candidates(and these mechanisms should include makingexaminers reports available to candidates). Four, thatexaminers might be enabled to comment on thestandard of doctoral education that the candidatehas received though the report does not advocateroutine comment from examiners on generalstandards of provision. We revisit these issues below.

3.5.1 Why an oral examination?

It is recognized in the literature (e.g. Green andPowell, 2005; Powell and McCauley 2002, 2003;Tinkler and Jackson 2004) that institutions vary inrespect of their use of the viva as an essential ornon-essential component of the doctoralexamination process – though for the majority ofUK universities the viva voce is seen as an integralpart of the process (albeit one where, in someinstitutions, exceptions may be made, for exampleto accommodate particular medicalcircumstances). In our view the questions needconsidering in terms of the quality control of theexamination process – its reliability and its fairness.

In as much as doctoral candidates should (again,in our view) set out their ‘thesis’ – their intellectualposition – which they then defend in their writtensubmission, then the viva becomes a matter ofenabling the examiners to pass judgment on theviability of the intellectual position and oncandidates’ understanding of it and their ability to

continue the defence that they have set out inwriting, in the face of an individual, oral, cross-examination.

For the quality of the doctoral award to be ensuredthere is a need for this final stage if part of thenotion of a ‘doctor’ is that the holder of such a titlecan initiate research that contributes to knowledgeand, importantly, be aware enough of the contextand the implications of their work to explain anddefend it and continue to initiate work of a similarstandard. In short, for the quality of the award to beensured then the examiners need to be sure thatnot only has the candidate contributed toknowledge but also that he/she can continue to doso in an independent and sustained way. This latterjudgment can best be made in an oral contextwhere questions can be asked and chances tojustify and expand can be given. An examinationwhere no oral component exists is open to thecharge that while the former condition might bemet (the written submission contributes toknowledge) the latter condition (of independentcontinuation) cannot adequately be met.

In short, the need for a viva as part of doctoralexamination cuts to the heart of what the award isabout. The decision as to the level of necessity fora viva therefore matters in terms of (a) how auniversity perceives the doctoral award and (b) thecontrols it places on the quality of successfuldoctoral candidates.

3.5.2 International equivalence

In one sense it seems straightforward to advocatethe involvement of international examiners toensure some notion of parity of doctoral awardsacross national borders. Indeed this is commonpractice in many other countries (e.g. Australia andSweden). Yet the question for those concerned withthe quality of the process is the extent to which anexaminer from overseas is judging like with like ifhe/she applies the standards of his/her own countryto the examining of a UK doctoral candidate. This isnot of course to argue for any imbalance in level

27

necessarily but that processes of doctoral studydiffer across international boundaries to such anextent (see Green and Powell, in press) that whatmay be expected in terms of the actual examination,if not in terms of the written submission, will alsodiffer. As a matter of ensuring the quality of theprocess of examining therefore, any involvement ofan examiner from overseas should incorporate someguidance on the nature of study undertaken, thecriteria for the award, the expectations of theexamination process and the qualities within thecandidate that are deemed necessary for success. Ofcourse, such guidance is necessary for all examiners(and we include here internals) though we wouldargue that guidance for overseas examiners requiresgreater contextualization.

Returning once more to Bologna and relateddevelopments, it has been suggested that UKdoctoral examinations might be improved if theywere more closely aligned to practice incontinental Europe. For example, in manyEuropean countries, a public defence of the thesisis required, which can address any doubts aboutthe candidate’s ability to present and defendhis/her arguments, at the same time assuring theoriginality of the work and sharing it with a wideraudience. To our knowledge, there are no movesto adopt such models at present in the UK.

Taking an international perspective, it is not auniversal requirement for doctoral candidates todefend their thesis (either in public or in a closedexamination) in order to be awarded the degree. InAustralia, for example, it has never been the normfor candidates to undergo an oral examination:assessment is based on examiners’ reading andcommenting on the candidate’s work. It seemsunlikely that the UK or other European countrieswould wish to move to a remotely assessed modelfor doctoral degrees as the defence of the thesis iscentral to the examiners’ being assured of thecandidate’s depth and breadth of knowledge andunderstanding, as well as the original nature of theresearch. The oral examination also enablesexaminers to check for plagiarism.

3.5.3 Codes of practice

Since the 1996 UKCGE Report the QAA Code hastaken forward the notion of Codes of Practice andthe importance of transparent appeals processes.We do not wish to diminish their importance –clearly they are central to any quality regime - butfor the purposes of this current report we willaccept such things as ‘givens’ within the UK scene.They are discussed in detail in the QAA Code andwe will therefore not duplicate that discussion here.

3.5.4 Remit for examiners

In our view the remit that is given to examinersneeds to reflect sharply the nature of the task that isrequired of them and the focus that they need tobring to bear on that task. To ask an examiner to passcomment on the whole of a department’s researchtraining provision (or indeed of such provisionuniversity-wide) is to potentially misrepresent theirtask and provoke unfair comment if they are basingtheir comments on judgments of just one case outof a much larger number. Of course if they areexaminer of cohorts of doctorates (e.g. of aprofessional doctorate programme) then they mightbe expected to make such judgments in the sameway as examiners of taught programmes might beexpected to do. But in the normal course of eventsan examiner is required to make complex decisionsabout a (if successful) high level of performance.Whatever processes of learning and teaching havegone on and whatever resources have beenprovided, doctoral candidates have to stand or fallon their own ability to decide on their thesis and onthe best way to present and defend it. To ask anexaminer to go beyond the decision-making on thecentral point of concern (the doctoral level of theindividual) is to confound process with product.Doctoral study does not involve a predeterminedcurriculum (which might be open to criticism) nor aset pattern of pedagogic delivery (which similarlymight be open to evaluation) therefore the usualkinds of judgment making undertaken by examinersof taught programmes are not applicable.

28

4. The Assessment of Quality

In the current climate, where the need foraccountability permeates the way in whichinstitutions are expected to behave, there is an everpresent demand for quality to be assessed. The actof assessing implies that some kind of measurementis possible and it is arguable if ‘quality’ is somethingthat is amenable to measurement in this way.Nevertheless the 1996 UKCGE paper engaged withthe notion and separated out such assessment intodimensions in an effort to make measurement of akind manageable.

There are several dimensions to the way in whichthe quality of higher educational provision ingeneral may be assessed and here we describethose dimensions within the context of researchdegree study, following from the 1996 paper.

� The kinds of inputs that an institution makesto the research degree process can bemeasured and hence assessed, for examplejudgments can be made about theappropriateness of the human and physicalresources that are applied within the researchdegree context

� The kinds of processes that are in place withinan institution can be judged, for example thecharacteristics of supervisory arrangementscan be measured against agreed criteria

� The outputs of research degree study, forexample the number and rate of success ofdoctoral submissions and the destinations ofresearch degree students can be measuredagainst agreed criteria

It is arguable that all three of these dimensions areimportant in any attempt to understand thequality regime of an institution. It is equally thecase that some dimensions are more amenable toassessment than others – notably it is relativelystraightforward to measure outputs such asnumbers of progressions and number ofsuccessful (examination) outcomes within a setperiod of time. It is easier to assess the outcomes

of students’ study than the input that theyreceived and the processes of study through whichthey went. The readiness of outputs to bemeasured in this way may lead to an unbalancedoverall approach in which the importance of inputand process factors is underestimated. Indeed, ifoutput factors alone are used to assess quality ofprovision then what results will not be a genuineassessment of the quality of the student's learningexperience and it is perhaps worth re-iteratingwhat was said in the 1996 UKCGE paper – namelythat such a regard for the learning experience has‘been the primary focus for the assessment of thequality of taught programmes in higher education’.We can assume that parity in these quality matterswith what goes on in taught programmes isdesirable (if not similarity of approach – taughtand research may require different approaches toreach a level of quality provision that has parity).

The 1996 UKCGE paper went on to list factors inthe three dimensions of input, process and output(pages 24 - 30). Interestingly, all these factors areaddressed in greater or lesser specificity in theQAA Code of Practice with two exceptions:

(i) In the 1996 paper UKCGE made some play of‘taught elements’ within doctoral programmesand noted quality concerns in that respect. Therevised QAA Code of Practice makes nomention of such elements, except once whenmentioning skills training; however, the qualityconcerns with which the Code deals doencompass - in spirit if not explicitly- anytaught elements that might exist within aprogramme of research study

(ii) One output factor suggested by UKCGE is:‘career progression as measured, for example,by information 5 years post-PhD’. In this regardthe Code does mention career progression butrather as something that is part of the processof learning rather than an output measure

It seems to us that neither of these exceptions issignificant in terms of what is covered and, to

29

summarise, the revised QAA Code of 2004addresses all the main areas considered importantby the 1996 UKCGE report. What remains as aquestion however is how the current qualityregime can ensure that a balance is achievedbetween the focus on inputs and processes on theone hand and outputs on the other?

5. International Dimensions

The final section in the 1996 UKCGE paper dealtwith the international dimension to the UKdoctorate stressing the importance of the quality ofthe UK doctorate both in terms of its standingabroad and its standard in relation to countriesoverseas. It also made the point that comparisonsbetween doctoral training here in the UK and thatabroad are hard to sustain where there arefundamental differences in conception of doctoralcharacteristics and hence of doctoral trainingrequirements. Since the 1996 paper somepublications (e.g. Powell and Green, 2007) haveexplored those differences and found them to bepersisting despite the general drive towardsharmonisation that comes from the EU. Certainlythere are moves to regularize frameworks forresearch degree study – particularly in terms of theplace of that study in relation to other levels of study(i.e. the Bologna and subsequent agreements) butdifferences in, for example, the way in whichdoctoral submissions are judged persist (e.g. thereare distinctive notions, between countries, of‘defence’ on the one hand and ‘examination’ on theother that share some characteristics but which aredivided by fundamental differences of view in termsof just what is being assessed at the examinationphase and how it may be judged).

6. Conclusions

The 1996 UKCGE Report concluded with a call fora wider debate about quality issues inpostgraduate research education and by theinvolvement of other named organisations. It is

interesting to re-read that call now as many of theorganisations have changed names or developedinto new forms with shifted responsibilities. Themain development being the publishing of theQAA Code of Practice three years after the UKCGEpaper – in 1999 and then its revision and thepublication of a 2nd Edition in 2004. Both of thesemoves were in line with what UKCGE wasadvocating at the time.

In one sense it seems futile to try to revisit thequestions of which organisation should bearwhich responsibility for quality assurance ofresearch degrees provision given the currentlandscape. However, it does seem prescient tomake some comment about the drawing of qualityassurance under the umbrella of a nationalorganisation with designated responsibility forthat assurance. We make that comment belowalong with some conclusions from the discussionswe have set out in this paper.

6.1 The effects of monitoring

Drawing quality assurance under the umbrella ofa national organisation with designatedresponsibility for that assurance means thatnational standards of research degree provisioncan be determined, described and communicatedin a way that is understandable by all concerned(including students) and then an institution’sattempts to meet those standards can bemonitored. The issue that may arise, and which wehave noted earlier in this paper, relates to theknock on effects of this monitoring process. For themonitoring to be effective it requires that thenational organisation assesses how well aninstitution is matching up to the precepts that itlays down. That process of assessment may lead tothe institution focusing on how best to describewhat it does in such a way as to meet the apparentcriteria that underpin the assessment and do so ina way that is compatible with the reporting cultureof the national organization. This would result in achange of description rather than a change inwhat it does so that those criteria are met.

30

The current situation, in England at least, is thatseveral organizations are involved in monitoringthe quality of research education. As noted above,the Research Councils, QAA and HEFCE all havedifferent perspectives on the quality of doctoraleducation and graduates, measured by distinctprocesses. However, it is always the highereducation institution in whose name the doctorateis awarded that is responsible for maintaining theacademic standards of the qualification and thegraduates thereof.

6.2 In-built self evaluation

Any system should seek to raise, in a progressiveway, the threshold standard. This is an integralaspect of any quality system – the notion of aninbuilt self-evaluative structure that seeksfeedback and accommodates it within acontinuing cycle of review and revision of bothprinciples and practices.

6.3 Expectations

For a quality learning environment to exist in aninstitution there needs to be an argued throughand thought out stance with regard to what can beexpected of students and what the institution’sresponsibilities are in enabling the student to meetthose expectations. For example, raisingexpectations of achievement without addressingissues raised by predetermined time periods forthat achievement is, arguably, a significant concern.

6.4 Measurable performance indicators

Governments and/or institutions may seek tosimplify quality issues by producing clear andmeasurable performance indicators; however suchsimplification may lead to an inability to evaluatethe quality of research education, or at least mayonly allow a narrow aspect of it to be judged in aninflexible way. The issue with any specific targetsetting with regard to standards is how to ensurethat meeting the standard will in itself realize anincrease in quality of the student experience.Where such realization is not automatic then

raising quality becomes a matter of an institutionmaking use of the specificity of guidance to informits practice and this in turn requires a reflectivemode of operation.

31

Biggs, M. (2002) The role of artefact in art anddesign research. International Journal of DesignSciences and Technology, 10(2):19-24

Clarke, G. (2008), Analysis of responses to QAAconsultation on doctoral discussion paper (May2007), web published only on QAA website at:http://www.qaa.ac.uk/academicinfrastructure/doctoralProg/progressMarch08.aspFurther results to be published as and when completed.

Felton, S.P. (2008) Why do a postgraduate researchdegree? In: G. Hall and J. Longman (eds) ThePostgraduate’s Companion. London, Sage.

Green, H. and Powell, S.D. (2005) Doctoral Study inContemporary Higher Education Buckingham, OpenUniversity Press. ISBN 0 335 21473 8 (pb) ISBN 0335 21474 6 (hb)

Harris, M. (1996) Review of Postgraduate Education.Higher Education Funding Council for England,Committee of Vice-Chancellors and PrincipalsStanding Conference of Principals HEFCE, ref: M14/

Higher Education Funding Council of England(HEFCE) (2003) Improving Standards in PostgraduateResearch Degree Programme Formal Consultation.HEFCE 2003/23. London

National Qualifications Authority of Ireland(October 2006) Review of Professional Doctorates,pages 3-5, available from the web:http://www.eua.be/eua/jsp/en/upload/Review%20of%20Professional%20Doctorates_Ireland2006.1164040107604.pdf

Nerad, M. (2007) Doctoral Education in the USA, in:S.D. Powell and H. Green (Eds) The DoctorateWorldwide. Buckingham, Open University Press.(ISBN: 978-0-335-22021-2)

Office of Science and Technology (OST) (1994)Consultative Document: A New Structure forPostgraduate Research Training, Supported byResearch Councils, Cabinet Office, London

Park, C. (2007) Redefining the Doctorate.York, HigherEducation Academy. (ISBN: 978-1-905788-29-3)

Pearce, L. (2005) How to Examine a Thesis.SRHE/Open University Press. page 119 ISBN:0 33521443 6

Powell, S.D. (2004) The Award of PhD by PublishedWork in the UK. Lichfield, UK Council for GraduateEducation. ISBN 0-9543915-2-7. page 81.

Powell. S.D. and Long, E. (2005) ProfessionalDoctorate Awards in the UK. Lichfield, UK Councilfor Graduate Education. ISBN 0-9543915-4-3

Powell, S.D. and Green, H. (Eds) The DoctorateWorldwide. Buckingham, Open University Press.(ISBN: 978-0-335-22021-2)

Powell, S.D. and McCauley, C. (2003) The Process OfExamining Research Degrees: Some Issues OfQuality Quality Assurance in Education (SpecialEdition: ‘Assessing and Examining ResearchAwards’), Vol 11, (2), pages 73-84. ISSN: 0968-4883

Powell, S.D. and McCauley, C. (2002) ResearchDegree Examining – common principles anddivergent practices Quality Assurance in Education(Special Edition ‘Standards and the Doctoral Award’),10, 2, pages 104-116. ISSN: 0968-4883

Quality Assurance Agency (QAA) (2004) Code ofPractice for the Assurance of Academic Quality andStandards in Higher Education: Section 1:Postgraduate Research Programmes (2nd Edition).Gloucester: Quality Assurance Agency.

Research Councils/AHRB (2001) Joint Statement ofthe Research Councils’/AHRB’s Skills TrainingRequirements for Research Students. Available fromthe web: www.grad.ac.uk/3_2_1.jsp.

Tinkler, P. and Jackson, C. (2004) The DoctoralExamination Process: a handbook for students,supervisors and examiners. Buckingham, OpenUniversity Press

References

32

Tinkler, P. and Jackson, C. (2000) Examining thedoctorate: institutional policy and the PhDexamination process in Britain, Studies in HigherEducation, 25 (2) 167-180.

UK GRAD Programme (2004) A national review ofemerging practice on the use of PersonalDevelopment Planning for postgraduate researchers.UK GRAD Programme, Cambridge.

UKCGE (2002) Professional Doctorates. Lichfield, UKCouncil for Graduate Education.

UKCGE (1996) Quality and Standards ofPostgraduate Research Degrees. Lichfield, UKCouncil for Graduate Education.

UUK, UK Higher Education International Unit,Research Series/2 (July 2008): The UK’s CompetitiveAdvantage: The Market for International ResearchStudents. Authors: Neil Kemp (Project Leader), WillArcher, Colin Gilligan, Christine Humfrey. Availablefrom the web:<http://www.international.ac.uk/our_research_and_publications/research/published_research.cfm>

Walker, G. (2001) Rethinking the Doctorate in the21st Century. Carnegie Initiative on the Doctorate,650/566-5100. Available from the web:http://www.carnegiefoundation.org/CID.

33

ISBN Title

0-9525751-0-8 Graduate Schools (1995)

0-952-5751-1-6 The Award of the Degree of PhD on the Basis of Published Work in the UK (1996)

0-9525751-9-1 Quality and Standards of Postgraduate Research Degrees (1996)

0-9525751-2-4 Practice-Based Doctorates in the Creative and Performing Arts and Design (1997)

0-952-5751-3-2 The status of published work in submissions for doctoral degrees in European Universities (1998)

0-952-5751-4-0 Preparing Postgraduates to Teach in Higher Education (1999)

0-952-5751-5-9 The International Postgraduate: Challenges to British Higher Education (1999)

0-952-5751-67 Research Training for Humanities Postgraduate Students (2000)

0-9525751-75 Research Training in the Creative and Performing Arts and Design (2001)

0-952-5751-83 Professional Doctorates (2002)

0-9543915-0-0 Research Training in the Healthcare Professions (2003)

0-9543915-1-9 A Review of Graduate Schools in the UK (2004)

0-9543915-2-7 The Award of PhD by Published Work in the UK (2004)

0-9543915-3-5 Confidentiality of PhD Theses in the UK (2005)

0-9543915-4-3 Professional Doctorate Awards in the UK (2005)

0-9543915-5-3 Access to Doctoral Examiners’ Reports (2007)

0-9543915-6-0 Higher Doctorate Awards in the UK (2008)

UK Council for Graduate EducationPublished Titles

34

First published in 2009 by the UK Council for Graduate Education

© UK Council for Graduate Education

ISBN 978-0-9543915-7-7

UK Council for Graduate EducationLichfield CentreThe FriaryLichfieldStaffordshireWS13 6QG

T: 01543 308602F: 01543 308604E: ukcge@ukcge.ac.ukwww.ukcge.ac.uk