Peer Review Teaching in Higher Education: An Application of the SWOT Framework- Dr Sony, Dr Susan,...

7/29/2019 Peer Review Teaching in Higher Education: An Application of the SWOT Framework- Dr Sony, Dr Susan, Dr. Thomas

1/49

http://rer.aera.netResearch

Review of Educational

http://rer.sagepub.com/content/early/2013/09/16/0034654313499617The online version of this article can be found at:

DOI: 10.3102/0034654313499617

published online 17 September 2013REVIEW OF EDUCATIONAL RESEARCHSusan Thomas, Qiu Ting Chie, Mathew Abraham, Sony Jalarajan Raj and Loo-See Beh

Education: An Application of the SWOT FrameworkA Qualitative Review of Literature on Peer Review of Teaching in Higher

Published on behalf of

American Educational Research Association

and

http://www.sagepublications.com

can be found at:Review of Educational ResearchAdditional services and information for

http://rer.aera.net/alertsEmail Alerts:

http://rer.aera.net/subscriptionsSubscriptions:

http://www.aera.net/reprintsReprints:

http://www.aera.net/permissionsPermissions:

What is This?

- Sep 17, 2013OnlineFirst Version of Record>>

by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from by guest on September 22, 2013http://rer.aera.netDownloaded from
http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.sagepub.com/content/early/2013/09/16/0034654313499617http://rer.sagepub.com/content/early/2013/09/16/0034654313499617http://www.aera.net/http://www.aera.net/http://www.sagepublications.com/http://rer.aera.net/alertshttp://rer.aera.net/alertshttp://rer.aera.net/subscriptionshttp://rer.aera.net/subscriptionshttp://rer.aera.net/subscriptionshttp://www.aera.net/reprintshttp://www.aera.net/reprintshttp://www.aera.net/permissionshttp://www.aera.net/permissionshttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://rer.sagepub.com/content/early/2013/09/16/0034654313499617.full.pdfhttp://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://rer.aera.net/http://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://rer.sagepub.com/content/early/2013/09/16/0034654313499617.full.pdfhttp://www.aera.net/permissionshttp://www.aera.net/reprintshttp://rer.aera.net/subscriptionshttp://rer.aera.net/alertshttp://www.sagepublications.com/http://www.aera.net/http://rer.sagepub.com/content/early/2013/09/16/0034654313499617http://rer.aera.net/


2/49

Review of Educational Research

Season XXXX, Vol. XX, No. X, pp. 148

DOI: 10.3102/0034654313499617

2013 AERA. http://rer.aera.net

1

A Qualitative Review of Literature on PeerReview of Teaching in Higher Education: An

Application of the SWOT Framework

Susan Thomas, Qiu Ting ChieMonash University Sunway Campus, Malaysia

Mathew AbrahamUniversity of Nottingham Malaysia Campus

Sony Jalarajan RajSt. Thomas University of Florida

Loo-See BehUniversity of Malaya

The issues of professional accountability, faculty member development,and enhancing higher education quality in universities are gainingimportance. A strategy that could increase personal control over teaching

practices in addition to improving professional development among fac-ulty members is peer review of teaching (PRT). Five themes that areimportant in determining the feasibility of PRT are (a) benefits of peerreview in developing faculty members, (b) barriers to peer review ofteaching, (c) gaps in literature, (d) potential problems to teaching prac-tice, and (e) opportunities. Of the 65 studies identified, 34 were selected

for further analysis, and drawing on PRT and the SWOT (strength, weak-ness, opportunity, and threat) framework, 27 studies were selected forcontent mapping. Textual narrative synthesis was used to further catego-rize the review findings into the four quadrants of the SWOT framework.This analysis highlights a positive strategy in promoting PRT in highereducation.

Keywords: peer review, observation, reflection, peer feedback, content-map-ping, SWOT framework

RER499617RER10.3102/0034654313499617Thomas et al.Peer Review in Higher Educa-

tion


3/49

Thomas et al.

2

In recent years, governments and universities are increasingly focused on thequality of education whereas parents and students are more concerned about themaintenance or upgrading of education standards. However, the fact that the qual-ity of education depends on giving faculty members more control of their practice

is seemingly neglected (Murray & Grant, 1998). Developments in the scholarshipof teaching and learning have seen the change in focus from an information trans-mission approach to a quality learning approach. This change suggests that theemphasis on facts and mastering information has given way to active forms oflearning, which require students to understand subject materials deeply and engagein making meanings (Hutchings, 1996). Similarly, there is a need to change thetraditional method of teaching evaluation to a more collegial design. As such, peerreview and tailored evaluation interventions are increasingly proposed in highereducation entities as alternatives to improve the evaluation process and teachingquality (Murray & Grant, 1998), especially to support accountability processes(University Teaching Development Centre, 2004).

Peer review of teaching (PRT) includes the observation of lectures and tutorials,monitoring online teaching, examining curriculum design, and the use of studentassessments (Hatzipanagos & Lygo-Baker, 2006). The essence of PRT is aboutfurthering the development of faculty members through the expert input based onknowledge and understanding, although it can be used as part of performanceappraisal and tenure portfolios (Kohut, Burnap, & Yon, 2007). PRT also sharpensindividual skills, such as the ability to observe as well as critically reflect on thedynamics and social context of teaching (Peel, 2005).

Statement of Purpose

The main purpose of this review is to map past studies on PRT in higher educa-tion. Second, the review attempts to highlight research gaps and issues with pastliterature on PRT and SWOT (strength, weakness, opportunity, and threat) analy-sis. Third, the feasibility of PRT is analyzed using the SWOT framework. It ishoped that the SWOT framework will provide an objective and critical perspectiveof the PRT concept as a whole.

The review begins with an introduction describing the significance of PRT forteaching in higher education. This section is followed by a discussion of the pur-

pose of this research and the problems identified with PRT and the SWOT litera-ture. A description of the procedures to identify resources and studies, the keywordsapplied, as well as the inclusion and exclusion criteria are also documented.Subsequently, research documenting the implementation of peer observation andPRT, giving and receiving feedback, as well as reflective practice within an institu-tion are content-mapped to derive themes that will help focus the review towardthe formation of the SWOT framework of PRT. The strengths, weaknesses, oppor-tunities, and threats that surface from conducting PRT will be derived throughtextual narrative synthesis, and the outcome of the analysis is reported. This reviewends with a short discussion on the implications of applying the SWOT framework

to PRT.

Problem Statements

An examination of PRT trends shows that peer observation is a recent develop-ment in U.K. universities after its successful introduction in the United States and


4/49

Peer Review in Higher Education

3

Australia (Lomas & Kinchin, 2006). As PRT is a relatively new practice in highereducation, there is some difficulty locating research detailing PRT performance ascompared to peer review associated with research journal publications. Prior tothis, the PRT in itself is not a dominant practice in higher education institutions due

to a lack of awareness of its impact on teaching performance and how PRT canhelp faculty members reflect and develop a deeper understanding on teaching phi-losophy (Hurst, Wilson, & Cramer, 1998).

Therefore, a shift in the view and beliefs about teaching is needed since there ismore to teaching than just technique. Course design, types of assignments, andstudent assessment criteria are reflections of the teachers perception about thefield of study. Since teaching is a scholarly work, PRT has the capability to captureoverlooked scholarly aspects of teaching (Boyer, 1990). An examination of recentattitudes toward PRT found more positive attitudes among faculty members fromthe liberal arts (Keig, 2000). A majority of these faculty members believe that theaccuracy of teaching assessment, the objectivity of assessment, and academic free-dom will not be compromised by PRT (Keig, 2000). According to Boyer, facultymembers within some disciplines have the tendency to be too focused on teachingtechniques, and many of them have the assumption that valuable input on teachingcan only come from peers within the same discipline. However, Boyer believesthat this can be overcome through frequent engagement in cross-disciplinary dis-cussions, debates, and exchanges. A main concern is about who is qualified to bea peer reviewer. The question is, Can reviewers without subject expertise provideequally valid feedback as reviewers who are with subject expertise. Hanson

(1993) addressed this concern through a pilot study in a PRT program by having asubject specialist evaluate a group of faculty members. The evaluation revealedthat feedback from nonspecialist appraisers and subject specialists are equally reli-able and valid.

Although reflective practice is accepted internationally as part of professionalcompetency for teachers, there is a lack of awareness among faculty members inhigher education about what constitutes reflective practice (Hammersley-Fletcher& Orsmond, 2005). There is insufficient emphasis on the process of reflectiveactivities, which involves knowledge about the action and self to create enhancedmeaning (Brockbank & McGill, 1998). Reflective activities also involve recreat-

ing enhanced meaning through the interplay of social and personal knowledge,along with experiential and conceptual insights (Kolb, 1984). The role of peers inaltering understanding and enhancing self-awareness in addition to the misconcep-tion about reflection as an individualistic activity needs to be underlined to facultymembers. Having PRT in higher education can encourage a deeper understandingon the role of peers as helpers who provide the critical energy needed for change(Barnett, 1997) and contribute to their colleagues professional development.

Another concern among faculty members is about what and how they will beassessed in PRT. The PRT process can be enhanced if acceptable standards of goodteaching assessment are established (Keig, 2000). Ramsden (1992) identified 13

characteristics of good teaching: (a) a desire to share the love of the subject, (b) theability to make the material stimulating and interesting, (c) the facility to engagewith students at their level of understanding, (d) a commitment to ensure clarity inwhat has to be understood and its reasons, (e) demonstrating concern and respectfor students, (f) encouraging student independence and experiment, (g) the ability


5/49

Thomas et al.

4

to improvise and adapt to new demands, (h) promoting active and cooperativelearning through teaching methods and academic tasks, (i) the use of valid and fairassessments, (j) providing high-quality feedback to students, (k) emphasizing keyconcepts, (l) a focus on current and future understanding, and (m) demonstrating

the desire to learn from others about ways to improve teaching.Through the years, many researchers have attempted to define principles of

effective teaching (Kember & McNaught, 2007; Saroyan et al., 2004; Young &Shaw, 1999). Young and Shaw, for instance, proposed six dimensions of effectiveteaching based on 912 students rating of a college teacher of their choice, whereasKember and McNaught (2007) derived 10 principles of effective teaching fromtheir research on 44 teachers from Australia who were nominated as exemplaryteachers and 18 teachers from Hong Kong who received the Vice-Chancellorsaward for exemplary teaching. In a review of literature on effective teaching,Hativa, Barak, and Simhi (2001) listed several overlapping characteristics, such as(a) being well-prepared and organized, (b) providing clear presentation of mate-rial, (c) the ability to fuel students interest, (d) engagement, (e) encouragement,(f) positive rapport with students, (g) motivation, (h) enthusiasm in studying teach-ing materials, (i) demonstration of high expectations, and (j) maintenance of apositive learning environment. The question is which principle best fits the overallcharacteristics of good or effective teaching. Comparatively, Ramsdens (1992)characteristics of good teaching contain all the overlapping characteristics ofeffective teaching, which are still used by many researchers today, includingHativa et al.

Nonetheless, Nicholls (2001) warned that faculty members should be aware ofthe fine distinction between teaching competence (i.e., efficiency and effective-ness) and cognitive understanding (i.e., content and academic competence). Thus,caution must be practiced when evaluating peers as the teaching characteristicsdefine only a part of the teaching competence. The primary purpose of effectiveteaching characteristics is to function as a guideline to provide clear specificationsof institutional objectives, with the aim of driving educational practice rather thanbeing used as a standard benchmark for teaching performance. Thus, faculty mem-bers need to be given autonomy on the interpretations of the characteristics to havegood teaching practices in their respective classrooms.

Therefore, this review attempts to incorporate the SWOT analysis as a reviewframework and method to analyze the feasibility of PRT. However, according toChermack and Kasshanna (2007), the fundamental issue with SWOT analysis isthat it emerged from practice and lacks a theoretical foundation. In the 1950s, theapproach used by Smith and Christensen in studying the relationship between theenvironment and organization shaped the basis of SWOT analysis, even thoughthere are no clear academic references to support the source of the word SWOT(Chermack & Kasshanna, 2007; Helms & Nixon, 2010). The SWOT concept wasused as a strategy tool developed by Learned, Christensen, Andrews, and Guth in1965 as a result of earlier efforts in analyzing case studies in Harvard Business

School (Chermack & Kasshanna, 2007). The basic framework of the originalSWOT table is seen in Table 1.

Second, although SWOT has been used in a number of areas such as healtheducation, social work (Sharma, 2005; Westhues, Lafrance, & Schmidt, 2001),


6/49

5

revisions to business undergraduate curriculum (Kuiper & Thomas, 2000), andstrategy formulation for vocational education (Lee, Lo, Leung, & Ko, 2000), theuse of a SWOT analysis in higher education is not highly published. The conclu-sions drawn from the Kuiper and Thomas and Lee et al. studies demonstrate thatthe SWOT model helps the principal stakeholders in higher education entities toidentify expertise that pose as strengths or opportunities and the shortcomingswithin the internal and external environment that pose as weaknesses or threats.By recognizing the areas in which they lack expertise, strategies can be developedto overcome weaknesses and thus increase the overall efficiency and efficacy ofthe planning process (Kuiper & Thomas, 2000).

Third, even though the SWOT is acknowledged as an established method forthe formulation of strategies (Dyson, 2004), as it simplifies complex issues intomanageable tasks, researchers also suggest the use of alternative methods or toolsin tandem with SWOT in strategy analysis (Helms & Nixon, 2010). Although theSWOT analysis framework has its drawbacks (Chermack & Kasshanna, 2007;Helms & Nixon, 2010), it is a useful tool for exploring possibilities, making deci-sions, brainstorming (Chermack & Kasshanna, 2007), and conducting first-levelinvestigations of internal and external environments that could favor or workagainst new concepts such as PRT. The direction of the PRT program and its oppor-

tunities for development lies closely with the objective of the program and theneeds of faculty members. On an individual level, faculty members have completeautonomy in determining steps to proceed after identifying their strengths, weak-nesses, opportunities, and challenges in teaching. Nevertheless, there may be anoverlap in the categorization of SWOT variables as PRT progresses (Helms &Nixon, 2010). For instance, strengths that are not maintained become weaknesses,whereas opportunities that are developed may become a weakness or threat.Alternatively, threats that are acted on efficiently may become opportunities.

To summarize, the synthesis of SWOT is a quick and easy method that couldhelp faculty members build on the strengths and opportunities gained from PRT as

well as eliminate the weakness and threats posed by PRT to their own unique cir-cumstances. Although it has its issues, the simplicity of its design allows an easygrasp of the four essential components needed to evaluate the feasibility of projectssuch as PRT programs. Researchers need to bear in mind that the success of PRT

TABLE 1The basic two-by-two matrix of SWOT analysis developed by the Harvard BusinessSchool

Strengths Weaknesses

Opportunities Achieve opportunities that

match the strengths

Overcome weaknesses to attain op-

portunitiesThreats Use strengths to reduce vulner-

ability to threats

Prevent weaknesses to avoid suscep-

tibility to threats

Source. Adapted from The Use and Misuses of SWOT Analysis and Implications for HRD Professionals,by T. J. Chermack and B. K. Kasshanna, 2007, Human Resource Development International, 10, p. 387.Copyright 2007 by Routledge, Taylor & Francis Group.


7/49

Thomas et al.

6

programs greatly depends on the depth of analysis within the institutional environ-ment and its influence on PRT. Weaknesses and threats such as the implications ofPRT on top of the lack of standardized and valid PRT instruments can be over-come. Therefore, higher education administrators need to support faculty members

by ensuring open two-way communication between the administration and staff,as part of efforts to improve teaching and learning through high-quality feedback.

Method

Literature Search

An extensive online search of peer reviewed literature from numerous data-bases such as Taylor and Francis Online, Elsevier, EBSCOhost, EducationResources Information Center (ERIC), Emerald, J-STOR, SpringerLink, SAGEJournals Online, ProQuest, MetaPress, and Wiley Online Library were conducted

until July 2012. The search was carried out using keywords such aspeer review ofteaching, teaching evaluation, and SWOT.

In this review article, the termpeer review of teachingis used to differentiatestudies on peer review using observation and evaluation techniques from peerreview studies associated with student learning and journal publications. Thisreview article defines PRT in line with Kinchin (2005), who described it as anintentional observation process in which a university faculty member attends acoworkers teaching session with the aim of providing feedback by being a criticalfriend. The PRT concept and process in this review article is adopted from the peerreview model by Gosling (2002). The model suggests that peer observation is

necessary as a prelude to discussion about teaching through shared experiences.The PRT provides opportunities for faculty members to mutually reflect and self-reflect. As a result, the outcome of PRT is the complete analysis of teaching meth-ods with a constructive feedback on teaching performances and learning materialsneeded, which will be communicated after observations are made for the mutualbenefit of the reviewee and reviewer. Gosling emphasized the advantage of dis-cussing immediate feedback through peer-shared perception, which establishes anequal relationship status between the reviewee and reviewer. The model by Gosling(2002) clearly focuses on formative peer review, which emphasizes academicsprofessional development rather than summative peer review, described by

Kinchin (2005) as audit-like.One of the main problems in the literature search is a lack of recent PRT litera-

ture, even among Western literature. There is also limited research published inthis area in high-impact journals. Most relevant studies on PRT programs wereconducted in the late 1980s to early 1990s (Freiberg, Waxman, & Houston, 1987;Hanson, 1993; Odell & Ferraro, 1992). Recent research by Bingham and Ottewill(2001), Kohut et al. (2007), Bell and Mladenovic (2008), as well as Kell andAnnetts (2009) were identified using specific search terms related to PRT such aspeer observation, peer feedback, and reflective practice.

Inclusion and Exclusion Criteria

The selection criteria for inclusion in this review are (a) featured PRT researchthat emphasized professional development among faculty members (i.e., formative


8/49

Peer Review in Higher Education

7

peer evaluation) and (b) empirical or conceptual studies published from 2000 to2012. Exceptions are made for some earlier studies that included theoretical ration-ale for the implementation of PRT within higher education. Furthermore, peerreview research used in performance appraisal, as part of human resource function

in organizations, was excluded. No limitations were placed on the geographicallocation of the studies.

Systematic Review of Literature Using the SWOT Framework

A qualitative review of the literature search yielded research gaps within PRTin higher education. The review was created based on 12 years of past literature(Figure 1). A total of 65 studies were screened to identify the most relevant litera-ture of PRT in higher education. From the 65 studies, 34 studies had PRT as themain research focus, 8 studies were associated with the SWOT framework, and theremaining 23 studies addressed general peer review issues. Of the 34 peer reviewedstudies, only 27 are reviewed in-depth based on the content-relevance to PRT andthe SWOT framework. Table 2 shows five themes derived from 27 past studies indetermining the feasibility of PRT.

The different types of research approaches and publications were identifiedfrom the 27 studies and are shown in Table 3. The result showed there were sixmixed-methods studies, seven qualitative studies, and two quantitative studies.The remaining 12 studies could not be classified according to type as they con-sisted of reviews or reflective publications. Using the 27 studies again, a content-mapping approach was used to sort and review key information such as the authors,

year of publication, research participants and instruments used, research variables,results, and general conclusion, as shown in Table 4. The review from the content-mapping also indicated that three studies in higher education were in a multidisci-plinary setting, whereas one study compared PRT between campuses in across-cultural setting. Only nine studies used some form of instrument such as thePeer Assistance and Review Form (PARF), Sheffield Peer Review AssessmentTool (SPRAT), mini Peer Assessment Tool (mini-PAT), behavior or observationchecklists, Likert-scale questionnaires, video recordings, and personal narratives.As a result, the content-mapping from past studies also reveals the overall impor-tance of feedback, responsibility, and training to improve teaching quality. The

results from the development of themes are discussed in detail in the next section.

Results by Theme

Benefits of Peer Review in Developing Faculty Members

There are a wide range of positive outcomes for the development of facultymembers as a result of practicing PRT. The benefits concluded from the literatureinclude the confirmation of existing teaching practices and motivation for facultymembers to teach from a different perspective (Hanson, 1993), the developmentof assurance to instruct and learn about teaching, change in educational perspec-tives (Bell & Mladenovic, 2008), the development of collegiality, respect for theapproaches of colleagues (Quinlan & Akerlind, 2000), and integration of tutorsinto the department (Allen, 2002). Formative peer evaluation with feedback hasthe ability to give the faculty and its members the responsibility for self-monitoring,


9/49

8

on content relevance

Development of themes:

Benefits of peer review in developing

faculty members

Barriers to peer review of teaching

Weakness: Lack of published literature on

standardized and validated peer review

instruments

Potential problems to teaching practice

Opportunities: Expansion of peer review;

Prospects for professional development

Content mapping

Key information sorted:

Author

Year of publication

Research participants

Instruments

Research variables

Results

General conclusion

Application of SWOT framework:

Textual narrative synthesis of 27

empirical and conceptual studies

Data extraction

Comprehensive search of database

from 12 years past literature

Application of inclusion

and exclusion criteria

8 studies on SWOT -background, strengths and

critique

34 PRT studies23 studies-general peer reviewissues for the introduction and

problem statement

Identification of 65

studies for literature

review

27 studies selected based

on content relevance

FIGURE 1. Qualitative review of the literature selection process.

autonomy over their work, and to practice self-regulation (Al Qahtani, Kattan, AlHarbi, & Seefeldt, 2011). Faculty members will be able to improve teaching prac-tice by identifying their weakness and correcting it, in addition to identifying theirstrengths and building on it. The value of PRT and supportive feedback was also


10/49

9

supported by Freiberg, Waxman, and Houston (1987). This study found that teach-ers who received feedback from colleagues and supervisors, in addition to attend-ing a 2-hour seminar to discuss with their peers about instructional strategies toimprove their classroom performance benefitted the most from the PRT program.A 4-year longitudinal study on 160 teacher trainees by Odell and Ferraro (1992)further revealed that 96% of the cohort valued the peer feedback and emotionalsupport provided by their mentors and, thus, motivated them to continue teachingafter 4 years.

The positive effects of PRT were also shown in a study by Carroll (1980), whoreviewed 13 studies that used observation of teaching in tutor training. This studyfound that 12 studies, with the exception of Haber in 1973, showed statisticallysignificant positive changes in teaching behavior due to training (Carroll, 1980).Research by Dalgaard (1982) captured the teaching performance of tutors beforeand after training on video. The tutors viewed their videos with an experiencedcolleague, and a questioning technique was used to help them self-evaluate and setobjectives in teaching. This study showed that the training group received signifi-cantly higher final teaching scores from trained raters compared to the controlgroup after considering initial differences in teaching skill. The tutors also high-lighted the usefulness of videotapes in the training session. The use of video has

the advantage of providing irrefutable evidence of teaching improvement andhelps focus feedback on specific behaviors (Brinko, 1993). Nevertheless, the vid-eos could be biased if only a portion of teaching performance is recorded.Moreover, peer feedback will only be effective if videos are reviewed immediately

TABLE 2Types of themes derived from 27 selected peer review studies

No Themes No. of studies

1 Benefits of Peer Review in Developing Faculty Members 52 Barriers to Peer Review of Teaching 43 Weakness: Lack of Published Literature on Standardized and

Validated Peer Review Instruments

4

4 Potential Problems to Teaching Practice 65 Opportunities

Expansion of Peer Review of Teaching 2Prospects for Professional Development 6

TABLE 3Types of research approaches and publications identified from the 27 studies

Research approaches/publications No. of studies

Mixed methods 6Qualitative 7Quantitative 2Reviews or reflection 12


11/49

10

TABLE

4

ContentmappingofpastliteratureonPRT

No.

Themes

Author(s)

Yea

r

Objective

ParticipantsInstruments

Varia

bles

Results

Conclusion

1

Benefitsof

Pe

erReview

in

Develop-

ingFaculty

Members

Carroll

1980

Presentacritical

analysisofem-

piricalresearch

ontraining

outcomes

Teachingas-

sista

nts

Cognitiv

e

outcomes,

attitud

es

Majorityofthe

studies(13)

demonstrated

significantpos-

itivechanges

inbehaviordue

totraining

Trainingoutcome:

En

hancedteach-

ingattitudes,

achievement,

an

dratingsof

instruction

Mosttrainingpro-

gramsprovidea

specializedrange

of

teachingskills

2

Dalgaard

1982

Examinetheef-

fectsofpeer

observation

andtrainingon

behaviorand

teachingquality

Tutors

Videoof

teaching

perfor-

mance

Teaching

attitud

es

Thetraining

groupreceived

higherfinal

teachingscores

Peer

observation

an

dtraininghad

po

sitiveeffects

on

teaching

be

haviorand

im

provementof

teachingquality

Videotapingwas

themostuse-

fulaspectof

training

(continued)


12/49

11

No.

Themes

Author(s)

Yea

r

Objective


Varia

bles

Results

Conclusion

Questioning

techniqueused

helpedinsetting

objectivesand

self-evaluation

3

Brinko

1993

Extrapolatefeed-

backgivingprac-

ticestoimprove

teaching

Literatureon

obtaining

feedbackin

education,

psychology

andorga-

nizational

behavior

Whothe

feedback

sourceand

recipient

Peers,t

heself,

experts(cred-

ible,knowl-

edgeable,well-

intentioned)

andsubordi-

nates

Feed

backpractices

werereviewed

ba

sedintheoreti-

calliterature

Whatthe

inform

ation

given

to

recipient

Concreteand

specificdata,

sandwichneg-

ativefeedback

betweenposi-

tivecomments,

createsmoder-

atecognitive

dissonancefor

change

Thereisaneedto

strengthenfeed-

ba

ckliterature

withempirical

studies

TABLE

4(co

ntinued)

(continued)


13/49

12

No.

T

hemes

Author(s)

Yea

r

Objective

Partic

ipantsInstruments

Varia

bles

Results

Conclusion

Howthe

mode

of

feedback

Varietyof

modes:verbal,

written,s

tatis-

tical,graphical,

behavioral,

structured/un-

structured

Allowances

mustbemade

forindividual

differencesin

feedbackgiver

andrecipients

Moreresearch

neededin

motivationfor

feedback-seeking

behavior

4

Peel

2005

Presentingacon-

ceptualframe-

workforPRT

(technicaldevel-

opment,class-

roomtechniques,

personalgrowth

andchanges)

Learningby

doing

(Kolb,

1984)

Teaching

competency

dependson

perception,

reflectiveabil-

ity,personal

insights,l

it-

eratureengage-

ment,policy

documentation

PRTAtransfor-

matoryinstru-

ment

Meaning

inthe

process

Self-reflection:

essentialto

complementpeer

ob

servationin

preparationfor

change

TABLE

4(continued)

(continued)


14/49

13

No.

Themes

Author(s)

Yea

r

Objective


Varia

bles

Results

Conclusion

5

AlQahtani,

Kattan,

AlHarbi,

and

Seefeldt

2011

Reflectingon

educationalpeer

evaluation

Rationale,

metho

ds,

useso

fpeer

evaluation:

formative

andsu

mma-

tive

Constructive

criticism:

improveweak

areas,amplify

strength

Form

ativepeer

ev

aluation:

he

lpsdevelop

responsibility,

thepowertobe

in

chargeoftheir

ow

nworkand

to

practiceself-

regulation

Multiplere-

sourcescan

beutilized:

observationis

mostcommon

Importantforju-

niormembersas

apartofteaching

im

provement

be

foretenure

an

dpromotion

review

Improvesteaching

thatlinkswith

facultydevelop-

mentprograms

TABLE

4(co

ntinued)

(continued)


15/49

14

No.

Themes

Author(s)

Yea

r

Objective


Varia

bles

Results

Conclusion

6

Barriersto

Pe

erReview

of

Teaching

Atwood,

Taylor,

and

Hutch-

ings

2000

Toidentifythe

rootbarriersto

PRT

Chemistry

facu

lty

mem

bers

from

7U.S.

univ

ersities

Rationalefor

PRT,per-

ceived

bar-

riers,results

andim

pact,

future

direc-

tions

Rationalefor

PRT:

Perceivedbarriers

forPRT:

Encouragesself-

improvement

Fear,uncertainty

in

fairnessof

process,personal

na

tureofteach-

ingstyles

Provides

recognition

ofteaching

Stud

entsneedtime

to

acclimatizeto

ne

wmethodsof

instructionfor

afairteaching

ev

aluation

Alternativeto

bureaucratic

accountability

TABLE

4(continued)

(continued)


16/49

15

No.

Themes

Author(s)

Year

Objective


Variables

Results

Conclusion

7

Hanson

199

3

Developand

implementpeer

observation

scheme

284fa

culty

mem

bersin

Bou

r-

nem

outh

University

Effective

Teachers

Behavior

checklist

Givingfeed-

back,

im-

media

cyof

feedbackand

follow

-upac-

tion,validity

andutilityof

feedback

Concernsabout

validityofnon-

specialistfeed-

back,w

hichis

equallyvalid

andreliableas

subjectspecial-

istfeedback

Unfairnessofpeer

reviewcanbe

ov

ercomebyob-

tainingfeedback

frommultiple

so

urcesbesides

ob

servationre-

co

rdsandexpert

op

inions

Satisfactionwith

feedbackvalid-

ity,comments

wereuseful

8

Lomasand

Nicholls

200

5

Toexaminethe

introduction

ofPRTina

pre-1992U.K.

university

Peerreview

documents,

archived

records,

interview

transcripts,

direct

observation

ofinter-

vention,

participant

observa-

tions,and

institution-

alreports

Faculty

memb

ers

perceptionof

PRT,opposi-

tiontoPRT,

managing

PRT,chang-

ingcu

lture

Non

objectiv-

ityofreviewers,

fearofreview

pr

ocess,critical

feedback,impact

on

facultymem-

be

rsrelation-

sh

ips,unfairness

in

one-session

as

sessments

TABLE

4(co

ntinued)

(continued)


17/49

16

No.

Themes

Author(s)

Yea

r

Objective


Varia

bles

Results

Conclusion

9

Kelland

Annetts

2009

Assessthepercep-

tionsaboutPRT

conceptand

clarifyissues

aboutthereview

process

20faculty

mem

bers

Group

discus-

siondata

transcript

Perceptionof

terms,reflec-

tionabout

existingPRT

process

Newerfaculty

membersper-

ceivePRTas

audit-like

Lack

oftime,

biasedreview,

pu

llingofranks

Seniorfaculty

membersper-

ceivePRTas

beneficialfor

personaland

professional

development

Own

ershipofthe

pe

erreviewpro-

cessencourages

thefacultyto

en

gageinPRT

10

Weakness:

La

ckof

Pu

blished

Literatureon

Standardized

an

dValidated

Pe

erReview

Instrument

Archer,

Norcini,

andDa-

vies

2005

Toinvestigatethe

feasibilityof

SPRATamong

pediatricians-in-

training

122pe

diat-

rics

enior

house

officersand

middle

SPRAT

Goodclinical

care,main-

taininggood

medic

alprac-

tice,teaching

andtraining,

assess

ingand

appraising,

relatio

nship

withp

atients,

andw

orking

withc

ol-

leagues

83%ofdoctors

neededfourrat-

erstoachievea

reliablescore(if

theintentwas

todetermine

thatscoreswere

satisfactory)

SPRATisafea-

sibletoolto:

TABLE

4(continued)

(continued)


18/49

17

No.

Themes

Author(s)

Yea

r

Objective


Varia

bles

Results

Conclusion

Grades(three

tertiary

and

five

seco

nd-

aryU.K.

hosp

itals)

Informhighstake

de

cisions

Prov

idefeedback

to

doctorsper-

sonaldevelop-

mentplans

11

Archer,

Norcini,

South-

gate,

Heard,

and

Davies

2008

Todesign,imple-

ment,andevalu-

atethemini-PAT

toassessclinical

trainees

553fo

unda-

tion

train-

eesfrom12

Deaneries

inEngland,

Wales,and

Northern

Irela

nd

Mini-PAT

Goodclinical

care,main-

taininggood

medic

alprac-

tice,teaching

andtraining,

assess

ingand

appraising,

relatio

nship

withp

atients,

andw

orking

withc

ol-

leagues

Highinteritem

correlations(r

=.9

8)

Mini-PATisavalid

methodtocollate

pe

erfeedbackto

assesstrainees

TABLE

4(co

ntinued)

(continued)


19/49

18

No.

Themes

Author(s)

Yea

r

Objective


Varia

bles

Results

Conclusion

12

Magno

2012

Constructinga

peerreview

rubricapplicable

foruseinhigher

learning

institutions

183teachers

inM

anila,

Philippines

PARF

Planningand

preparation,

classenvi-

ronme

nt,in-

struction,and

profes

sional

responsibilityH

ighreliability

(overallinter-

nalconsistency

=.9

8)

Thre

ehighlightsof

study:

Concordance

validityoftwo

raters(=.4

7,

p

Peer Review Teaching in Higher Education: An Application of the SWOT Framework- Dr Sony, Dr Susan,...

Documents

Transcript of Peer Review Teaching in Higher Education: An Application of the SWOT Framework- Dr Sony, Dr Susan,...