Improving Teacher Education Through Action Research

Improving Teacher Education

through Action Research

Routledge Research in EducationSERIES EDITOR, Editors School

1. Learning Communities in EducationEdited by John Retallick, Barry Cocklin and Kennece Coombe

2. Teachers and the StateInternational PerspectivesMike Bottery and Nigel Wright

3. Education and Psychology in InteractionWorking with Uncertainty in Inter-Connected FieldsBrahm Norwich

4. Education, Social Justice and Inter-Agency WorkingJoined up or Fractured Policy?Sheila Riddell and Lyn Tett

5. Markets for SchoolingAn Economic AnalysisNick Adnett and Peter Davies

6. The Future of Physical EducationBuilding a New PedagogyEdited by Anthony Laker

7. Migration, Education and ChangeEdited by Sigrid Luchtenberg

8. Manufacturing CitizenshipEducation and Nationalism in Europe, South Asia and ChinaEdited by Vronique Bn

9. Spatial Theories of EducationPolicy and Geography MattersEdited by Kalervo N. Gulson and Colin Symes

10. Balancing Dilemmas in Assessment and Learning in Contemporary EducationEdited by Anton Havnes and Liz McDowell

11. Policy Discourses, Gender, and EducationConstructing Womens StatusElizabeth J. Allan

12. Improving Teacher Education through Action ResearchEdited by Ming-Fai Hui and David L. Grossman

Improving Teacher Education

through Action Research

Edited by Ming-Fai Hui

and David L. Grossman

New York London

First published 2008 by Routledge 270 Madison Ave, New York, NY 10016

Simultaneously published in the United Kingdom by Routledge 2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN

Routledge is an imprint of the Taylor & Francis Group, an informa business

2008 Taylor and Francis Group

All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereaf-ter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers.

Trademark Notice: Product or corporate names may be trademarks or registered trade-marks, and are used only for identification and explanation without intent to infringe.

Library of Congress Cataloging in Publication DataImproving teacher education through action research / edited by Ming-Fai Hui and David L. Grossman. p. cm. (Routledge research in education ; 13) Includes bibliographical references and index. ISBN 978-0-415-95629-1 (hardback : alk. paper) 1. Action research in education. 2. TeachersTraining of. I. Hui, Ming-Fai.II. Grossman, David L.

LB1028.24.I47 2008 370.71'1dc22 2007024412

ISBN10: 0-415-95629-3 (hbk) ISBN10: 0-203-89537-1 (ebk)

ISBN13: 978-0-415-95629-1 (hbk) ISBN13: 978-0-203-89537-5 (ebk)

This edition published in the Taylor & Francis e-Library, 2008.

To purchase your own copy of this or any of Taylor & Francis or Routledgescollection of thousands of eBooks please go to www.eBookstore.tandf.co.uk.

ISBN 0-203-89537-1 Master e-book ISBN

Contents

List of Figures viiList of Tables ixForeword xi

PROFESSOR PAUL MORRIS, PRESIDENT,

THE HONG KONG INSTITUTE OF EDUCATION

Acknowledgments xiii

Introduction: Teacher Educators Conducting Action Research in an Era of Accountability 1MING-FAI HUI AND DAVID L. GROSSMAN

SECTION 1 Assessment for Learning

1 Developing Productive Synergies Between Formative and Summative Assessment Processes 9DAVID CARLESS

2 Teaching and Learning in the Context of Mass Lecture 24AMY AH-MAY CHAN-YIP

3 Cultivating Creativity in the Classroom: Assessment Strategies to Improve Teaching and Learning 43MING-FAI HUI

4 Enhancing Effective Learning in a Language Teaching Module 60PAMELA PUI-WAN LEUNG

SECTION 2 Innovative Pedagogy

5 Adopting the Guided Discovery Method to Teach Creative Problem Solving 79MING-FAI HUI

vi Contents

6 Using IT to Foster Cooperative Learning and Peer Assessment 99EUGENIA MEE-WAH NG

SECTION 3 Linking Theory to Practice

7 From Theory to Practice: Curriculum for Autonomous Learning 117RITA SHUK-YIN BERRY

8 Catering Individual Learning Differences Through an E-learning Environment 137KAI-MING LI

9 Becoming an Agent of Change: The Application of Process Writing in a Primary Class 157INA YUEN-MEI SIU

10 Constructing a Professional Knowledge-Base for Teacher Education Through Action Research 168JOHN ELLIOTT

Contributors 185Author Index 189Subject Index 193

List of Figures

Figure 4.1 Impact of the modified module on student teachers motivation to learn the Chinese language 73

Figure 5.1 Comparison of the basic CPS and Pedagogical CPS steps 82

Figure 6.1 Students stimulate peers to discuss critical issues related to their presentation topics 104

Figure 7.1 A curriculum framework for fostering autonomous learning 121

Figure 8.1 The instructional model for teachers learning 140

Figure 8.2 The conceptual framework of using an e-learning environment for individual learning differences 141

Figure 8.3 The interface of SOLETE in this study 143

Figure 8.4 Students-teachers interaction in a discussion forum on mathematical problems in the SOLETE 150

Figure 8.5 Learning resources for the low ability group in the SOLETE 151

List of Tables

Table 2.1 Sequence of Module Content and Activities 26

Table 2.2 Perceived Levels of Understanding 31

Table 3.1 Action Research Objectives, Processes and Modes of Assessment for Learning 48

Table 5.1 Sample of Lesson Plan Prepared Prior to Teaching the CPS Sessions 91

Table 6.1 Excerpts of Students Using the Electronic Discussion Forum for Communication 103

Table 6.2 A Sample of Students Peer Assessment 105

Table 6.3 Summary of the Opinions on Cooperative Learning 106

Table 6.4 Summary of the Opinions on Peer Assessment 107

Table 6.5 The Most Important Learning Activities Perceived by Students 108

Table 6.6 Evaluation on Learning and Teaching of the Module 109

Table 7.1 Mini-Curriculum Adopting the Learner Autonomy ApproachCharities 126

Table 8.1 Percentage of Responses of the Teacher Participants and Their Students to the Questionnaire Items After the School-Based E-Learning Initiative 146

Table 8.2 Percentage of Responses of the Participants to the Questionnaire Items after the School-Based E-Learning Initiative 147

Foreword

The aspiration that teachers should serve as role models for their students is self-evident and oft repeated. For those involved in educating our future teachers, the implementation of that aspiration is vital. If student teachers directly experience worthwhile pedagogic practices in their own education, they are more likely to emulate them in their role as teachers.

This book is a result of a project that involved a group of teacher edu-cators from the HKIEd working together to research their own pedagogic practices. The contributors were drawn from very different subject domains, varying from language teaching to information technology. However, they shared a common goal that was to reflect and review their pedagogy, espe-cially in the light of the evidence on student learning and to identify ways in which it could be improved.

For student teachers to have the opportunity to be taught by staff who promote concepts such as reflective practice and action research is a pow-erful experience that will serve to ensure the emergence of a generation of teachers who operate as extended professionals.

The book also reflects the priority that has been given in the HKIEd to develop forms of research that focus on the improvement of pedagogy and that specifically involve teachers to focus on improving their own practice. This contrasts with the tendency for academics to focus on discrete top-ics that are more readily amenable to publication. However, this volume is a testimony that research that involves the author reflecting on their own practice is not only a personal and private activity.

This book is a reflection of the commitment and academic leadership provided by Professor David Grossman within the Institute over the last 12 years and the success of the advisory professors scheme. This was designed to encourage scholars to visit the HKIEd and help stimulate and support research activities. Professor John Elliott, through his contribution to this project and others, demonstrated how visiting academics can play a signifi-cant role in the process of institutional upgrading. He was instrumental in providing guidance, encouraging collaboration, and ensuring that the proj-ect progressed. The Institute has benefited greatly from his support, enthu-siasm, and collegiality.

xii Foreword

Finally, Dr. Ming-Fai Margaret Hui, who coordinated the work of the teams, created an open and supportive environment that allowed her col-leagues the space to develop their case studies, while at the same time, in a true spirit of collaboration, she produced case studies of her own teaching.

Professor Paul Morris President The Hong Kong Institute of Education

Acknowledgments

Support for the case study research that resulted in this book was generously provided by the former School of Foundations in Education (SFE) of the Hong Kong Institute of Education (HKIEd) from 2001 to 2005 through the Institutes Teaching Development Grant program, which in turn is funded by the Hong Kong University Grants Committee. We are most grateful for the financial support that made this research possible. We would also like to briefly acknowledge some of the many individuals that helped to bring this multiyear and complex project to fruition.

First and foremost, the editors and authors of this volume owe very spe-cial thanks to Professor John Elliott, our project consultant and advisory professor at the Hong Kong Institute of Education, who has dedicated his time and effort throughout the 4 years of our research. He has been our teacher, our mentor, and our guide. Internationally renowned for his role in developing the theory and practice of action research in education, Profes-sor Elliotts ongoing supervision and constructive comments gave us the assurance and momentum to pursue this professional endeavor in action research. The fact that our individual case studies can now be narrated and disseminated to teacher educators in the global community and the educa-tion sector at large is largely due to the facilitation of this effort by John Elliott.

Next we would extend our sincere gratitude to Professor Paul Morris, President of the Hong Kong Institute of Education, who provided the impe-tus for the project, and to Professor Wing On Lee, who helped to initiate the project when he was Dean of School of Foundations in Education at HKIEd. Without their support and encouragement, this project and this publication would not have been possible.

We owe special debts of gratitude to the three research assistants who contributed to the project in different phases: Chan Sau-Yee Maggie, Kam Lok-Sze Iris, and Li Kin-Ling Michelle. Further, we would like to express our gratitude to Associate Professor Barbara Zamorski, who offered her profes-sional guidance to colleagues in the first phase of action research toward Assessment for Learning.

xiv Acknowledgments

In addition, we thank several scholars who gave us critical and yet con-structive comments and suggestions on drafts of various chapters. These committed reviewers were Bob Adamson, Phil Benson, Rita Berry, Yin-Ping Rita Chan, James Flaitz, Ming-Fai Hui, Mo-Ching Magdalena Mok, Wai-Shing Li, and Yiu-Chun Lo.

We would like to acknowledge the Asia-Pacific Journal of Teacher Educa-tion and Development, for granting permission to use portions of articles previously published in a special edition of the Journal in September/Octo-ber 2003.

As editors of this published work, we have been blessed with an extra-ordinary set of faculty researchers who volunteered their time and commit-ment to carry out their action research project. Through their case studies they have successfully illustrated how teaching can be improved through the use of action research.

Finally, this book, in its capacity as a catalyst for further research, should also be dedicated to those who are firm believers that action research can ultimately make a difference in improving the quality of teaching, and par-ticularly the quality of teaching future teachers.

Ming-Fai Hui Associate Professor Department of Curriculum and Instruction The Hong Kong Institute of Education

David L. Grossman Professor and Dean Faculty of Languages, Arts and Sciences The Hong Kong Institute of Education

IntroductionTeacher Educators Conducting Action Research in an Era of Accountability

Ming-Fai Hui and David L. GrossmanHong Kong Institute of Education

In Torrances Incubation Model of Teaching, he emphasized the importance of preparing students to make connections between what they are expected to learn and something meaningful in their lives. He had been bothered by the little impact that the courses in psychology and education had with what happened in the classrooms. He saw the need to motivate teachers to keep them thinking about their experiences and insights obtained in teach-ing (Torrance & Safter, 1990). Most courses offered in teacher education programs have yet to meet these needs. In many instances teacher education programs appear to lack relevance and overlook the importance of aligning learning objectives with assessment. Other major shortcomings identified by researchers include fragmentation of the coursework and a lack of coher-ence between theory and practice (Ferguson, 1993).

Given this context the challenge of improving instruction for teacher edu-cators is daunting. Just as teachers are expected to engage actively in their own teaching and to think critically about their practice, teacher educators should also be expected to take up a role not only as scholars of education but also as inquirers into practice, including their own. The authors in this book are all teacher educators at an institution dedicated to the profes-sional preparation and development of teachers: the Hong Kong Institute of Education (HKIEd). As such, we sought opportunities to apply the scholar-ship of teaching to our own practice. In 2000, with funds from a HKIEd Teaching Development Grant, we brainstormed a way to move this agenda forward in consultation with our advisory professor, John Elliott.

This deliberation led to the development of a multiyear project, the Teach-ing and Learning Action Research Project (hereafter the Project), and the formation of three project teams focusing on different aspects of improving teaching and learning: (a) assessment for learning, (b) innovative pedagogy, and (c) linking theories into practice. The chapters in this book are case studies derived from this Project, and the book is divided into sections based on these three themes. It is hoped that these case studies demonstrate how action research can be conducted to help teacher educators reflect on and improve in their own teaching.

2 Hui and Grossman

ACTION RESEARCH AND ITS MULTIFACETED INTERPRETATIONS

The choice of action research as the methodology for the Project was pur-poseful but at the same time, a potential point of controversy. One depart-ment head famously advised us he didnt believe in action research. Action research is now widely recognized as a mode of inquiry conducted in an informal way by teachers or other educational practitioners in a single set-ting or series of settings. It originates with a strongly felt need in a particular context, usually associated with a dissatisfaction or obstacle, and is carried out to resolve the concern (Charles & Mertler, 2002). Action research for-mulates and interprets research questions rather than testing the hypotheses in a statistical way. It collects data during the research process to measure, observe, analyze, and interview its sample of subjects.

Although some educators believe that action research is relatively easy to carry out and can make classroom experiences more insightful to the teach-ers and more enjoyable for the students, some critics opine that with its lim-ited perspective, it is the least precise of educational research methodologies and the most subject to errors of bias, generalization, reliability, and valid-ity (Best & Kahn, 2003; Charles & Mertler, 2002). From our standpoint, the merits in action research far exceed its drawbacks in the context of a collaborative approach to improving teaching and learning. Whereas many research methodologies are conducted to obtain and analyze original data to help answer questions or test hypotheses, action research utilizes avail-able and convenient subjects and resources to put together a plan of actions to resolve an existing, imminent problem at hand. The consequences of these actions, however, cannot be entirely foreseen in advance, creating surprising delight to the researcher if he or she can tolerate ambiguity.

One of the foremost advocates of action research, John Elliott, calls our attention to the beauty of the term action as described by Hannah Arendts (1958) account in the Human Condition. According to Arendt, there were three basic modes of human activity. Labor is activity dictated by what is required to sustain life; work is activity involved in the creation of endur-ing objects or artifacts for use rather than consumption to satisfy basic needs; and action is the third mode of human activity that transcends freedom and initiates change in a social situation to bring about something new. We should interpret this as unconstrained freedom and that the indi-viduals humane self is located in its actions. In Arendts terms, we sought to promote actions that would initiate change to bring about improvements in teaching and learning in an institute dedicated to teacher education.

In any case, we emphasize that the creative aspect of action research and its development lies in the researchers ability to organize, implement, refine, and evaluate the new try out product, plan, or procedure. This is further explained by Elliott (2007) that in action, the agent or researcher takes into account the plural voices of others as well as reveals his or her

Introduction 3

own distinctive view of the situation. Hence, a value is built into the process of action itself (Elliott, 2007). Those who agree that action research is rela-tively easy to carry out have most probably neglected the demanding tasks of a researcher to re-arrange, elaborate, be flexible, operate with both diver-gent and convergent thinking skills, and to evaluate problems in specific situations. Only through using such creative thinking abilities in the process of inquiry can we conduct sound action research.

Indeed, our group of researchers found it inevitable to avoid the attitu-dinal and affective aspects in the process of conducting their actions. The actions reported in this book are proven creative when the teacher educators were able: to be playful with ideas, willing to take risks, to tolerate uncer-tainty and complexity, and to refrain from overly subjective judgments. A good researcher must possess a curious attitude that motivates him or her to look deeper into situations, identify hidden causes, and discover astonishing solutions. The more we experience the process of actions, the more we are nurtured to be creative.

PREPARING TEACHERS IN THE ERA OF ACCOUNTABILITY

We feel that action research offers a creative approach to dealing with the new emphases on teacher accountability. Challenges for teachers today are enormous. Most believe that the ultimate goal in running a school program, as held by most teachers, school administrators, policy makers, parents, curriculum developers, and teacher educators, is the attainment of student success. However, definitions of student success are fraught with miscon-ception and controversy. Todays teachers are often held accountable and have to work with specific standards and benchmarks, but as a result may find themselves pressed into exploring an adventureland in which their predecessors have heretofore not tread. They are being driven into this land, whether it is high ground or marshy lowland as Schn (1983) described it, without a well-informed guidebook, map, or manual. We argue that action research offers as one effective tool that can help teachers, including teacher educators, build their own knowledge base for effective practice.

A prime advantage of gathering data for accountability using action research is that it provides an informed basis for improving instruction in a manner that is self-regulated and self-rewarded. It may be a bit sarcastic to suggest that too often those who hold important positions in the education sector and are most interested in accountability usually are quite remote from classroom practice. Unfortunately, there is a tendency that the further removed someone is from the scene of action in teaching and learning, the more he or she is tempted to use external rewards or punishments to bring about desired teaching performance and learning outcomes. This we find inconsistent with the notion of teaching as a profession, and in the end probably self-defeating. Action research, on the other hand, relies almost

4 Hui and Grossman

entirely on a teachers intrinsic motivation for change, and thus, we argue, more closely fits the model of a teaching profession.

In this era of accountability, teachers need to visualize learning to be transformational in lifes experiences rather than merely the accumulation of disparate knowledge and skills. Learning means re-examination and recon-struction of new knowledge to be internalized from what already exists in the learner, guiding him or her to the profound appreciation of human accomplishment. As supported by many theories of learning, the nature of learning now is interpreted as more than the accumulation and storage of knowledge; it is adaptive and interactive, constantly changing and evolv-ing, transmitting, and transformational to maximize ones potential. Action research is consistent with this approach to learning. Teacher educators, as this book amply demonstrates, can learn much about teaching and learning through action research in their own classrooms, and in so doing provide a model for their students as future teachers.

THE TEACHING AND LEARNING ACTION RESEARCH PROJECT EXPERIENCE

Having identified action research at its basic methodological approach, the Project team adopted a set of procedures and principles consistent with the best practices in action research. These included the recruitment of a volun-tary action research group of academic teaching staff at the HKIEd; selec-tion of a single focus theme within the broad area of improving teaching and learning; commitment from each group member to attend a weekly meeting of 90 minutes focusing on issues related to the design of their project, its implementation, and their findings; and a commitment to produce a public case study report on their individual action research project.

Over a 5-year period, three teams focused on a single key question related to how their teaching within teacher preparation programs could be posi-tively related to:

1. Adopting strategies to provide assessment for learning in the context of teaching and learning,

2. Adopting innovative pedagogy to improve teaching and learning, and 3. Linking theory to practice in their own teaching.

The chapters in this book are divided into sections based on the above three themes. Each chapter describes a process in which a teacher educator tested an approach to improving instruction in his/her own context to find out what worked well and what did not. Data was collected in the form of case studies using a variety of methods including direct observation, anec-dotal records, note taking, student feedback, and occasionally by the use of statistical measures. In the concluding chapter, John Elliott provides a

Introduction 5

useful and insightful overview and analysis of the chapters individually and collectively.

SHARING AND APPLYING RESEARCH ON PEDAGOGY IN TEACHER EDUCATION

Although critics claim that the findings of action research should be lim-ited to the setting in which the research was done and not be applied else-where, via this book we do intend to extend the use of our findings and their implications to other education professionals with common concerns (Hui, 2003). We claim not only local relevance, but also attempt to claim wider applicability for our findings. We encourage the readers of this volume to selectively adopt our research questions and procedures and adapt them for use in their own educational situations. It is crucial that teacher educators maintain an intellectual discourse with their peers to exchange their insights and reflections about teaching. It was on this challenge that this modeling of pedagogy through action research was initiated and developed.

We believe that success can be contagious across all kinds of borders. When teacher educators within the same institution or even from other soci-eties have an opportunity to see how Hong Kong teacher educators have benefited from action research in their own classrooms, it is hoped that they will be motivated to try a few ideas in their own classroom. Once a comfort zone of formulating a research objective and agenda is established, more dramatic steps can be taken to facilitate significant and positive changes in pedagogy.

In fact, traditional research on teaching and learning has always had a tendency to credit contributions made by scholars and academics other than teachers as the experts in teaching and learning. Teacher educators, who believe in both theoretical and practical approaches to intellectual inquiry, should aim to conduct action research that will eventually build a knowl-edge base about what works for them and what does not. This should not be a one-shot effort that is completed and then published, but rather as con-tinual and cascading attempts of academic pursuit. The findings and con-clusions from one project sets new beginnings for another. That is how the creative tradition of action research can be carried on to the next generation of teacher educators and educational researchers.

REFERENCES

Arendt, H. (1958). The human condition. Chicago: University of Chicago Press.Best, J. W., & Kahn, J. V. (2003). Research in education (9th ed.). Boston: Allyn &

Bacon.Charles, C., &Mertler, C. A. (2002). Introduction to educational research (4th ed.).

Boston: Allyn & Bacon.

6 Hui and Grossman

Elliott, J. (2007). Rethinking pedagogy as the aesthetic ordering of learning experi-ences. In J. Elliott (Ed.), Reflecting where the action is: The selected works of John Elliott. New York: Routledge.

Ferguson, P. (1993, April). Novice-assisted research as a means of promoting inquiry into the relationship between instructional theory and practice. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, Georgia.

Hui, M. F. (Ed.). (2003). Teaching and learning action research [Special issue] Asia-Pacific Journal of Teacher Education and Development, 6 (1).

Schn, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books.

Torrance, E. P., & Safter, T. (1990). Incubation model of teaching: Getting beyond the aha. Buffalo, NY: Bearly

Section 1

Assessment for Learning

1 Developing Productive Synergies Between Formative and Summative Assessment Processes

David CarlessFaculty of Education, University of Hong Kong

INTRODUCTION

The powerful impact of assessment on student learning has long been rec-ognized. One of the earliest formulations of this notion runs as follows: the quickest way to change student learning is to change the assessment system (Elton & Laurillard, 1979, p. 100). Assessment affects the con-tent, mode, and orientation of students learning (Gibbs, 2006). Students, being essentially pragmatic, have a primary focus on achieving a high mark rather than demonstrating deep understanding of material (Greer, 2001). Following from this, summative assessment practices have tended to dominate the assessment process, frequently to the detriment of formative aspects (Knight & Yorke, 2003). It is increasingly being recognized that these formative dimensions or what is commonly referred to as assessment for learning, have a powerful potential for enhancing student learning, yet are often not exploited fully (Black & Wiliam, 1998; Gibbs & Simpson, 2004).

At the outset, I wish to make some brief terminological clarifications. I use formative assessment and assessment for learning interchangeably and place them within a constructivist teaching and learning approach that sees students as taking an active part in the development of their own learning (cf. Shepard, 2000). I see formative assessment as an essential component of good teaching and as much about learning as about assessment. For the pur-pose of this chapter, I use formative assessment to denote dialogues between a tutor and students or between students and students that serve as feedback with potential to stimulate enhanced student learning. Summative assess-ment is seen as a judgment leading to the award of a mark or grade, usually as part of a certification process that leads to a qualification or academic award.

The improvement of assessment practices (both formative and summa-tive) has the potential to make a major contribution to the enhancement of student learning outcomes. Some form of marriage between summative

10 Carless

assessment and formative assessment might satisfy the accountability aspects of the former, whilst reinforcing the learning aspects of the lat-ter. A suggested guiding principle is that summative assessment should be designed with learning foremost, whilst minimizing threats to the reliability of measurements.

The development of productive synergies between formative and summa-tive functions of assessment has a number of potential advantages. In terms of motivation, students are most likely to buy into the formative aspects if they help them to do better in a summative assignment. With respect to peer feedback processes (Liu & Carless, 2006) and self-assessment (Boud, 1995, 2000), these core elements of the formative process of learning also have potential to support higher achievement in summative assessments (McDon-ald & Boud, 2003). Building productive relationships between the formative and summative functions of assessment is, however, not without its chal-lenges. For example, Pellegrino, Chudowsky, and Glaser (2001) warned that the more purposes a single assessment seeks to serve, the greater the possi-bility of each purpose being compromised. A way forward perhaps is not to regard formative and summative assessment as separate entities, but to see them as informing each other (Taras, 2005).

The focus of this chapter is on action research seeking to develop pro-ductive synergies between formative and summative assessment within a module in a teacher education institute. Teacher education is a particu-larly important site for close scrutiny of assessment practices in view of the future role of trainee teachers in assessment of and feedback to their students. The specific problem that the action research sought to address was ineffectiveness of feedback on summative assignments. The aims of the chapter are threefold: to illustrate how formative assessment can be inte-grated within a summative assignment, to discuss how students responded to the assessment processes for the module, and to draw out some wider implications relevant to the interface between assessment and learning in teacher education.

The specific teaching context was a module for English major students on the topic of assessment. The module comprised 30 teaching hours with 15 weekly 2-hour sessions. The class size was somewhat large with 50 pre-service students enrolled, native speakers of Cantonese mainly in their early 20s. I was the sole instructor teaching the module. The objectives of the module can be summarized for students to demonstrate: an understanding of approaches to formal and informal, formative and summative assess-ment so as to promote pupil learning; and an ability to construct and jus-tify appropriate types of assessment tools for use in primary English as a second language (ESL) classrooms. The fact that the module itself is on the topic of assessment is significant because it facilitates integration between the concepts being taught and the way that the module assessment is car-ried out.

Developing Productive Synergies 11

ASSIGNMENT RATIONALE AND OPERATIONALIZATION

This section firstly discusses the rationale for the module assignment in terms of how it tackles identified problems in feedback processes. Secondly, it describes how this rationale was operationalized.

Rationale

My initial thinking about summative assignments was that they are some-times not well-integrated with the teaching and learning during the module and that as feedback on assignments is summative in nature, there is little opportunity for students to make improvements. A key problem is the failure to include iterative cycles of feedback and revision that normally character-ize academic writing (Taras, 2006). In addition, once a mark is awarded and summative feedback is received, it is difficult for students to use feedback so that they can do better on a subsequent task (Stefani, 1998). James (2000), for example, outlined dissatisfactions from university students concerning both the timeliness and usefulness of written feedback on their assignments. A repercussion of these kinds of scenarios is that the potential learning ben-efits of feedback on summative assignments are not being fully exploited (Carless, 2006a).

In terms of a potential solution to the problem of the relative ineffective-ness of feedback on summative assignments, the literature provided some inspiration. A study by Butler (1988), afforded a high profile by the Black and Wiliam (1998) meta-analysis, compared three forms of feedback treatment: grade only, comment and grade, comments only. This study of 11-yearold Israeli students found that comments only led to greater learning gains than comments with grade, due to the ego involving properties of the award of a grade. Even with mature students, James (2000) noted that marks had a major impact on students self-perceptions and self-esteem. Following from this, I decided to arrange the summative assignment in such a way that there would be emphasis on formative assessment through instructor feedback and student peer interactions before a grade was awarded.

A group assignment was chosen so as to encourage collaboration and peer learning amongst students. The issue of awarding grades in group assignments has long been recognized as a thorny area (Race, 2001), in par-ticular the phenomenon of social loafing or the presence of free riders may enable students to be rewarded for work they have not done (Heath-field, 1999). There are various procedures for awarding grades for group assessments (see Falchikov, 2005, for a review). For the current module, I required each group of three (small groups tend to reduce the likelihood of social loafing) to indicate clearly the specific contributions of members to the overall assignment. I awarded two grades, one for the individual contri-bution (40% weighting) and one for the group as a whole (60% weighting).

12 Carless

I believed that this procedure supported the award of a reliable assessment of student work, whilst retaining the advantages of group collaboration.

The assignment (see Appendix 1.1) was for students to develop a portfolio of assessment tools, write a rationale, and relate the tools to primary school pupil learning, in line with the module theme of assessment for learning. During the process of completing the assignment, peer and self- assessment within their groups of three were encouraged.

Operationalization

The above rationale was operationalized through the following steps. Firstly, the details and criteria for the assignment were distributed in the first session of the module and a brief introduction was provided so as to make expecta-tions and processes explicit. Secondly, students were given some class time in the sixth class to form groups and begin preparation of the assignment. Thirdly, in Week 8, students submitted via email an outline for their assign-ment and received prompt, albeit brief, written feedback to support ongo-ing planning. Fourthly, in Weeks 9 and 10, the groups met with me for a tutorial of about 15 to 20 minutes to receive further feedback on their draft assignment. Students who submitted a full or almost complete draft were best placed to gain from this tutorial. Fifthly, students handed in the com-pleted final version of their assignment in Week 12, three weeks prior to the completion of the module.

The final stage of this process was that after my provisional marking of the assignment, students attended in Weeks 13 and 14 a mini-viva, that is, an oral defence of their work. In this case, the mini-viva was not designed to award marks or distinguish between borderline grades, but to generate dialogue and provide feedback. My idea for a mini-viva was prompted by the desire to provide an opportunity for timely oral feedback before a mark was awarded (cf. Butler, 1988). Brown and Knight (1995) provided support for this kind of process as follows, Feedback, ideally, ought to involve the interplay of the tutors understanding and the learners . . . Dialogue is there-fore not simply desirable but arguably essential (p. 112).

The mini-viva, which replaced the normal weekly class, took the form of a tutorial for each group lasting 15 to 20 minutes. The content of the mini-viva differed slightly from group to group according to their assign-ment content and student characteristics, but essentially focused on: Tutor requests for clarification or elaboration on some of the points made; ques-tions and feedback relating to the most challenging part of the assignment, the relationship between assessment items and pupil learning; and feedback on matters related to English language usage, because the students were second language learners training to be teachers of English.

The focus of the mini-viva was on discussion and feedback; student performance was not assessed. I believe it was motivating for students,


however, because the mark was not yet awarded, and they were still eager to demonstrate their knowledge, understanding, and diligence. For one or two students who I perceived as possible free riders, I was also able to use the mini-viva to satisfy myself that they had made an adequate contribution to the group assignment.

After the mini-viva tutorial, as the final stage to the assignment process, I briefly reexamined the assignment scripts and finalized the grades. Sample scripts were then moderated by a colleague, as per standard departmental quality assurance procedures. The marked assignments were then returned to students.

DATA COLLECTION

Getting students to think about teaching, learning, and assessment processes is valuable in any context, but particularly so when participants are future entrants to the teaching profession. I collected data from students through three main data collection methods. Firstly, five-minute papers, adapted from Angelo and Cross (1993), were used to collect written feedback from students from the whole class (see sample in Appendix 1.2). Five-minute papers, completed by students at the end of a class or series of classes, are user-friendly ways to elicit views from all participants about instructional issues. To complete the feedback loop, I extracted selected student responses and distributed them to students in a subsequent session, explaining how student feedback would be acted on. An end of module 5-minute paper was also distributed to evaluate the assessment processes discussed earlier.

The second means of data collection supplemented the above by means of a focus group of three student volunteers. Two discussions were held lasting about 30 to 40 minutes. The first focus group meeting was at the mid-point of the module to review progress of teaching and learning, and elicit suggestions for improvement. The second discussion was at the end of the module and sought particularly to evaluate the assessment processes outlined in the previous section. Both meetings were taped, transcribed, and analyzed with respect to the aims of the action research.

Thirdly, formal module evaluation and teaching evaluation data were also collected at the end of the module using standardized institute forms. These included numerical data and open-ended questions on the most valu-able aspects of the module and areas that could be improved.

Whilst not being directly a data collection method, discussions in our reg-ular action research team meetings also helped to stimulate further insights into the teaching, learning, and assessment processes. The session in which I presented provisional findings from my action research was particularly influential in enabling me to collect views from other team members and refine my own thinking.

14 Carless

EVALUATION OF ASSIGNMENT PROCESSES

This section explores in turn students responses to the feedback processes on the module, their perceptions of working in groups, and my comments on the quality of their assignments.

Feedback Processes

Most students expressed positive views about the feedback processes used for the module assignment. In the end of module 5-minute paper and in the formal module evaluation, they stated appreciation at receiving prompt feedback on the progress of their assignment and a number of them high-lighted the pre- and post-assignment tutorials as being positive. Some stu-dent comments on the pre-assignment tutorials are quoted below:

I think the assignment-related tutorial was very useful as we could con-sult him [the instructor] when we had problems.

We can get useful feedback to improve our work.

We could know whether we are on the right track for the assignment and also ask some questions.

In the tutorial before the assignment, we can have a chance to get feed-back on our own particular problems.

In the tutorial, we can ask you the questions that puzzle us, so we have a clearer picture on how to do the assignment.

These comments provide some indication that students perceived that they were able to act on the feedback in the tutorials to improve their assignment.

For the mini-viva post-assignment tutorials, comments were also gener-ally positive:

In the post-assignment tutorial, both parties can clarify the points that they do not understand.

We get feedback quickly rather than having to wait a long time for the marking. Its quite good because we are trying to explain what we were doing in the assignment. Sometimes, we feel that professors didnt know what we had done and what our difficulties are. When we have discus-sion with you, we can talk about it and we can get feedback.

For some points when you didnt understand why we wrote in a certain way, then we can explain it more clearly to you.


Some of the general points that emerge from these statements are that feed-back should: be specific to student needs and difficulties; include negotia-tion of meaning; be provided promptly when it is most relevant to students; and include helping the teacher educator to understand students thought processes.

There were also two mixed or negative opinions with respect to the mini-viva:

It is quite good; the idea is good. Its quite creative but the time is too limited for discussion; it needs longer time.

Its quite frustrating not to know the mark straight after the tutorial. It seems a bit cruel to leave us guessing and not knowing.

The last point resonates with a study (Smith & Gorard, 2005) that also attempted to implement the ideas from Butler (1988). In that case, students (and teachers) in a secondary school context did not understand the ratio-nale for feedback without marks and students felt uncomfortable with the withholding of grades. The association of assessment with marks is a deep-ly-ingrained one that represents a threat to some formative assessment strat-egies, particularly if their rationale is not fully explained or understood.

Working in Groups

The majority of students expressed a favorable orientation towards working in groups on their assignment. Some illustrative comments are as follows:

I enjoyed working with my friends and the peer evaluation played a remarkable part in my experience.

I was happy to work with my partners. We outlined the framework of the portfolio together, divided jobs and then had meetings to discuss content and organization. In that process, we inspired each other, raised problems and then shared views and found ways to solve the problems. I think such kind of learning is called collaborative learning.

A few students were not so enthusiastic about doing a group assignment, one commenting:

We have to spend a lot of time on discussing and it is very difficult as there are so many assignments to do at the end of the semester.

In the focus group, one student also voiced a frustration about resolving conflicts between group members, in that she claimed to be on the right track but was outvoted by her two partners who were slightly off-track.

16 Carless

The focus group members also observed that in their view a group assign-ment usually generates a higher workload than an individual assignment because of the meetings and discussions that are required. With respect to learning, they believed that they generally achieve deeper understanding from a group assignment than an individual assignment: reasons suggested were that the process of debate with team members compels them to clarify their thinking and that peer or cooperative learning takes place (see also, Appendix 1.3).

In sum, collaborating through group work is challenging, but an essential skill that is much required in the workplace, including the school context. Group assignments, such as the one described here, tend to focus on out-comes or the final product. It may also be worthwhile to focus on the pro-cess of group work, which can be assessed, for example, through peer and self-assessment (Bryan, 2006).

Learning Outcomes

An important aspect of any innovative assessment process is the extent to which it leads to superior student learning outcomes. In this case, the assign-ments themselves provide some indication of student performance on the module, at least in terms of final product rather than process. Although there are obvious limitations to what can be claimed from a single set of assignments without systematic comparison with previous cohorts, a few comments follow. Overall, I was pleased with the quality of work produced by the students and tentatively formed the impression that as a whole, the body of work was superior to that produced by other similar cohorts when I had not carried out such a systematic feedback process. Within a criterion-referenced system, the marks were mainly clustered in the good or sat-isfactory grades, with three students attaining marginal passes. From a cohort of 50, with a wide range of abilities and motivation, that there were no failure cases was considered a positive outcome.

Student work can provide some tentative evidence of learning outcomes and to provide a brief flavor of student performance two extracts from assignments are included in the appendices. The first (Appendix 1.3) illus-trates the kind of reflective processes that students carried out during their assignments. A number of groups included this kind of reflection on the development of their assignment. Several groups also included some kind of self-assessment tool (Appendix 1.4). As indicated earlier, self-assessment is both a core formative assessment process and one that can aid performance in summative assessments. During the entire process of the assignment, stu-dents were encouraged to evaluate their performance with reference to the stated criteria. The self-assessment tool in Appendix 1.4 is chosen because the students acknowledged that they were only partially able to meet some aspects of the criteria, rather than claiming, as some others did, high achieve-ment in all of their selected categories.


The balance of evidence from the student data indicate that they gener-ally perceived that the formative assessment processes played a useful role in supporting the development of their learning. In terms of further devel-opment, desired learning outcomes could have been stated more explicitly and formed part of the development of constructive alignment (Biggs, 1999) between outcomes, content, and assessment methods. Outcomes based approaches, a particular current emphasis in higher education in Hong Kong, have potential for positive impacts through a focus on the outputs rather than inputs of teaching. When learning outcomes are worthwhile and clearly stated, and assessment tasks require students to work productively toward these outcomes, then students are being primed for deep learning experiences (Carless, Joughin, & Liu, 2006).

REFLECTIONS ON IMPLEMENTING MINI-VIVAS

The main innovative aspect of the action research was the mini-viva concept that sought to minimize some of the main student concerns about feedback highlighted at the outset of the paper. Drawing on and extending Carless (2002), some of the advantages and disadvantages of the mini-viva are now discussed. The processes of the assignment and mini-viva were evaluated positively by the students. In particular, the mini-viva illustrated how feed-back can be prompt (within a week of submission) and interactive (through verbal dialogue with students). Discussions with the action research team members and my own subsequent reflections identified, however, a number of further issues relevant to the implementation of mini-vivas.

The first issue relates to workload. Procedures that require staff to work harder to provide more and quicker feedback to students are likely to be evaluated favorably by students. The processes reported in this chapter did generate some additional workload for me. There were, however, some fea-tures that kept the workload manageable. The group assignment, justified on pedagogic grounds in terms of collaboration and cooperative learning, was also relatively efficient in terms of reduced marking. Group tutorials were instead of, not in addition to, a regular taught class. Written annota-tions on the assignments were less detailed than normal as the main points had already been discussed orally during the mini-viva. Overall, the addi-tional workload was not excessive, but the arrangements and timing of the assessment process did require careful planning and prompt responses. These feedback processes seem most practical with either a small cohort of students doing individual assignments or a larger one doing group assign-ments, as otherwise carrying out the tutorials and mini-vivas would tend to become too time-consuming. A drawback of such feedback processes, not fully addressed in the current study, is that they can encourage student dependency on the tutor rather than promoting greater student self-mon-itoring of their own work. In the teacher education context, there is also

18 Carless

the further risk that cultures of students dependent on their teachers can become duplicated or reinforced in schools.

Secondly, the mini-viva requires the willingness of students to cooper-ate in an innovative assessment method. There is some discussion in the literature (e.g., McDowell & Sambell, 1999) that students may resist inno-vative assessment. In this specific case this was not a major challenge. The concepts of assessment for learning, feedback, peer, and self-assessment had been emphasized throughout the module. An additional factor was that hav-ing taught the students previously during their program, a relationship of mutual trust had been developed. My own standpoint is that if students understand the rationale for an assessment process and can see its potential learning benefits, they are generally willing to cooperate. There is also the possibility that teacher education students, through their exposure to edu-cational theory and practice, might be more open to alternative assessments than students from other disciplines.

Overall, I believe that the mini-viva is a useful tool to add or adapt to ones repertoire of assessment techniques. There are a number of variations that are worth considering (Carless et al., 2006). For example, a mini-viva could be assessed with a proportion of marks allocated to it. It could also be done on an individual basis if time and class size permits. A viva could be integrated with other assessment methods, especially those that may be difficult or time-consuming to mark or assess, such as electronic portfolios. In such a scenario, the e-portfolio could be a required task and the viva or mini-viva based on the e-portfolio would form the graded component.

To conclude this section, it is also worthwhile mentioning constraints that discouraged me from using the group assignment and mini-viva concept in the subsequent running of the module. In view of the need for English lan-guage students to demonstrate a benchmark level of language proficiency (Glenwright, 2002), individual assignments began to predominate, so that writing ability could be more reliably assessed. This is a stark reminder of tensions between learning aspects and measurement aspects of assessment. The group assignment may be a more productive learning experience, but can be undermined if stakeholders have doubts about its effectiveness in reliably assessing individual writing abilities. Of relevance are the notions of trust, distrust, and accountability (Carless, 2006b) and how protection from external criticism may impact on the implementation of assessment (Eccle-stone & Swann, 1999). This is illustrative of tensions between competing demands of assessment also alluded to below.

CONCLUSIONS AND IMPLICATIONS

The chapter now concludes by outlining some wider implications for assess-ment processes. Firstly, the study reinforces recent attempts to reinvigorate


feedback processes (Gibbs, 2006; Gibbs & Simpson, 2004). Key principles are that feedback should be provided quickly enough to be useful to stu-dents and that feedback needs to be acted on by students to improve their work or their learning. The pre-assignment feedback through small group tutorials caters for both of these conditions; the mini-viva meets the con-dition of timeliness more effectively than the issue of student action. A proposition following from this is that instructors might provide more feedback on work in progress and less feedback on completed summative assignments. Feedback on the latter is most useful if it highlights generic issues that may be transferable to other future assignments (Knight & Yorke, 2003). A further strategy to support feedback on summative tasks is designing a coherent series of assignments that enable students to use feedback from an earlier task on a later one. The study also reinforces the desirability of feedback being verbal and interactive rather than solely through a written mode, a principle that seems better established with respect to schooling (e.g., Gipps, McCallum, & Hargreaves, 2000) than higher education. An unanswered overarching question is the extent to which teacher education participants experience good feedback practices during their programs and internalize them for future use in their teaching careers.

A second theme is that teacher educators need to model progressive assessment practices. Innovative assessment methods can be particularly valuable in teacher education contexts, especially for a module such as this one on the topic of assessment. In this case, academic content and assign-ment processes were integrated by introducing concepts in assessment (the content) and using those same concepts to inform assessment strategies (the processes). If we want trainee teachers to do more than recycle traditional assessment practices, then we need to demonstrate to them both in theory and in practice the potential of alternative assessments. In the current study, there was evidence (see, e.g., Appendix 1.3) that students were changing their orientations toward assessment. The seeds of school assessment change are formed in the teacher education classroom.

Finally, one of the complexities and challenges of assessment is that it concerns many things at once (Ramsden, 2003). In this chapter, a num-ber of potentially conflicting dimensions have been salient: assessment for learning (formative) and assessment of learning (summative); tutor feedback versus student self-monitoring; group assignments for peer learning and individual grades for personal achievement; and (peripherally) assessment for productive learning versus assessment for accountability. Assessment is beset with many tensions and competing demands, which is perhaps one of the main reasons why it is so difficult to manage successfully at both indi-vidual and institution-wide levels. How these tensions might be reconciled or ameliorated in teacher education contexts is in urgent need of further exploration.

20 Carless

APPENDIX 1.1

Assignment Description

Task

Design a portfolio of informal and formal assessment tools for the primary English as a second language (ESL) classroom. Explain the rationale for the assessment tools/items. Indicate the relationship between the assessments and pupil learning.

Organization and Procedures

Work in a group of three. Indicate clearly on the assignment, the responsi-bilities of each member of the group, as you will receive one individual mark (40% weighting) and one group mark (60% weighting). After submission of the assignment, I will then mark the portfolio and award a provisional grade. I will invite the group for a tutorial and ask questions to clarify any queries about the assignment and the respective individual student contribu-tions. Final grades will be awarded after these tutorials.

APPENDIX 1.2

Sample 5-Minute Paper Used in the Module

5-minute paper (review of Weeks 1 to 5 of the module)

Complete the following questions and hand to your instructor at the end of todays class. The information will not be used for any assessment purpose.

1. What are the most important attitudes, understandings or knowledge that you have gained about assessment so far during the module?

2. According to your understanding, what are the main principles of for-mative assessment/assessment for learning?

3. Do you have any questions related to assessment which need answer-ing or clarification?

4. Do you have any suggestions as to how the instructor could teach the topic of assessment in a better, more interesting or more effective way?

5. Are there any other assessment-related topics or classroom activi-ties that you would particularly like to cover in the remainder of the module?

6. Is there any other feedback that you would like to give to the instructor?


APPENDIX 1.3

Sample Student Reflection Submitted As Part of the Assignment

Many problems came across when my group members and I were doing this assignment. For example, are assessment tasks/activities well-designed? How do I know what difficulties students have when doing the tasks? Also, we needed to design appropriate assessment tools that are beneficial to learning and teaching. It is true that a diversity of assessment modes, according to the purpose and process of learning and not just academic outcomes, should assess students. Above all, teachers should decide what aspects they want to assess students.

My classmates opinions gave me incentives to think more. As they had better in-depth thinking, it broadened my views. Though sometimes we had different opinions, it was good to share and brainstorm ideas together. With classmates opinions and support, I can modify and improve what have been done continuously. Also, I understood that assessment facilitates reflection, not only for students, but also for teachers. Teachers should adjust and improve teaching materials and activities in light of the results of assessment.

I remember that I was quite afraid of being assessed at younger age, as I could not get high marks. Indeed, it is wrong. Assessment is an aid of learn-ing. Hence, teachers should share with students the goals of learning and let them know and recognize the standards they are aiming for. More face-to-face, written and oral feedback on work should be reinforced as they can increase the interaction between teachers and students.

Assessment should be instructive challenging, engaging, and even enjoy-able. Being teachers, we strive for making assessment feed back into the learn-ing and teaching process. Always bear in mind that every student can improve and all the time is assessment time, if we can make use of every opportunity.

APPENDIX 1.4

Self-Assessment Tool Submitted as Part of the Assignment Fully Able Partly Able Not Yet Able

Portfolio is well-organized Clear rationale Language proficiency Creativity Well-designed tasks Able to motivate students to learn Can improve the assessment in local schools Can truly reflect students ability Good assessment (valid, reliable, practical and authentic) Support students before or after the assessment

22 Carless

REFERENCES

Angelo, T., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers (2nd ed.). San Francisco: Jossey Bass.

Biggs, J. (1999). Teaching for quality learning at university. London: Society for Research Into Higher Education & Open University Press.

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education 5(1), 774.

Boud, D. (1995). Enhancing learning through self-assessment. London: Kogan Page.

Boud, D. (2000). Sustainable assessment: Rethinking assessment for the learning society. Studies in Continuing Education 22, 151167.

Brown, S., & Knight, P. (1995). Assessing learners in higher education. London: Kogan Page.

Bryan, C. (2006). Developing group learning through assessment. In C. Bryan & K. Clegg (Eds.), Innovative assessment in higher education (pp. 150157). London: Routledge.

Butler, R. (1988). Enhancing and undermining intrinsic motivation; the effects of task-involving and ego-involving evaluation on interest and performance. British Journal of Educational Psychology, 58, 114.

Carless, D. (2002). The mini-viva as a tool to enhance assessment for learning. Assessment and Evaluation in Higher Education, 27, 353363.

Carless, D. (2006a). Differing perceptions in the feedback process. Studies in Higher Education, 31, 219233.

Carless, D. (2006b, August). Trust, distrust and their impact on learning-oriented assessment practices. Paper presented at the European Association for Research on Learning and Instruction Assessment conference, Northumbria, England.

Carless, D., Joughin, G., & Liu, N.-F. (2006). How assessment supports learning: Learning-oriented assessment in action. Hong Kong: Hong Kong University Press.

Ecclestone, K., & Swann, J. (1999). Litigation and learning: Tensions in improving university lecturers assessment practice. Assessment in Education, 6, 377389.

Elton, L., & Laurillard, D. (1979). Trends in student learning. Studies in Higher Education, 4, 87102.

Falchikov, N., (2005). Improving assessment through student involvement. London: Routledge Falmer.

Gibbs, G. (2006). How assessment frames student learning. In C. Bryan & K. Clegg (Eds.), Innovative assessment in higher education (pp. 2336). London: Routledge.

Gibbs, G., & Simpson, C. (2004). Conditions under which assessment supports stu-dents learning. Learning and Teaching in Higher Education, 1,331. Retrieved June 5, 2005, from http://www.glos.ac.uk/departments/clt/lathe/issue1/index.cfm

Gipps, C., Mc Callum, B.,& Hargreaves, E. (2000). What makes a good primary school teacher. London: Routledge Falmer.

Glenwright, P. (2002). Language proficiency assessment for teachers: The effects of benchmarking on writing assessment in Hong Kong schools. Assessing Writing, 8, 84109.

Greer, L. (2001). Does changing the method of assessment of a module improve the performance of a student? Assessment and Evaluation in Higher Education, 26, 127138.

Heathfield, M. (1999). Group-based assessment: An evaluation of the use of assessed tasks as a method of fostering high quality learning. In S. Brown & A. Glasner (Eds.), Assessment matters in higher education (pp. 132145). Buckingham, Eng-land: Society for Research Into Higher Education & Open University Press.


James, D. (2000). Making the graduate: Perspectives on student experience of assess-ment in higher education. In A. Filer (Ed.), Assessment: Social practice and social product (pp. 151167). London: Routledge/Falmer.

Knight, P., & Yorke, M. (2003). Assessment, learning and employability. Maiden-head, England: Society for Research Into Higher Education & Open University Press.

Liu, N.-F., & Carless, D. (2006). Peer feedback: The learning element of peer assess-ment. Teaching in Higher Education, 11, 279290.

McDonald, B., & Boud, D. (2003). The impact of self-assessment on achievement: The effects of self-assessment training on performance in external examinations. Assessment in Education, 10, 209220.

McDowell, L., & Sambell, K. (1999). The experience of innovative assessment: Stu-dent perspectives. In S. Brown & A. Glasner (Eds.), Assessment matters in higher education (pp. 7182). Maidenhead, England: Society for Research Into Higher Education & Open University Press.

Pellegrino, J., Chudowsky, N., & Glaser, R. (Eds.). (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.

Race, P. (2001). A briefing on self, peer and group assessment. Retrieved June 12, 2006, from http://www.heacademy.ac.uk/resources.asp? process = full _ record & section = generic & id = 9

Ramsden, P. (2003). Learning to teach in higher education (2nd ed.). London: Routledge.

Shepard, L. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 414.

Smith, E., & Gorard, S. (2005). They dont give us our marks: The role of forma-tive feedback in student progress. Assessment in Education, 12, 2138.

Stefani, L. (1998). Assessment in partnership with learners. Assessment and Evalua-tion in Higher Education, 23, 339350.

Taras, M. (2005). AssessmentSummative and formativeSome theoretical reflec-tions. British Journal of Educational Studies, 53, 466478.

Taras, M. (2006). Do unto others or not: Equity in feedback for undergraduates. Assessment and Evaluation in Higher Education, 31, 365377.

2 Teaching and Learning in the Context of Mass Lecture

Amy Ah-May Chan-YipHong Kong Institute of Education

INTRODUCTION

This is a preliminary case study report of an action research of a larger proj-ect of the School of Foundations in Education on Teaching and Learning. The project was coordinated by David Grossman and Ming-Fai Hui. Nine colleagues from different departments were involved in investigating their own teaching, within the following general aims:

How do we procedurally establish a sustainable process of free and ropen dialogue with students about how to improve the quality of their learning?What type of evidence do we need to gather to support students self-revaluation of the learning and the development of our teaching?

This report begins from the initiation of participation in the project to the various stages of module delivery and action research. It provides a context of the case study and lessons learned from the researchers perspective.

THE CONTEXT OF STUDY

My case study focused on a core module titled Innovation and Change in Primary Schools in Hong Kong (ICPS) offered for the Bachelor of Educa-tion (primary; 2 year) full-time program. This was a two-credit point mod-ule spread over 13 weeks for 2-contact hr each. The level of study was Year 2 and the cohort consisted of 133 students. Two departments jointly deliv-ered this module, namely the Department of Education Policy and Admin-istration (EPA) and the Department of Curriculum and Instruction (C&I). The former department was the custodian by program design. The module aimed to help student teachers understand innovations and changes in Hong Kong primary schools in relation to the political, economic, social, and cul-tural context. It also attempted to prepare teachers the capacity to cope with and make a positive contribution to the curriculum reform in Hong Kong.

Teaching and Learning in the Context of Mass Lecture 25

The education reform of Hong Kong was first characterized lifelong and life-wide education as represented by Learning for LifeLearning Through Life (Education Commission, 2000). One of the focuses of the reform objec-tives was to develop learning citizens for Hong Kong. And this could only be accomplished through the reform of the school curriculum. School subjects are now grouped into eight key learning areas with focus on nine generic skills and four cross-curriculum perspectives that include moral and civic education, reading, project learning, and using information technol-ogy (Curriculum Development Council, 2000). It is evident that the overall reform rationale lies on the lessening of the impact of public examinations, catering for all students with diverse abilities and linking all educational changes to the economic competitiveness of Hong Kong (Kennedy, 2005). Hence, our student teachers who are still receiving training should be intro-duced to this new structural direction and to the basic principles of curricu-lum reform.

Prior to the commencement of the module, academic staff in the School of Foundations in Education received a memo from the Dean advocating mass lecture for the sake of cost effectiveness and creating time and space for colleagues professional development. This proposal was made with an assumption that mass lectures would reduce the work load of an academic staff in lecture preparation and increase time available for research and fur-ther studies. In support of this call, we, tutors of the said module decided to adopt the mass lecture-tutorial mode. Based on the focus of the module, and the preservice background of the students, we agreed to bring in school practitioners to share their day-to-day field experience, making the mod-ule content closer to the reality. The details and sequence of activities were arranged as shown in Table 2.1. The shaded items represent the tutorial sessions when instructors met their respective group. As you can see in the table, the number 20 to 25 shows the group size for the tutorial sessions, and the group size for the mass lecture was 133 students.

The module assessment was in the form of an essay on an investigation on either (a) a selected policy initiative or (b) a case study of a school-based innovation. Students were required to gather information and data in various forms and use a theoretical framework to critically evaluate its implementation. Students were also encouraged to select their own topics for their assignments.

MOTIVATION TO CONDUCT ACTION RESEARCH

I was curious about the impact of action research and interested in getting involved for the sake of building up experience to help front line teachers conduct their own action research. As soon as the mode of instruction for ICPS was decided, I was at a crossroads. Informed of the drawbacks of many previous attempts to use the mass lecture mode, I was motivated to

26 Chan-Yip

investigate the teaching and learning aspects of mass lectures. I hesitated in participating in any form of action research lest there would not be much freedom for taking any individual action or intervention to improve student learning because of this mass lecture mode. Such limitation would likely defeat the purpose of action research.

At the beginning of my participation in this project, I perceived my part as a trial effort and was prepared to give up if other researchers in the team

Table 2.1 Sequence of Module Content and Activities

Number of Session Activity Durationa

Number of Students in Group

1 Mass lecture: Module introduction and HK Education and the EC reform

2 133

2 Mass lecture: HK Education and the EC Reform

1 133

Tutorial: General introduction and HK education and the EC reform

1 20 to 25

3 Mass lecture: Understanding educa-tional change

2 133

4 Mass lecture: Curriculum reform 2 133

5 Tutorial: Curriculum reform 2 20 to 25

6 Mass lecture: School-based management 1 133

School principals talk on school-based management

1 133

7 Tutorial: School-based management 2 20 to 25

8 Mass lecture: Changing role of a teacher 2 133

9 School Visit: Understanding school phe-nomenon & collection of infornation for the assessment task

NA 133

10 Tutorial: Changing role of a teacher 2 20 to 25

11 School principals seminar on changing role of a teacher

2 133

12 Tutorial: Discussion on innovation and change in HK primary schools

2 20 to 25

13 Reading week NA NA

14 Examination week NA NA

Note. Shaded items are tutorial sessions. HK = Hong Kong; EC = Education Commission; NA = Not Applied.

aGiven in hours.


thought that my actions would be of minimal significance. Meanwhile there were ongoing meetings and talks by the external consultant from the Univer-sity of East Anglia, United Kingdom. There were ongoing discussions among the action research coordinators, the external consultant, and myself. The collegiality developed in the team was strong. Colleagues who did not know each other before had been engaged in an open dialogue of effective teaching strategies, student characteristics, action research, and the writing of a case study report. The talks conducted by the external consultant were inspiring (Elliott, 2001). I was convinced of the value and possibility of exploring teaching and learning aspects of mass lecture in this action research setting. I started to learn more about student learning in mass lectures and to think about how actions could be planned to facilitate learning through observa-tion of other colleagues presentation. I formulated my specific questions:

How would students learn best in mass lectures?rHow should tutorials be conducted to facilitate learning?rWhat are students concerns in the course of study?rWhat are the effective types of lecturerstudent interaction in the con-rtext of mass lectures?How do I help students evaluate their own learning?r

THE STAGES OF THE RESEARCH

Stage 1: Before Commencement of the Module

Problems Associated With Mass Lecture

Before the commencement of the module, tutors met and agreed to adopt a mass lecture-tutorial-seminar mode. This was a move to support the Insti-tutes advocacy of mass lecture. The decision was an attempt to team teach the module and to bring in specialists: tutors and guest speakers. Based on student feedback sought from previous modules with mass lecture, prob-lems associated with this mode of teaching were identified as follows:

1. Weak linkages among lectures delivered by different professors and instructors

2. Weak linkages between lectures and tutorials 3. Communication block among professors and instructors leading to

ambiguous assignment guidelines and requirements 4. Boring lectures due to extensive teacher talk

Actions Adopted

Aware of the drawbacks mentioned above, tutors decided to take the fol-lowing actions:

28 Chan-Yip

A website would be prepared to serve as a teaching and learning sup-rport tool, in which there would be a module outline, a schedule of activities, assignment guidelines, videotaped mass lectures, activities or discussion questions for tutorials, reading materials, a list of refer-ence materials, a list of useful websites, and relevant organizations (addressing Problems 1, 2, & 3);All lecturers should attend all mass lectures and seminars to facili-rtate communication and show mutual support (addressing Problems 1, 2, &3);Each mass lecture should be well prepared and recorded for future ref-rerence, as for example, planning of the subsequent lectures or for those who missed a particular lecture (addressing Problems 1 & 2);Lectures should be prepared in advance and team members be well-rinformed about the content, the reading materials and details of the tutorial activities to follow (addressing Problems 1 & 2);The module outlines would be disseminated in the first lecture. Stu-rdents would be informed of the mode of study, schedules of mass lectures, tutorials, seminars, and school visits (addressing Problems 1, 2, &3);Mode of delivery would be diversified into mass lectures, tutorials, rseminars, and school visits. Team members were encouraged to use multimedia in mass lectures, such as video shows and PowerPoint presentations; (addressing Problem 4)Tutorials would be in the form of discussions, analysis of case and rstudent presentation. (addressing Problems 1, 2, & 3).

Reflection-on-Action

The website was well received at the beginning. Many students inquired about the date of the operation. Students curiosity diminished gradually as they realized that they had got adequate information from mass lectures or from friends. It was quite disappointing to learn that we were unable to put the video clips of mass lectures on the web because of technical problems. A remedial action was taken to let students get access to the videotaped lectures via departmental general offices. Nevertheless, not many students borrowed the tapes. These tapes, in fact, were very useful to lecturers who missed a particular lecture. They portrayed students learning process in details to facilitate tutorial preparation.

Stage 2: Before, During, and After the Mass Lecture

Delimitations and Preparation

My actions were delimited by the team teach mode. There was only one chance for me to mass teach the class and the schedule of tutorials was


predetermined. Activities were mostly prescribed. The power of action research to constantly review and revise teaching strategies was limited. Nev-ertheless, I managed to design activities within my area of responsibility. I was scheduled to conduct one mass lecture to 133 students and four tutorials for 22 students. Chances to observe the effect of three mass lectures by other colleagues allowed me to identify popular instructional methods. Students seemed to be attracted by video shows and PowerPoint presentations. They were mostly attentive and engaged in note taking. Students also showed interest in listening to the presentation of a school principal on school-based management. However, there were not many lecturer-student interactions. It was also difficult to organize student activities in a lecture theatre.

Based on these observations, I paid special attention to the production of PowerPoint presentation and the selection of a video show. To arouse interest, I made use of a pop song for analyzing the lyrics as a lead-in to the introduction of features of post modernism. Music was also used as a background to the introduction of a series of curriculum reform proposals. With the deliberation to create opportunities for student to student interac-tion and teacher interaction, two worksheets were produced for a shift of instructional mode.

During the Mass Lecture

The mass lecture covered the following main teaching points:

Review of key concepts relevant to curriculum change. r (Worksheet 1 for group work)Examples of past curriculum initiatives. r (Video show on current cur-riculum reform)Curriculum reform: Theories and modelsrFeatures of curriculum reform in a postmodern society r (Pop song & lyrics)Curriculum reform in Hong Kong r (Pop song background)Factors affecting the effectiveness of a curriculum reformr(Worksheet 2 for individual work)r

During the mass lecture, I observed that students paid full attention to the lecture and the video show in the first hour. They enjoyed the video and the pop song. There were two worksheets for completion, one on student-teachers previous knowledge on curriculum innovations and the other dis-tributed in the second half of the lecture on curriculum reform proposals (see Appendix 2.1 & 2.2). Students were confident of completing the work-sheets by themselves and showed little attempt to interact with other people. Small group discussion was encouraged, but students were not keen on it. It seemed that students were indifferent and were not concerned about their ability to give the correct answers during the lecture.

30 Chan-Yip

Eighty minutes after the commencement of the lecture, some students stood up and began to leave the lecture hall. They returned after a short break. In view of this, I decided to stop lecturing and give everyone a 5-min-ute break. Towards the end of the lecture, I started to invite students to reflect on their strategies to select teaching materials and teachers reliance on textbooks. Their responses were as predictable as what they had been taught in instructional theories. They claimed that they had analyzed the learners needs prior to their own teaching materials. However when details were asked, they said their major references were the available textbooks. These student-teachers were reluctant to admit that, like most practicing teachers, they relied heavily on textbooks. This informs what teacher educa-tors should strive for when we aim to prepare teachers to face the challenges of the education reform.

Feedback Sought

A questionnaire of a few short open-ended questions was distributed imme-diately after my lecture. This questionnaire aimed at (a) identifying the instructional media and strategy considered most popular by students to inform the next instructional design, and (b) identifying areas of student interest and levels of understanding for their self-assessment of learning as well as my follow-up action. Results showed that most students enjoyed the video presentation and slide show. They also appreciated the music element and acknowledged lecturers good preparation. However, they said that human interaction was inadequate and they also expected more examples of previous curriculum innovations in Hong Kong primary schools. Some quotations of student feedback were as follows:

PowerPoint presentation with music background was refreshing.

Wonderful slides.

The video was interesting and informative.

The lecture and the PowerPoint were well prepared.

Examples of previously adopted curriculum innovations would help.

Students were invited to rank content items according to levels of under-standing. Table 2.2 shows the results.

Students ranked content Items a, b, and c the highest in understanding and i the lowest. An interesting pattern is shown here. It seems that the sequence of presentation is an important factor. The highly ranked items were taught first and the lower ranked last. It is also worth noting that students were unaware of the overall design of the relationship between the mass lecture


and the follow-up tutorial session. The latter was for handling content Items g and h. If students had been informed of this plan, they would not have worried about their knowledge of the curriculum reform proposals. A more detailed description about the learning task for the tutorial is given below in the section, Students Interested Areas and the Planned Learning Tasks, of Stage 3.

Postlecture Interview

A postlecture interview was conducted after the tutorial session with stu-dent volunteers from my tutorial group. Questions similar to those in the worksheet were asked with the aim to obtain additional information about the selection of answers to those questions. Responses showed that there was little discrepancy between the verbal and written answers. Students were also asked why many of them had not talked with their neighbors when they were completing the first worksheet, which was about the cur-riculum reform proposals of the Curriculum Development Council (2000). Two reasons were provided. First, students had heard about reform propos-als from the mass media and other teacher educators. In addition, the video show had also given them some hints. There was not much need to consult

Table 2.2 Perceived Levels of Understanding

Content Itema

Rank Order of Perceived Levels

of Understandingb Interested

Areasc

a. Basic concepts 1 6

b. Rationales of curriculum reform 2 4

c. Curriculum reform and education policy 3 12

d. Models of curriculum orientations and change 5 5

e. Social phenomena of postmodern era 6 22

f. Curriculum reform of the postmodern era 4 19

g. Points of emphasis in HKs curriculum reform 8 44

h. Proposed learning experiences in HKs cur-riculum reform

7 26

i. Factors affecting the implementation of inno-vative curriculum

9 8

Note. HK = Hong Kong.aArranged according to sequence in presentation. bWhere 1 = highest. cFrequency.

32 Chan-Yip

peers in completing the task. Second, students did not mind if they could not fill in all correct answers because other students in the mass lecture could supplement their incomplete answers. They anticipated that I would cover all of those areas in my lecture. When asked about the reasons for leaving the hall in the midst of the lecture, students explained that they were used to taking a break after an hours lecture. It seems that students concentration span was not favorable to lengthy lectures.

Reflection-on-Action

Based on the student feedback provided above, I found some reasons for inactive student interaction. Given that students in this group were potential teachers, they were well informed of a teachers technique to focus students attention on key teaching points and to motivate their learning needs. In other words, they perceived the worksheet as a device to motivate students and considered that interaction was not necessary. However, they did not react to my other concern, that is, learning through peer interaction. The aim of the worksheet was not merely for creating a need to learn and focusing attention. It was also for shifting the mode of instruction from lecture to an interactive mode. To encourage student interaction, it is therefore advisable to distribute one worksheet for a group instead of one for each individual. The task should be an open-response type of activity, so that some students would be invited to share their views.

From my observation as well as student feedback mentioned above, I am convinced that multimedia teaching

Improving Teacher Education Through Action Research

Documents

Transcript of Improving Teacher Education Through Action Research