High-Stakes Language Testing and Young English ... High-Stakes Language Testing and Young English...
Embed Size (px)
Transcript of High-Stakes Language Testing and Young English ... High-Stakes Language Testing and Young English...
High-Stakes Language Testing and Young English Language Learners: Exploring
Motivation and L2 Test Validity in South Korea
John F. Haggerty
A thesis submitted to the Faculty of Graduate Studies and Research
in partial fulfillment of the requirements for the degree of
Master of Arts
© John F. Haggerty
1*1 Library and Archives Canada Published Heritage Branch
395 Wellington Street Ottawa ON K1A 0N4 Canada
Bibliotheque et Archives Canada
Direction du Patrimoine de Pedition
395, rue Wellington Ottawa ON K1A 0N4 Canada
Your file Votre reference ISBN: 978-0-494-79568-2 Our file Notre reference ISBN: 978-0-494-79568-2
The author has granted a non- exclusive license allowing Library and Archives Canada to reproduce, publish, archive, preserve, conserve, communicate to the public by telecommunication or on the Internet, loan, distribute and sell theses worldwide, for commercial or non- commercial purposes, in microform, paper, electronic and/or any other formats.
L'auteur a accorde une licence non exclusive permettant a la Bibliotheque et Archives Canada de reproduire, publier, archiver, sauvegarder, conserver, transmettre au public par telecommunication ou par I'lnternet, preter, distribuer et vendre des theses partout dans le monde, a des fins commerciales ou autres, sur support microforme, papier, electronique et/ou autres formats.
The author retains copyright ownership and moral rights in this thesis. Neither the thesis nor substantial extracts from it may be printed or otherwise reproduced without the author's permission.
L'auteur conserve la propriete du droit d'auteur et des droits moraux qui protege cette these. Ni la these ni des extraits substantiels de celle-ci ne doivent etre imprimes ou autrement reproduits sans son autorisation.
In compliance with the Canadian Privacy Act some supporting forms may have been removed from this thesis.
Conformement a la loi canadienne sur la protection de la vie privee, quelques formulaires secondares ont ete enleves de cette these.
While these forms may be included in the document page count, their removal does not represent any loss of content from the thesis.
Bien que ces formulaires aient inclus dans la pagination, il n'y aura aucun contenu manquant.
This thesis explores the L2 attitudes, beliefs, and self-conceptions of young
language learners in South Korea, based on six learner-background characteristics, in an
attempt better understand their L2 motivational patterns. In addition to considering
gender, educational level, and self-reported L2 proficiency, this study will also consider
the time learners spent preparing for L2 tests, the level of L2 test completed (middle-
school or university-entrance), and the frequency of university-entrance level test
completion. The inclusion of the last three research questions is an explicit attempt to
widen conceptions of L2 motivation to more properly include the effects of L2 testing,
especially in test-driven educational environments. A questionnaire was developed and
administered to middle school students enrolled in a private language school in Seoul
(N=341). An exploratory factor analysis (EFA) approach was utilized in order to identify
factors for further comparison of subgroups. The results indicated that middle school
students differed significantly on a majority of L2 motivational factors depending on: 1)
their perceived L2 proficiency level; 2) the amount of time they spent preparing for L2
tests; and 3) the completion of university-entrance level L2 tests. The implications of
these findings are discussed in relation to some of the major motivational theories
advanced in the fields of psychology and second language acquisition (SLA) and the
potential implications this has for modern notions of validity in language testing. Finally,
some recommendations are offered to improve the administration of high-level, high-
stakes tests that affect English language learners (ELLs) around the world.
I would like to first thank my thesis advisor, Dr. Janna Fox, for her invaluable
guidance in navigating the troubled waters of field research in a foreign country. I
benefitted tremendously from her wealth of experience in language testing and her wise
words of encouragement when things did not go quite as I had planned. She was often the
only sympathetic ear I could find once my closest friends grew weary of my ramblings.
Without her, I am quite sure I could not have completed this research. I would also like to
thank my graduate studies supervisor, Dr. Devon Woods, for having the faith to allow me
to do off-campus research and for inspiring me to follow my heart and pursue research
where I felt it was necessary most. His dogged pursuit to uncover what lies beneath our
beliefs, assumptions, and knowledge has had a far greater impact on this inquiry than he
may have realized. I am also greatly indebted to Joan Grant for her excellent
administrative support and for fully explaining all the options I had available to me. A
special thanks to Karen Rishel for getting necessary paperwork to the correct people on
campus when I could not be there myself. I would also like to thank Young Jin for not
only being an invaluable translator, but also a dear friend whose insight into education
and motivation in Korea helped me to better organize and understand my research. I
would also like to thank the Kiervin family for opening up their home and providing
support during my thesis defence in Ottawa, Canada. Finally, this research would not be
possible without the support of Seong Hye. When I started this study she was my fiancee,
but when I finished she was my wife. I think this speaks the loudest to not just how
important she was in this study, but as it turns out, in my life as well.
Table of Contents
Abstract ii Acknowledgements iii Table of Contents iv List of Tables vi List of Figures and Illustrations vii List of Symbols, Abbreviations and Nomenclature viii
CHAPTER ONE: INTRODUCTION 1
CHAPTER TWO: VALIDITY IN LANGUAGE TESTING 7 2.1 The 'Early' Years: Criterion-related and Content Validity 9 2.2 The Ascendance of Construct Validity 12 2.3 Positivism, Psychometrics, and Validity 13 2.4 The Messick Model 15 2.5 The Bachman and Palmer Model 18 2.6 Kane's Argument-Based Approach 20 2.7 McNamara and the Social Context of Language Testing 22 2.8 Consequential Validity: A Role for Motivation? 24
CHAPTER THREE: MOTIVATION RESEARCH 27 3.1 Motivation in Psychology 29
3.1.1 Achievement Motivation Theory 30 3.1.2 Goal-Setting and Goal-Orientation Theory 31 3.1.3 Attribution Theory 32 3.1.4 Self-Efficacy 33 3.1.5 Self-Worth Theory 34 3.1.6 Self-Determination Theory 35 3.1.7 Weiner's Social Motivation 36
3.2 Motivation in SLA 37 3.2.1 Gardner and Lambert's 'Integrativeness' 37 3.2.2 1990 to the Present Day: Rethinking and Revisualizing L2 Motivation 38 3.2.3 Dornyei's L2 Motivational Self-System 40 3.2.4 A Greater Appreciation of Young ELLs in L2 Motivation 43 3.2.5 A Greater Role for Language Testing in L2 Motivation Research 45
3.3 Development of L2 Motivational Constructs 46 3.3.1 Dornyei -Inspired Constructs 47 3.3.2 Pilot Study Constructs 50
3.4 Learner-background Characteristics (Variables) 51 3.4.1 Gender 51 3.4.2 Education Level 52 3.4.3 Self-reported L2 Proficiency 53 3.4.4 L2 Test Preparation 54 3.4.5 Level and Frequency of L2 Test Completion 54
CHAPTER FOUR: RESEARCH METHOD 56 4.1 Participants 57 4.2 Setting 57 4.3 Instrument 58 4.4 Procedure 59 4.5 Data Analysis 60
CHAPTER FIVE: RESULTS 64 5.1 Exploratory Factor Analysis Results 64
5.1.1 Factor 1: Ought-To L2 Self + Instrumentality (prevention) 67 5.1.2 Factor 2: Attitudes to L2 Community 68 5.1.3 Factor 3: Attitudes to L2 Learning 69 5.1.4 Factor 4: Ideal L2 Self 69 5.1.5 Factor 5: Attitudes to L2 Testing 70 5.1.6 Theoretical Constructs Not Replicated by EFA Procedure 70
5.2 Factor Scores 71 5.2.1 Correlation among Factor Score Variables 71 5.2.2 Results for Gender and Education Level 72 5.2.3 Results for Self-Reported L2 Proficiency 73 5.2.4 Results for L2 Test Preparation 74 5.2.5 Results for Level and Frequency of L2 Test Completion 77
CHAPTER SIX: DISCUSSION 80 6.1 L2 Motivational Factors Identified 80 6.2 Gender and Education Level 82 6.3 Self-Reported L2 Proficiency 84 6.4 L2 Test Preparation 86 6.5 Level and Frequency of L2 Test Completion 87
CHAPTER SEVEN: CONCLUSIONS AND RECOMMENDATIONS 90 7.1 Limitations 90 7.2 Conclusions about L2 Motivation 91 7.3 Conclusions about Validity in Language Testing 95
List of Tables
Table 1: Theoretical Constructs Utilized for Present Study 49
Table 2: Pattern Matrix for 5-Factor Solution (EFA) 66
Table 3: Pearson Product-Moment Correlations for Factor Score Variables 72
Table 5: Factor Scores and Self-Reported L2 Proficiency Level (N=329)