For, With, By: Researchers and Teachers CREOLE, City of Bristol College & CfBT Education Trust...
-
Upload
olivia-lyons -
Category
Documents
-
view
221 -
download
0
Transcript of For, With, By: Researchers and Teachers CREOLE, City of Bristol College & CfBT Education Trust...
For, With, By: Researchers and Teachers
CREOLE, City of Bristol College & CfBT Education Trust
Contextually Situated Research as a Driver Informing Policy, Practice & Theory
Professor Pauline Rea-Dickins
Overview of Presentation
Introduce the research: SPINE
Stakeholder involvement & partnerships
Research drivers, including the stakes &
consequences: real world problems
Some empirical findings from triangulated studies
Some conclusions
2
3 What is SPINE?
• Student Performance in National Examinations: the dynamics of language in school achievement (SPINE) www.bristol.ac.uk/spine (ESRC/DfID RES-167-25-0263)
• Bristol team: Pauline Rea-Dickins, Guoxing Yu, Oksana Afitska, Rosamund Sutherland, Federica Olivero, Sibel Erduran, Neil Ingram, Harvey Goldstein
• Zanzibar team: Zuleikha Khamis, Abdulla Mohammed, Amour Khamis, Mohammed Abeid, Shumbana Said, Haji Mwevura
4
Research Planning, Advisory Group Meetings
• Partnership
• Planning for the research
• Ministry of Education & Vocational Training SMT
Members – Advisory Group
• National Launch
• c. twice yearly updates of key findings
• Discussion of research communication
strategies/dissemination
• Channels for interaction, n.b. Znz Research Team
5
Why is MoEVT Interested? Supportive?
• Previous work by Bristol
• Institutional links and respect (e.g. doctoral training)
• Levels of achievement at end of Basic Education
alarmingly low
• Interest in SPINE: Evidence of policy changes English as MoI to be brought in earlier at primary (std4)
but commitment to raising children bilingually SUZA: universities do research
6
International Research Initiatives & Policy Decisions
Examinations in English: high stakes ‘real world’
problem Accommodations for ELL in US (e.g. Work of Abedi;
Bailey & Butler; Rivera et al) Teaching & learning of Science and Maths through
English: reversal of policy in Malaysia (see also Lan,
2010) Sri Lanka: at O and A level – students are using 2
languages to study & not compulsory to sit the exam in
EL2: they can choose (Punch, 2009)
7
Policy Issues: The Stakes & the Consequences in SSA
• Sub-Sahara Africa: what is the impact of an unfamiliar
language on learning progression & education outcomes?
• Research focus in sub-Saharan Africa more on classroom
interactions & subject learning with less attention for the
formal examining of this subject knowledge• Examining: Formal examinations at end of 2nd year of
secondary in English> 50% of school aged children leave school at the end of
Basic Education as unsuccessful learners: what does the evidence say?
8
Theory-focused Issues: Applied Linguistics
L2 academic language development across school
subjects
Working bilingually in the classroom
Monolingual (L1) examining
Negotiating double constructs in examinations
NS-orientation in examining; prescriptive vs supportive
assessment modalities
9
10 SPINE: theoretical issues - assessment
How valid:• is the dominant view of assessment as quality
measurement model vs. a supportive & context sensitive model (e.g. Pollitt & Ahmed 2009) + dynamic assessment research (Poehner 2008; Poehner & Rea-Dickins 2010)
• is the role of a dominant world language in LTA processes
• are current constructs, e.g. that assume NS norms, and monolingual performance as the normal context for language use
11 SPINE: EXEMPLAR FINDINGS (STUDY 5.1)
What are the four things that man could resemble
whales?
45 students took this item:– 35.6% = no answer– 26.7% = wrong answer– 28.9% = partially correct answer– 8.8% = correct answer
12Int: D1 – No response to Q3 explains (STUDY 5.1)
D1: “because I did not understand by this … this … resemble” (lines 115-117)
Int: “If I tell you that resemble means ‘to look like’ … can you do the
question now?D1: “Yes”Int: OK so what’s the answer?D1: “Man … is warm blooded … and
whales also … whales have lungs and man also have lungs …” (122-133)
Original Items: Summary of Findings (STUDY 5.1)
Very low mean scores across subjects
Student achievement lowest in Maths
Significant differences in achievement across the subjects
Strong correlation between English (exam + vocabulary
test) & performance in other subjects
Factors affecting performance from learner interviews: Not understanding the task, specific words & phrases or the
meaning of tables and diagrams Partial knowledge of topic area & question type Low levels of students’ language proficiency
13
Modified Items - some findings (STUDY 5.1)
• Increase in student response rates
• Increase in accuracy of responses
• Correspondence between ability to provide a correct
translation of task & ability to solve/provide partially
correct answer
• Learners who had difficulty translating task generally
performed poorly
Evidence of linguistic factor + other factors
14
COURSEWORK ASSESSMENT : implementation issues (STUDY 4)
Significant variability among teachers:
• the types of specific activities that constitute teacher
assessment
• the number of activities from which they take marks for
each month
• decisions on how to handle student absenteeism
• how to arrive at the CWA mark: some picked the one
that students did better on, some picked randomly, and
some averaged of all marks
16Schools A-F – limited exposure to EnglishTests of significance: comparison of students performance across the 3 different versions and per subject (STUDY 5.3)
ANOVA
4087.505 2 2043.753 43.666 .000
17411.053 372 46.804
21498.558 374
125.327 2 62.663 1.274 .281
18246.171 371 49.181
18371.497 373
12.587 2 6.293 .256 .774
9149.103 372 24.594
9161.689 374
Between Groups
Within Groups
Total
Between Groups
Within Groups
Total
Between Groups
Within Groups
Total
biology
Chemistry
maths
Sum of Squares df Mean Square F Sig.
17
language version
EnglishBilingualKiswahili
Mea
n of
bio
logy
20
18
16
14
12
10
8
18 Qualitative Analysis of Student Responses(STUDY 5.3) Top performing students
Qn 5: Write about an animal you have studied
• Name of the animal:
• What does the animal look like?
• Where does the animal live?
• Describe how the animal eats?
Qualitative Analysis of Student Responses(STUDY 5.3) Top performing students
Reaching potential: analysis of higher performing
students (see handout)
19
20
Findings from National Form II Data: summary of the multilevel modelling analysis(STUDY 2)
• It is very clear that ENGLISH language proficiency is a significant and substantial predictor of the students’ performance in MATH, BIO & CHEM.
• But: the school-level variances explained in the cons models as well as in the models including ENGLISH as the single explanatory variable demonstrated that a substantial proportion of the variance is attributable to school factors
21 CONSEQUENCES: High stakes classroom assessment - the realities & impact
Impact/Potential Disadvantage
Consequences/Injustice
•Teachers do not use full range of LTA procedures & processes•Use of inappropriately constructed assessment frameworks•Inaccurate CWA of learners•CWA implemented as a series of tests
•Learners not fully supported in their language & content knowledge development & fail to reach potential •Test performance valued over learning •Leave school with poor educational outcomes•Inadequate d-base for decision making about student & learning progression
22 Enhancing Learning & Social Justice
Impact/Potential Disadvantage (examples)
Consequences/Injustice: (examples)
•Learners do not engage or respond poorly in examinations•Subject area (e.g. Biology, maths) construct can only be assessed where a linguistic construct has been successfully negotiated
•Loss of self-esteem & motivation for learning•Learners fail to reach their potential (glass ceiling effect) or fail altogether•Leave school as unsuccessful (e.g. at end of Basic Education)•Unequal access to available resources, educational experiences & work opportunities •Unskilled & unable to join the workforce in turn contributing to social & economic deprivation
Research Evidence as Driver for Change? For Awareness Raising? (1)
For policy:
• Potential through partnership: SUZA, MoEVT
• Triangulated evidence, progressively focused studies
• All learners affected, not only lower achieving learners
For professional practice
• Stakeholder engagement with findings e.g. through
workshops
For community
• Through media, e.g. Kiswahili press
23
Research Evidence as Driver for Change? For Awareness Raising? (2)
For Language Testing and Assessment Community/International Examining Boards
• What evidence is there of the impact of research findings such these on actual testing and assessment policies and practices? (e.g. Shohamy)
• Time to overthrow the dominant & unquestioned role that EL1 has in many examining contexts: time for a coup!
• Time to reconceptualise constructs: NS orientations• Investigate supportive & context sensitive approaches
to assessment
24
Research Evidence as Driver for Change? For Awareness Raising? (3)
• Where is the most impact likely?– For policy?– For professional practice?– For community?– For LTA/Applied Linguistics community– For International Examining boards?
• Where do partnership strengths lie?
25