“Evaluation of ‘Best Practice’ Research in Modern Foreign Languages”(DRAFT: not to be quoted without permission)

Rosamond MitchellUniversity of Southampton

Ivy HoggThe Mountbatten School & Language College

Paper presented at the Symposium 'Towards a research strategy for teaching and learning in Modern Foreign Languages', Annual Conference of the British Educational Research Association, Leeds University, 13-15 September 2001

AbstractThis contribution presents preliminary findings from a cluster of ‘Best Practice’ teacher action research projects conducted during 2000-1 at a specialist Language College in collaboration with the University of Southampton. The focus of the projects has been to develop aspects of grammar teaching methodology, to assess the impact of different grammar teaching strategies on pupils’ learning, and to work towards increased coherence and progression in the handling of grammar in the MFLs curriculum. The ‘Best Practice’ research model is evaluated and prospects for the development of longer term research partnerships between schools and HEIs are reviewed.

Introduction

The ‘Best Practice Research Scholarships’ (BPRS) scheme was launched by DfEE in spring

2000, as an opportunity for teachers to engage in classroom research which “can spread

excellent practice and improve teaching and learning in the classroom” (DfEE 2000:2). The

scheme appeared to have a double motivation, partly to offer teachers a fresh opportunity for

continuing professional development, and partly to strengthen the impetus for educational

research which would be perceived as immediately relevant to practitioners’ concerns, and

focus centrally on the improvement of classroom learning and teaching. (This overall policy

steer for educational research is fully apparent in a range of other initiatives, including the

ESRC Teaching and Learning Research Programme and the debates of the National Forum

for Educational Research: see e.g. <http://www.ex.ac.uk/ESRC-TLRP>.)

The purpose of the BPRS scheme was defined as:

“To enable teachers to undertake classroom-based and sharply focused small scale studies in priority areas, and to apply and disseminate their findings. DfEE is keen to support teachers in using research processes –such as observation of pupils, interviews and analysis of relevant data and documents – to investigate classroom

1

practice. This is a good way of building knowledge and understanding about raising standards of teaching and learning. It can have benefits for the individual teacher; for their school; and for other schools in sharing lessons learned. It can be a particularly valuable activity for experienced teachers to share good practice…” (DfEE 2000: 1).

A number of priority areas were defined for the scheme, and research proposals were to be

“particularly welcomed” within these areas. These included a number of themes which

overlap with current subject-specific concerns of MFLs teachers, including:

Raising standards and gender issues;

Raising achievement at Key Stage 3.

Consequently the scheme appeared attractive, as a possible new funding stream which could

support language teacher research into topics such as: the motivation of boys for MFLs

learning; the quality and appropriacy of the present MFLs curriculum for its target audience;

the effectiveness of current teaching approaches for MFLs.

This paper will report on a small selection of MFLs projects in which the authors were

involved as research partners in the first year of the scheme. These 2000-1 projects are still

not fully analysed, but we will focus on lessons learned to date on a range of issues,

including:

Research designs and approaches appropriate for BPRS ‘partnership’ research;

Sourcing and developing appropriate research skills and instrumentation within a

partnership;

The likely contribution of involvement with BPRS research to language teachers’

professional development, at individual and local level;

The potential of BPRS research to contribute to understandings of learning and teaching

processes, beyond the local context;

Practical issues involved in the design and implementation of BPRS research.

The school-university partnership

In the first year of the BPRS scheme (2000-1), educational researchers from Southampton

University have been involved in a number of partnerships with individual teachers, school

2

based groups and LEAs. Two MFLs partnerships were established, one with Hampshire

LEA, the other with The Mountbatten School & Language College.

In both cases, the best practice MFLs link was based on an established relationship.

Southampton MFLs staff have collaborated with Hants LEA MFLs inspectorate for a number

of years for a variety of purposes, most recently in joint management of the Southampton &

Hants Comenius Centre. Mountbatten School has been a longstanding member of the

Southampton teacher training partnership; members of the MFLs staff have undertaken

masters level study and a substantial research secondment at the university (Fleming 1997,

Hogg 1998), and the school has given substantial access to university research students

working on SLA topics, for lesson observations and measurement of aspects of pupils’

MFLs ability (e.g. Rule 2001). The successful bid for Language College status, recently

submitted jointly by Mountbatten and a neighbouring school, was supported by the university

and contained explicit commitments to an ongoing programme of classroom research.

In the Mountbatten case, therefore, the BPRS scheme provided resources for this agreed

commitment to research to be taken forward. Similarly, little debate was needed over the

broad research theme to be pursued, as grammar pedagogy had already been identified on

both sides in earlier research and publications as a theme of central interest for improvement

of consistency and progression in UK MFLs curriculum and pedagogy (see e.g. Hogg 1998,

Mitchell 2000).

The BPRS scheme provides small scale resourcing which can be used for a range of

purposes: the provision of advice/ support by HE institutions and other research advisers; the

purchase of supply cover, to allow participating teachers an element of time for research

planning, data analysis etc; and purchase of consumables and other research resources. At

Southampton this resourcing was used following a number of different models. The Hants

LEA MFLs partnership originally envisaged staging a taught ‘outreach’ course, for a cohort

of MFLs teachers from one Hampshire region, which could provide participants with a

formal element of research training for BPRS purposes, and also earn M level credit through

the submission of assessed coursework, if desired. In the event, insufficient teachers

volunteered to take part for a formal ‘outreach’ course to be viable. Instead, three out of the

four participants enrolled individually with the university for a masters programme by

individual research (MAEd Flexible Study), using BPRS funding to cover relevant part time

3

fees. Each individual participant received personal tutoring and support from a designated

university researcher, at the university and in school, as well as advice from LEA personnel.

These teachers are writing papers concerning aspects of their projects which are being

presented as MAEd assignments and will appear in due course on the BPRS website as

“Level 2” papers.

The Mountbatten BPRS collaboration worked differently. Having been successful very

recently in gaining joint Language College status, the Mountbatten MFLs department was

exceptionally busy with the Language College launch during 2000-1. Indeed the practical

effect of the launch of the BPRS initiative was to accelerate the research strand of Language

College activity, which had been expected to become a main focus somewhat later in the

Language College development plan. Despite all their other commitments, three Mountbatten

teachers undertook to run BPRS scheme projects in their classrooms, linked to the shared

overall theme of effective MFLs grammar pedagogy; the Deputy Language College

Coordinator acted as coordinator for the group, and oversaw the bidding process and financial

arrangements for the project cluster. Given their other commitments and career stages, it did

not make sense for the Mountbatten teachers to carry out their BPRS work within a higher

degree framework. Instead, a doctoral student from the university (Marsden) was appointed

as Research Assistant to the project cluster, with a brief to assist the teachers in

operationalising their research plans, including the development of research instruments

appropriate to the research design, data gathering and data analysis. A permanent university

staff member acted as research adviser to the group; this paper is co-authored by the school

coordinator for the Mountbatten BPRS cluster (Hogg) and the university adviser (Mitchell).

(While this paper is mainly concerned with the experience of the Mountbatten cluster, the

involvement of university MFLs researchers in two different BPRS models allows for some

comparative perspective when evaluating the usefulness of the scheme, and this will be

drawn upon briefly in the conclusions to the paper.)

The Mountbatten project cluster

The shared overall aim of the three BPRS projects within the Mountbatten cluster, as stated in

the joint applications made in summer 2000, was:

4

“To investigate the teaching of target language grammar at Key Stage 3, for a variety of languages and levels, so as to explore its effectiveness and impact on learners’ knowledge and use of grammar”.

This aim arose from a longstanding concern of the MFLs department that grammar

progression was insufficiently explicit in the Key Stage 3 National Curriculum for MFLs, and

an ongoing commitment to developing a departmental scheme of work which would include

an explicit grammar strand. The development of an appropriate shared pedagogy was a major

concern, and the BPRS projects were seen as a way of tackling this issue which could benefit

the MFLs department in the short to medium term. At the same time the Head Teacher in her

supporting comments recognised the potential staff development value of BPRS activity:

“It offers an excellent opportunity for […] the department to benefit from structured and reflective professional development”.

Thus the BPRS cluster in this school was expected to assist in promoting curriculum

development, and staff development within the institution. At the same time the participants

envisaged eventually contributing to the evidence base available to the MFLs profession

more generally, on “effective MFLs grammar teaching”.

The three projects within the cluster had a broadly similar design, though they differed in

detail. Each participating teacher planned and implemented a teaching intervention on some

aspect of the target language verb system (either French or German), with a single Key Stage

3 class, intended to run over a period of a single term maximum (some timespans stretched

later). With the support of the Research Assistant, language tests were devised reflecting the

particular verb focus and estimated language level, and these tests were administered as both

pre- and post-tests to the selected classes (only), to measure pupil progress on the selected

grammar themes. Classwork relating to the grammar themes was also collected as

complementary evidence of pupil performance; the teachers kept lesson plans and other

records of the teaching; and the university members undertook observations of selected

lessons, some of which were audiorecorded.

All three projects thus followed an ‘action research’ design (Cohen, Manion & Morrison

2000: 226-41, Nunan 1989), and could be expected to share the strengths as well as the

limitations of this approach, as far as explorations of effective learning and teaching are

concerned. The teachers had a good preliminary understanding of what they were aiming to

5

achieve, and were ‘testing’ their own beliefs regarding effective pedagogy. (Of course, these

understandings and beliefs could be expected to evolve and develop in the course of the

research; a first step towards this was the need to articulate what the research intervention

would be.) The work was embedded in an existing teacher-class relationship, and was

informed by teachers’ professional understanding of achievement to be expected at Key Stage

3, as well as by their knowledge of the particular classes involved in the research. Teachers’

‘insider’ perspective on classroom pedagogy, e.g. on which activities elicited the most

positive responses from pupils and moved learning forwards, was in principle available

(though we were aware that articulating these professional judgements and understandings is

not an easy or routine process). Thus in principle, action research should make it possible to

develop clear understandings regarding the relationship between teaching activities

undertaken with the selected classes, and the patterns of grammar learning which ensued.

On the other hand, as may be expected of an action research approach, the research was

informed only to a limited extent by the published research literature on grammar pedagogy

(mainly as mediated by the university members of the group); it can be argued that not a great

deal has been published which is directly relevant to the KS3 language level, but even if there

was, BPRS teachers are not in a position to undertake any ambitious literature survey work

first-hand. Thus in particular, the teaching activities which were the focus of the research

derived primarily from teachers’ craft knowledge and experience, rather than connecting

directly with any existing research programme on language pedagogy (e.g. into task based

learning, processing instruction, input enhancement, learning strategies…). Secondly, all the

teachers opted to work with a single class which would receive the ‘special’ grammar focus,

without attempting to run any kind of ‘control group’ in parallel. This clearly influences the

kinds of claims which can be made regarding the impact/ effectiveness of the teaching

procedures which we are investigating. The influence on learning outcomes of the unique

characteristics of the given class group, in combination with the pedagogic style of the

individual teacher, is a significant variable which is not straightforward to capture and

describe. However, we believe that teacher researchers, with their ‘insider’ perspective, are

well placed to undertake this task using a qualitative/ interpretive approach. The main

alternative approach to investigating learning-teaching relationships - formal classroom

experiments involving control groups and statistically based hypothesis testing – is anyhow

not normally appropriate or manageable for part time teacher researchers.

6

All fieldwork for the three projects within the Mountbatten cluster was completed as planned

during the 2000-1 school year, and writing up in BPRS format is now in progress. The

projects are outlined below, in varying detail reflecting progress to date in analysis/ writing

up.

Project 1: the ‘verb’ concept (Year 7, German)

This investigation was devised by Teacher 1, and conducted with a Year 7 mixed ability class

taking German as their first foreign language (27 out of the 28 class members were complete

beginners). The study had the following specific objectives:

“1. To devise and implement activities for the explicit teaching of German verb morphology, focussing on the introduction of the concept ‘verb’ and verb inflection, plus work on present tense verb paradigms.

“2. To monitor pupils’ developing understanding of the target structure, through interviews and group discussions

“3. To monitor and evaluate pupils’ developing control of the target structure, through short, focussed language tests.” (Proposal, Project 1.)

A test battery was devised by the teacher and RA, and administered to the class as a pre-test

in November, and then re-administered as a post-test in May. This written test battery

included a mix of metalinguistic activities (e.g. answering the question “What is a verb?”,

spotting verbs and pronouns in German sentences), translation and gap-filling, which

sampled pupils’ knowledge of the German pronoun system and verb inflections. As part of

the post-test battery only, a sample of pupils took part in a controlled German speaking task

with the RA (answering a set of questions). In addition, a sample of the pupils undertook a

range of metalinguistic problem-solving tasks individually with the RA (both pre- and post-

intervention: tasks included sorting jumbled sentences; giving e.g.s of verbs; spotting errors

in German sentences). Together, this range of activities provided very full data on pupils’

growing understanding and conscious control of the present tense verb paradigm. Finally,

samples of written classwork were collected in April and in June, which provided evidence

on children’s ability to generate German sentences containing verbs, when asked to produce a

piece of extensive writing.

From November through May, while working with the German coursebook Logo 1, Teacher

1 followed the topics and content of the coursebook, but capitalised on all appropriate

opportunities to introduce/ practise /analyse elicit production of different parts of the present

7

tense paradigm, bit by bit. (Some ingenuity was needed, late on, to ensure that plurals ‘wir’

and ‘sie’ were introduced!) Early in the intervention, a full lesson was taught (in English)

with a “Knowledge about Language” focus, eliciting the conceptual understanding of ‘verb’

which pupils had brought from their primary school National Literacy Strategy experience,

and extending the commonly-offered definition of a ‘doing word’ to include concepts of

‘having’ and ‘being’; to introduce/ consolidate the concept of verb tense; and to introduce the

notion of verb inflection, and its relationship with the pronoun system. Specially devised

materials were used for this, offering cross-language comparisons (English, Italian, German),

plus various verb-spotting activities. This conceptually focussed lesson formed a reference

point later, though the full present tense verb paradigm was presented only at the end of the

intervention, after pupils had encountered and practised almost all persons and endings.

Teacher 1 explains that the cycle concluded with another metalinguistic episode, i.e. “a whole

class discussion on what we had learned about verbs this year so far. I made notes of their

rules and examples and wrote them up. They learned them in yet another homework. They

were rightly proud of their ‘verb factsheet’” (Draft report, Project 1).

To date, a draft report has been written by Teacher 1 which contains a reflective/ evaluative

account of the teaching style and activities undertaken within the project, plus a summary

overview of pupils’ achievement on the pre- and post-test battery. Sections analysing pupils’

performance in classwork, and relating the performance of this particular class to that of the

other four parallel Year 7 German classes in the year group, on an end of year test, remain to

be written.

The evidence available so far suggests that this Year 7 mixed ability class made good general

progress in developing their understanding of key grammar concepts, and in developing

analytic knowledge of the present tense paradigm. Impressionistic analysis of their less

planned performances (e.g. extensive writing) suggests that all members of the group are able

to construct German sentences around verbs, and most attempt inflections, though these are

not yet fully under control (as would be predicted by much interlanguage research). The oral

interview data with a sample of pupils (post-test only) produced generally accurate 1st person

inflections, and more variability with 3rd person; these pupils retained and could reproduce an

‘impressive’ store of chunks. Data from the end of year test administered to this and four

other Year 7 mixed ability German classes is not yet analysed, but will allow Teacher 1 to

8

cross-check her impression that this class are more confident “sentence builders” than their

peers.

Teacher 1 has produced a full and insightful account and evaluation of her teaching during

the BPRS project, which echoes the findings of much ‘generalist’ research on effective

pedagogy, as well as research on effective language teaching. For example, she set herself a

clear strategic aim for the period, and specific lessons, activities and individual classroom

exchanges can be ‘read’ as a sequence which leads clearly in a single overall direction. See

her comments on lesson planning:

“Each lesson is very carefully planned and orchestrated around that day’s linguistic objectives. (This planning included a conscious decision on when to include Knowledge about Language, and when to make grammar explicit or leave it implicit.) I decide in advance whether I am going to teach the language as a chunk only, whether I am going to break the chunk down partially or whether I want to break it down totally and do grammatical analysis. Generally I do a lot of revisiting of vocabulary and structures in a lesson, as well as teaching something new and making the link between what is new and what they know already… Finally I try to ensure that a lesson does not degenerate into a list of vocabulary: I guide [the pupils] into using it as part of a meaningful structure before the end of the lesson.” (Draft report, Project 1.)

In addition to this strategic planning, Teacher 1 was observed running skilled, whole class

‘interactive teaching’ episodes, where reasoning and thinking processes were modelled and

displayed. For example, in the KAL lesson cited above, she diagnosed pupils’ existing state

of understanding of ‘verbs’ effectively and encouragingly in a whole class dialogue, and

connected the new, more complex understanding she wished to develop with their learning

from the primary school Literacy Hour. These features relate clearly to characteristics of

effective teaching identified in the generalist literature (see e.g. Scheerens 1992, Creemers

1994, Harris 1998). Similarly, Teacher 1’s active but focussed style of target language error

correction finds justification in a range of language pedagogy research (see review in

Mitchell 2000). For example, she typically elicited corrections actively from pupils, rather

than relying only on recasting incorrect utterances, and encouraged accuracy with regular

praise of correct utterances.

Teacher 1’s reflections on her teaching include some critical self-evaluations, which are

concerned primarily with issues of balance and relationships between explicit grammar

pedagogy and other key elements of her overall MFLs approach. One concern was the

9

balance between English-medium and TL-medium instruction. Teacher 1 felt that it was

appropriate to conduct activities such as the KAL ‘concepts’ lesson in English, but felt also

that this created a precedent for later discussions about grammar, so that a higher proportion

of classroom talk was conducted in English than she had anticipated or felt desirable. Other

concerns were the balance between “talking in the language” and “talking about it”, and a

possible loss of pupil opportunities to build fluency as opposed to accuracy; and a sense that

at least the weaker pupils may have been “overloaded” with grammar talk and grammar

forms:

“I began to feel towards the end that I was doing too much total breakdown (grammatical analysis [of sentence-length chunks]). It seemed to me that I was taking every opportunity when a verb presented itself, which is obviously very frequently, to question them about their learning as if I was obsessed by it [!]. Mostly this would be in English too…”

In evaluating her teaching, Teacher 1 also eventually questioned her commitment to

introducing and practising the entire verb paradigm in the time allowed for the intervention.

A related worry concerned possible pupil confusion over ‘sie/Sie’ forms. (Interestingly, the

task requiring translation of pronouns showed increases at post-test in recognition of

sie=they, Sie=you, but a slight decrease in recognition of sie=she.)

In conclusion, then, when fully written up, Project 1 is likely to offer valuable guidance on a)

how to take young secondary pupils’ grammar understandings beyond the basic concepts

developed in primary school; b) how dense the content of an early MFLs grammar

curriculum can reasonably be, in terms of learners’ cognitive capacity as well as their interest

and confidence; c) the tensions and trade-offs involved in interweaving an explicit grammar

strand into a varied and balanced MFLs experience; and d) the likely impact on learners’

explicit knowledge of TL forms, and their ability to manipulate them at least in controlled

situations.

Project 2: Past tense morphology (Year 8, French)

This investigation was devised by Teacher 2, with guidance from the school coordinator, and

was conducted with a Year 8 upper ability set in their second year of French study, for whom

this was their first foreign language (28 pupils). The study had the following specific

objectives:

10

“1. To devise and implement activities for the explicit teaching of French verb morphology, focusing on the introduction of past tense forms (passé composé).

“2. To monitor pupils’ developing understanding of the target structure, through interviews and group discussions

“3. To monitor and evaluate pupils’ developing control of the target structure, through short, focussed language tests.” (Proposal, Project 2.)

Again, a test battery was devised by Teacher 2 together with the RA, to reflect these

objectives, and was administered both as a pre- and post-test. Overall, the battery explored

pupils’ ability to recognise and produce present and perfect forms, using all four skills

(L,S,R,W). The post-test also included a questionnaire which explored pupils’ explicit

knowledge about the formation of these tenses; this questionnaire was an example of unaided

instrument design, by Teacher 2. Most parts of the battery involved selected responses or

constrained constructed response formats; the oral test was closest to a free response format,

as it required pupils to generate 5 utterances describing their holiday, and negotiate a meeting

day and time.

The pre-test battery was administered in November, prior to any explicit instruction for the

new grammar focus (passé composé). The implementation of the passé composé instruction

has not yet been written up in detail (a draft is in preparation). Impressionistically, it is

possible to say that Teacher 2 adopted a more formal, explicit instructional style than Teacher

1. She also found at the start of the project, that her assumptions regarding pupils’ prior

control of present tense morphology were too optimistic, and some time was devoted to

reviewing and consolidating this topic before the pre-tests, and before the passé composé was

introduced. The post-test battery was administered in May.

Results of the pre- and post-tests for Project 2 have been analysed in some detail (Marsden

2001). Comparisons are affected somewhat by pupil absenteeism (only 19/28 pupils

completed all tests). However, overall, statistically significant gains were made on the

listening, reading and writing components of the battery. Speaking perfomance showed some

qualitative improvement, but this development was not sufficient to be statistically significant

on a test where only formally correct utterances received credit. Speaking test results were

also confounded by pupils’ better-than-predicted performance on the pre-test, where a range

of perfect tense forms were apparently produced correctly despite lack of prior instruction!

11

(However the difficulty should be noted in distinguishing between forms such as ‘je/j’ai’ and

‘-er/é’, in learners’ oral production.)

An exploration of relationships among the different types of knowledge/ skill tested has also

been carried out (Marsden 2001). While tentative, this suggests that pupil achievement in

reading and writing was (weakly) positively correlated, but that achievement in listening and

speaking was not statistically related. Furthermore, achievement in productive skills was not

statistically related to receptive skills (unlike findings reported by e.g. Page 1999, Macrory &

Stone 2000). Finally, it was shown that pupils’ metalinguistic knowledge was significantly

related to their reading and writing ability, but was not significantly related to listening and

speaking performance. These findings suggest overall that the instructional style adopted had

its biggest impact on metalinguistic knowledge, and on language skills other than speaking.

This is useful confirmation of the likely benefits, but also the limitations (at least in the short

term), of an ‘up front’, explicit grammar instruction style.

In conclusion, when fully written up, Project 2 is likely to offer evidence on a) learners’

developing control of target language verb morphology, including contrast between present

and perfect tense forms; b) relationships among different kinds of knowledge (both

metalinguistic understanding, and ability to recognise/ produce forms under varying

conditions); and c) the impact on this development of a relatively formal and explicit style of

grammar instruction.

Project 3: Verb movement (Year 9, German)

This investigation was devised by Teacher 3, with guidance from the school coordinator, and

was conducted with a Year 9 middle ability set taking German as their first foreign language

(30 pupils). The study had the following specific objectives:

“1. To devise and implement activities for the explicit teaching of German verb morphology, focusing on verb placement and word order in a variety of sentence patterns.

“2. To monitor pupils’ developing understanding of the target structure, through interviews and group discussions

“3. To monitor and evaluate pupils’ developing control of the target structure, through short, focussed language tests.” (Proposal, Project 3.)

As in the other two studies, the instructional programme was devised by Teacher 3, and a pre-

and post-test battery was devised in collaboration with the RA. The pre- and post-tests were

12

administered in January and May; reports are not yet available from this project, though from

observational data it can be said that the grammar teaching approach was more implicit and

lessons more functionally organised than in Projects 1 and 2. Preliminary test results suggest

that learning gains were also made, but no details are yet available.

When written up, therefore, Project 3 should offer evidence on a) learners’ developing

control of target language sentence structure including aspects of verb movement, and b) the

impact of a relatively implicit/ inductive grammar teaching style on this development, as well

as c) teacher thinking and evidence regarding the range of current practice which contributes

to grammar pedagogy.

Discussion and evaluation

In this final section, we evaluate the main benefits and limitations of this approach to

classroom research on MFLs learning and teaching, in the light of our shared experience to

date.

Project design issues

The overall approach adopted (a cluster of 3 related, but relatively independent, action

research projects ) was appropriate to the special circumstances of the Language College in

its first year of operation, with an exceptionally heavy range of demands on the MFLs staff/

department. (It is significant that the 3 participating teachers comprised one senior member of

the MFLs department with an existing interest in research, plus two relatively junior

members, with limited Language College responsibilities. A fourth, experienced teacher who

considered joining the research group, eventually withdrew because of competing LC

commitments and responsibilities.) The three action research projects could and did run

relatively independently, though successfully implementing a common 3-stage model (pre-

testing, teaching intervention, post-testing), with the part time RA in practice providing the

main cross-project link, as well as offering a ‘sounding board’ and technical support for

individual teachers.

Meetings of the teacher group were held from time to time, both with and without inputs

from the university participants e.g. on project design and report-writing, but LC

commitments meant these were not frequent enough for detailed planning and monitoring of

progress to be carried out in a collaborative way. Arguably this led to continuing dependency

13

on the RA, to a greater extent than might have been needed had internal cluster meetings

been more frequent - this should be possible in less exceptional circumstances.

One important lesson learned related to the scale of the action research projects undertaken –

all three teachers felt, with hindsight, that they had been too ambitious, and that a smaller

‘patch’ of target language grammar as focus would have led to a more satisfactory/ complete

investigation.

A strategic design issue worth further consideration, concerns the relationship among the

different projects undertaken. While they shared a common focus on grammar pedagogy,

each teacher was testing his/her own individual set of teaching principles, with different year

groups (and two different languages). This approach threw a lot of responsibility on

individual teachers for all phases of their particular project, but also meant that teachers could

operate relatively independently (as we have seen), and also seemed to allow for greater

‘coverage’ of the Key Stage 3 curriculum, shedding light on a range of different aspects,

though each in a one-class context. An alternative approach, which could in principle

increase the power/ generalisability of such small scale research, would be to ‘pool’ more

aspects of the research design. For example, a group of teachers could agree on a common set

of principles for grammar pedagogy, and attempt to implement them with a number of

different classes. All group members could work with the same language/ grammar focus,

and a common set of measurement instruments could be adopted. Different group members

could take responsibility for different elements of the design and analysis process, across

class groups… perhaps making the research process less daunting through this division of

labour, and also building a tighter ‘team’ ethos. Such an approach would however presuppose

a teacher group which was able to prioritise the research as its main development activity for

the relevant block of time – which was not the case during this Language College founding

year.

Making sense of teaching

The distinctive value of this style of classroom action research depends crucially on teachers’

ability to articulate the principles which guide their classroom teaching, and to document how

these principles work in practice, through records, analyses and evaluations of actual lessons.

Where such accounts of teaching can be linked in detail to patterns of pupil development, as

revealed by suitable test data and analyses of ongoing classwork, action research is capable of

14

making its own contribution to our understanding of cause-effect relationships, for classroom

instruction. The outline of Project 1 given above, is beginning to make such a contribution in

our view, which will be strengthened further when additional analysis means links can be

made in even more detail to the evidence on pupil progress/ development.

Articulating ‘what works’ in this explicit way does not come naturally to every teacher

however. No teacher in this study is finding this an easy process. The university participants

have offered some support (in the shape of lesson observation records, suggestions regarding

report writing formats, and feedback on drafts), but the quality of the ‘insider’ reports on the

teaching process which will be produced by the less experienced teachers in the cluster is not

yet known. (Teacher 3 has now left the school and is working independently on his report.)

Making sense of learning gains

Effective measurement of learning is a second fundamental requirement of research into the

language learning-teaching relationship. The design of valid and reliable measurement

instruments, whether tests of language skills, or measures of language attitudes and

motivation, or metalinguistic knowledge, is a timeconsuming and technical business, which

could dominate the timescale and resources of a BPRS-size project, if undertaken from

scratch each time (Bachman & Palmer 1996; Munn & Drever 1995). While teachers may

have extensive experience of writing class tests, this does not mean they already possess the

necessary skills to devise research instruments, and in the present context, the RA’s

contribution to the design of instrumentation for all three Mountbatten projects was essential

to their success. (The other Hampshire BPRS teachers devised attitude questionnaires as well

as language tests for their own projects, and succeeded in doing so only by imitating existing,

published examples very closely.) Similarly, the analysis of test statistics is a technical

business, and to date the only statistical analyses conducted for the cluster have been the

work of the RA (Marsden 2001).

On this issue of measurement, two issues suggest themselves. One is that classroom research

groups learn to exploit, as far as is appropriate, the increasingly sophisticated databases

maintained by schools on pupils’ general abilities and achievement (including e.g. CATS and

/or SATs data). Used properly, these databases can provide background data on the

‘typicality’ of classes singled out for action research initiatives, at least in terms of their

cognitive and general academic ability, and thus increase confidence in the robustness of

15

findings deriving from one-class samples. (For example, in final reporting of projects such as

these Mountbatten examples, it should be possible to provide comparative statistical data

which will clarify what is really meant by the labels ‘mixed ability’, ‘upper ability’, ‘mid

ability’ as applied to the particular classes involved, and will help a teacher readership to

clarify the relevance of the teaching methods described to their own Key Stage 3 groups.)

Secondly, an obvious development which could be extremely beneficial to any more

sustained programme of collaborative classroom research, would be the establishment of a

shared databank of MFLs test items and indeed entire tests, tagged by language (French,

German, Spanish), by level (Key Stage if not National Curriculum level), by skill, and by

linguistic focus. The availability of a good range of fine-grained achievement tests with

known characteristics, at least for the most popular languages, could help teachers working

on many types of teaching-learning research to gather baseline language proficiency data, and

to measure progress thereafter. Such off-the-shelf tests would not meet the complete

measurement needs of every research project, but could form the basis of useful comparisons,

of a much more focussed and analytic nature than e.g. global assessments of National

Curriculum levels, or indeed GCSE grades. We wonder whether this fairly obvious idea has

occurred to others working in BPRS networks, and whether the DfES sponsors would be

interested in promoting collaborative activity to design and produce such instruments…

Practicalities of the BPRS scheme

As members of the first cohort involved in BPRS research, the Mountbatten group

experienced a number of uncertainties which will hopefully not affect later cohorts. These

related to the expected timescale of projects, to the mechanisms for claiming funding and the

duration of awards, and to the nature of the reports expected. For example, the original DfEE

document outlining the scheme envisaged research projects approved during summer 2000

for an autumn term start would “last up to three terms in length”, with reports to be submitted

by December 2001. However it turned out belatedly that actual funding related to the

financial year 2000-1, i.e. all monies supporting the research had to be claimed and spent on a

much shorter timescale, by the end of March 2001 (nine months before the planned end of the

research). Ensuring the money was effectively used by this unexpectedly early deadline was

diversionary of effort, mid year; and report-writing was well under way, to suit the

constraints of the teachers’ working year, by the time the definitive guidelines and expected

formats for reporting became available (July 2001!).

16

A more significant constraint on the effectiveness of the BPRS scheme however, which

seems to have been carried forward into the second year of the scheme (the 2001

competition: see DfEE 2001) are the inbuilt expectations regarding the balance of

expenditure on supply cover and other elements (HEI advice, resources etc). The ‘cap’ on the

supply cover element severely limits the amount of planning and reflection time which

teachers can invest in their BPRS research, unless they are willing to somehow find, and

invest, considerable amounts of voluntary, ‘spare’ time. Without exception, all participants in

the 2000-1 Mountbatten MFLs cluster (and also those in the Hants LEA MFLs group) were

placed under considerable personal pressure by the demands of their BPRS projects. All have

been very ‘successful’ to date, in the sense that original research designs were fully

implemented and considerable volumes of good quality data have been generated. But

because of priority given to worthwhile implementation and data gathering, within a limited

amount of available time, full analysis and writing up will mostly not have been completed

by the time the new school year begins in September 2001. These teacher participants will

then be faced with a choice: a partial report which does not do full justice to the data

collected, or multi-stage reporting (in the first instance meeting DfES formal requirements for

reporting on 2000-1 projects, but with ongoing further analysis undertaken in their own

‘spare’ time, which leads to richer, more digested accounts of individual projects and perhaps

to cross-project comparisons and links…). It is noteworthy but not surprising, that none of the

Hampshire MFLs participants in 2000-1 have applied for further BPRS funding for 2001-2!

Even for the committed, 2 or 3 year gaps between fresh classroom initiatives like this seem

desirable to allow individual projects to be fully analysed, reported and ‘digested’. But more

generally, unless competing demands on teachers’ time are systematically reduced, it seems

unlikely that the BPRS scheme will appeal as a routine part of teachers’continuing

professional development, or achieve its full potential in contributing to our understanding of

learning and teaching processes.

Conclusion and future prospects

This paper has provided us with the chance to reflect at an interim stage on our shared BPRS

experience, and to evaluate the achievements and further potential of the scheme both for

teachers’ CPD, and also as a means to develop the evidence and research base underpinning

MFLs education. We remain committed to our classroom research partnership, and have

other collaborative projects planned/ under way within the Language College environment.

17

These include a formal teaching experiment on grammar pedagogy (Marsden, this

symposium); a survey of Key Stage 4 pupils’ attitudes towards the increased language

learning requirements of the new Language College; and ongoing SLA research on French

interlanguage, which moves to Key Stage 4. Compared with these initiatives however, the

most distinctive features of the BPRS work in 2000-1 were the teacher leadership role, and

the relationship of the research to ongoing curriculum development needs within a particular

MFLs department, as well as to more general MFL research concerns. As presently

constituted the BPRS scheme is not yet ideally suited to promote these features in the longer

term. We hope that others with interests in teacher professionalism as well as in the health

and coherence of MFLs classroom research will join us in arguing for the development and

consolidation of the scheme, so that these vital features are not lost in future research activity.

References

Bachman, L E and Palmer A 1996 Language Testing in Practice: Designing and developing useful language tests. Oxford University Press.

Cohen, L, Manion, L and Morrison, K 2000 Research Methods in Education (5th Edition). Routledge Falmer.

Creemers, B P M 1994 The Effective Classroom. Cassell.

DfEE 2000 Best Practice Research Scholarships: Guidance notes for teacher applications. Department for Education and Employment.

DfEE 2001 Best Practice Research Scholarships: Guidance for applicants. Department for Education and Employment.

Fleming, F 1997 The Second Foreign Language: An appraisal of its position in a secondary school. MAEd dissertation, University of Southampton.

Harris, A 1998 ‘Effective teaching: a review of the literature.’ School Leadership and Management 18(2): 169-83.

Hogg, I 1998 Marrying Form and Function: A place for grammar and total target language in the secondary MFLs classroom. CLE Occasional Paper 52, University of Southampton.

Macrory, G and Stone, V 2000 ‘Pupil progress in the acquisition of the perfect tense in French: the relationship between knowledge and use.’ Language Teaching Research 4(1): 55-82.

Marsden E 2001 ‘Exploring achievement data from learners of French’. Unpublished coursework assignment, RGSE/ University of Southampton.

18

Mitchell, R 2000 ‘Applied linguistics and evidence-based classroom practice: the case of foreign language grammar pedagogy.’ Applied Linguistics 21(3): 281-303.

Munn, P and Drever, E 1995 Using Questionnaires in Small Scale Research: A teacher’s guide. Scottish Council for Research in Education.

Nunan, D 1989 Understanding Language Classrooms: A guide for teacher initiated action. Prentice Hall.

Page, E 1999 An Investigation into the Interlanguage of Learners of French as a Foreign Language in a Classroom Context. MA Dissertation, University of Southampton.

Rule, S 2001 A Cross-Sectional Study of French Interlanguage Development in an Instructional Setting. PhD Dissertation, University of Southampton.

Scheerens, J 1992 Effective Schooling: Research, theory and practice. Cassell.

19