Evidence-based teaching: advancing capability and capacity for enquiry in schoolsCase study

April 2017

Dr Joanne Pearson

Teamworks Teaching School Alliance

ContentsBackground and methodology........................................................................................................... 3

Time and personal capabilities..................................................................................................................4Relationships and trust..............................................................................................................................4

Defining a research broker.............................................................................................................................5Methodology.............................................................................................................................................7Ethical issues.............................................................................................................................................7Summary...................................................................................................................................................7

Data collection and findings................................................................................................................ 9Quantitative data...........................................................................................................................................9

Number of Alliance Schools involved in January 2015.............................................................................10Number of Alliance Schools involved in October 2015.............................................................................10Number of Teachers involved Research (types 2 and 3)..........................................................................10Numbers of schools external to the TSA involved in type 2 research engagement..................................10

Qualitative data...........................................................................................................................................12Section 1..................................................................................................................................................12Section 2..................................................................................................................................................13Section 3..................................................................................................................................................14Section 4..................................................................................................................................................14

Discussion and recommendations................................................................................................. 15Conclusions..................................................................................................................................................15

Next steps................................................................................................................................................16Recommendations beyond the TSA.........................................................................................................17

References.............................................................................................................................................. 18

2

Background and methodology

Teamworks is a Teaching School Alliance made up of two primary Teaching Schools and, at the time of the project commencement, three primary, one secondary, one special school and two HEIs as strategic partners. The alliance has been working to develop their engagement with research since designation in 2013 but up until this point the main involvement had been to recruit schools to participate in the Closing the Gap project run by CFBT. In total four partner schools participated in the projects; three as active partners, one was placed in the control group and did not choose to undertake the intervention at the end of the period. The aim of this project was therefore to: increase levels of engagement with research from this initial base; to increase both the number of schools involved and to broaden the type of engagement in which schools were involved.

For the purposes of the project practitioner engagement with research was considered as being one of three types:

1. Schools engaging with existing published research to develop and inform practices. (For example using the Education Endowment Fund (EEF) toolkit or reading research journals to develop ITE programmes or adapt classroom practice)

2. Schools engaging to support the primary research of external organisations. (For example participating as control or intervention schools in randomised control trials or providing data for research project ran by Higher Education Institutions)

3. Schools engaging in their own primary research. (For example identifying research questions and designing research protocols)

As a Teaching School the idea that teachers should engage with research is taken as a given indeed Research and Development is one of the targeted ‘Big Six’ areas in which TSA’s should be leading practice. Improved student outcomes and teacher quality closely correlate with teachers’ research literacy and practice (BERA/RSA 2014). The issue of teacher research was raised by Stenhouse (1981) with his vision of the ‘teacher as researcher’, exploring and identifying issues in teaching and more recently the idea of a medical model of research as ‘solving problem’ and finding ‘what works’ (Goldacre, 2013). Whilst the two approaches have significant differing emphases, both do raise and promote the importance of teachers and schools engaging with research.

There are however key issues that have been identified as constraining factors on teachers’ abilities to engage effectively:

The fragmentation of initial teacher education in England and the decrease in University involvement (Beauchamp, Clarke, Hulme and Murray, 2014)

3

The lack of ‘the basic structures needed to support evidence based practice’ (Goldacre, 2013 p.15)

Time and personal capabilities (Leat, Lofthouse and Reid, 2014)

Relationships and trust (Timperley and Parr, 2007)

It is therefore apparent that whilst research informed schools are both desirable and effective in developing teachers and improving outcomes for children, it is neither a simple process to embed research effectively in schools nor an automatically positive experience for staff involved.

The challenge for the project was to find ways in which to increase the research involvement within schools across the TSA by tackling some of the identified issues raised by prior research through creating structures to reduce or mediate these tensions. The first two challenges identified lie at a national policy level so the project design focused on tackling the latter two bullet points: time and personal capabilities and relationships and trust.

Time and personal capabilities

In an era characterised by change and pressures on workload engaging in either reading about or undertaking primary research can appear to be a ‘luxury’ to staff (DfE: Workload Challenge Survey, 2015). This is particularly the case where individual teachers are engaged as sole research practitioners within their school. Collaborative approaches, where teachers have the opportunity to talk through and share their findings and thoughts with colleagues within the same setting, as well as with those outside the setting appear to be more successful (Leat, Lofthouse and Reid, 2014). The issue around personal capabilities is teachers’ ability to find research; most do not have access to peer reviewed journals. In addition if they have not undertaken a research based masters programme, teachers may struggle to both critically interpret research data and to design research methodologies and analytical tools that allow them to undertake meaningful primary research.

Relationships and trust

Leat et al (2014) argue that there are few examples of bottom up innovation in schools, where the agenda for school development has been led by classroom practitioners’ questions and research findings. They further argue that school control over professional development can be seen as a proxy for state control through centralised policies. This issue has meant, for some staff, that their experience of engaging with research was negative, especially where ‘real or apparent criticism of current policies or practices’ occurred as a result (BERA/RSA, 2014, p.18).

4

The project design therefore had to attempt to provide staff with: time, personalised support with finding, interpreting and designing research; relationships, both internal and external and support with navigating school systems. Our research question became:

Can a dedicated research broker support research engagement in schools?The aim of the research project was to support the use of evidence in practice through the use of a dedicated research broker (RB) to connect teachers to relevant published research and support their capacity to access funding to develop their own research skills though practitioner enquiries.

The project’s objectives were to:

develop research champions in each school

connect staff with research relevant to their setting and practice

support staff to engage with their own research by connecting them to funding streams and supporting methodologies.

build research knowledge and capacity across a range of teachers and schools

develop effective dissemination tools to share findings within and beyond the participating schools.

And the planned outcomes were:

to develop teachers’ confidence and ability in engaging with and undertaking educational research

to develop a platform for the effective sharing of knowledge about teaching and learning practices across schools.

Defining a research brokerThe role had the following remit:

to maintain an engagement with external research findings and target staff from across the Alliance with specific pieces of research

to identify opportunities for schools to become involved in external research projects

to identify sources of external funding streams to support staff to undertake primary research

to create small research teams interested in submitting research bids

5

to support the team with the writing of research bids

to provide on-going support with research design and data analysis for on-going primary research

to support the dissemination of research findings beyond the school into the wider alliance and educational community.

The funding was therefore used to support staff time from the Head of the Teaching School Alliance who has experience in both educational research and supporting teacher professional learning to undertake this role.

The role was allocated 38 days across the academic year to allow the remit to be met. It became very evident that the majority of time would be taken to develop type 3 research (research designed and completed by staff in school) as this was the area in which schools had the least experience and confidence. The flow chart below shows the typical pattern of engagement across the project.

The majority of the time was allocated to the preparation of the initial bid with the group of interested teachers. This was to ensure that:

we had enough interest and staff time to ensure that the project could be achieved

there were real benefits both for the children, staff and schools involved but also for wider professional knowledge

leadership within the schools of the staff involved were supportive of their time and involvement in the project (project team meetings were held during the day and after school)

6

RB supports group to disseminate findings

Project group established to manage bid. RB supports and takes on QA role

Bid is submitted

RB works with the group around methodology

RB identifies pre-existing research to inform the bid

Initial meeting is set up to establish the bid

Expressions of interest in the project are sent to staff relevant to the areas via email

RB identifies funding opportunties relevant to the alliance

there was robust scrutiny of the data collection and methodologies being adopted. Where it was felt that we needed additional support this was sought it from partner HEIs

Methodology

The project was evaluated through two main sources of data:

Quantitative data. The numbers of staff involved in primary research through working with external providers or through undertaking primary research of their own. Benchmarking data was taken at the start of the project and on a termly basis throughout the project

Qualitative data. Questionnaires of teaching staff involved in the strategic partner schools to assess their: level of active engagement in research; their perceptions of the role of the research broker role and its impact

The gathering of the quantitative data allowed us to track the number of staff who both expressed an interest in developing research projects and those who were able to successfully undertake a project. The limitations of this purely quantitative approach however are that it alone cannot evaluate the impact of being involved on the staff, their perceptions and indeed whether some of the limiting factors above were mitigated by having a research broker. This data will however be gathered through the use of questionnaires with both closed Likert questions and open ended questions that invite a personalised response. The design of the questionnaires was informed by the guide given by Cohen, Manion, Morrison and Bell (2011, p.379). Each questionnaire was piloted with one teacher to check for issues such as wording and the interpretation of questions then sent to three distinct groups:

Those directly involved in primary research projects. Each member of staff identified through the quantitative data was sent a copy of the relevant questionnaire

A sample of 5 staff from across each partnership school

Ethical issues

Staff were asked to consent to their data being used as part of completing the questionnaire. Staff were identified in each questionnaire in order to ascertain with which project they had been involved, however all reported staff data was anonymised for the purposes of reporting the findings of the study.

Summary

The key question that the project investigated was:

7

can a dedicated research broker support research engagement across a teaching school alliance? The sub questions were:

Can a research broker raise the number of staff involved directly in primary research?

Can a research broker develop staff perceptions of the role and relevance of research to their daily practice?

8

Data collection and findings

Quantitative dataQuantitative data was chosen as the first research approach; for the role of a research broker to be deemed effective it is important that numbers of staff interacting directly with primary research increased within the alliance; i.e. engagement types 2 and 3. This data is also not self-reported but can be verified and therefore has increased reliability over self-reported data on attitudes and behaviours towards research within the classroom i.e. engagement type 1.

The numbers of schools involved in primary research within the alliance was monitored across a ten-month period from January 2015 until October 2015. The involvement was categorised as either type 2 (external research) or type 3 (their own primary research) and an additional category detailing the level of their involvement was also noted for type 3 research: involved in the preparation and writing of a research bid/project; involved in the delivery of a research bid/project; involved in the dissemination of a research bid/project. Each separate project was counted as a being a new involvement so some schools are counted twice if they participated in more than one research project.

The research projects that were prepared and submitted for external funding were:

Closing the Gap Test and Learn. Funding granted. Type 2 research January –June 2015. Completed

Early Years connecting schools and PVI settings to develop outcomes. Funding granted. Type 3 research January 2015- March 2016

Using small scale randomised controls in education. Funding granted. Type 3 February 2015- October 2015

Phonics improving outcomes using Systematic synthetic phonics. Funding not granted. June 2015 not begun funding not yet secured

EEF Effective deployment of Teaching Assistants. Funding granted. September 2015- September 2016

9

Number of Alliance Schools involved in January 2015

January 2015 Type 2 Type 3Preparation 0Delivery 3 0Dissemination 0

Table 1: Number of alliance schools involved in January 2015

Number of Alliance Schools involved in October 2015

October 2015 Type 2 Type 3Preparation 9Delivery 0 12Dissemination 4

Table 2: Number of alliance schools involved in October 2015

Number of Teachers involved Research (types 2 and 3)

January 2015 January 2015 October 2015Preparation 11Delivery 4 10Dissemination 2

Table 3: Number of teachers involved research (types 2 and 3)

In addition two of the projects (the Early Years project and the EEF project) were written to work with schools and settings outside of the alliance. These schools fall into category 2 of the research descriptors. They were participants in the primary research of others but were not part of the design, delivery and dissemination of the projects.

Numbers of schools external to the TSA involved in type 2 research engagement

Schools external to the TSA

January 2015 1October 2015 64

Table 4: Number of schools external to TSA involved in type 2 research

10

In terms of quantitative data the figures suggest that there was a positive impact from the research broker. The numbers of schools running type 3 research projects grew from a baseline of 0 to 12 (with 1 research project still to be funded in October 2015). Whilst engagement in type 2 research fell within the alliance over the same period the numbers of schools outside the alliance engaging with type 2 research grew from 1 to 64 during the same period.

As table 2 indicates the most common way schools are involved with research is in the planning and delivery of the project. Once funding opportunities had been identified by the research broker numbers of staff across a range of schools helped to prepare the proposal and then carried out the primary data collection once the bid had been successful. For example the EYFS bid involved a group of five schools sending one or more members of staff to a small working party to prepare a bid which would support staff in Private and Voluntary (PVI) settings to develop provision to meet the needs of the most deprived children. This meant reflecting on their own professional knowledge; looking for existing research that suggested practice that had made significant impacts and creating protocols that would allow this practice to be disseminated and evaluated beyond their own setting. Once the bid was successful the same staff continued to implement and evaluate the project within their own setting and to train and support staff in the settings beyond the alliance (in this case 10 PVIs). Whilst the project has not been completed and is being internally evaluated for its impact on pupils we have also commissioned an external evaluation from our local HEI for its impact on the staff in the PVI setting.

The dissemination of the research is the area that is currently showing lowest engagement levels. This may be because most projects are on-going and have not yet reached the point of needed to be written up but may also be because teachers feel a lack of expertise or time to be able to undertake this element fully. This area will require further data collection at a later date to be able to fully understand why the level of engagement falls off at this point.

One project, based around effective practices in phonics, has been unsuccessful in terms of attracting funding. This has meant that, whilst the school/research broker has written the project and the protocols, the project itself has not been begun. This points to a possible deficit in the research broker model; if the bids are not successful in finding funding then they may never get beyond the planning phase.

Qualitative dataIn order to assess the impact of being involved in all three forms of research identified a qualitative data collection was also undertaken between July and September 2015. This was done with two groups of staff: those who had been directly involved in one of the

11

research bids listed above and one in alliance schools but with staff who had not been part of the projects directly. This was to assess:

The impact of any direct involvement with research on the teachers involved

The impact of any research in their schools on those staff who were not directly involved

The data was collected through the means of a single questionnaire designed with a mixed of closed response and open questions. The questionnaire was split into four sections:

Section 1. The background of the teacher; highest qualification, length of service

Section 2. The teacher’s view on research in general

Section 3. Their experience of research within their setting

Section 4. Their own direct experience of research

The questionnaire was adapted from a prior study (Borg 2006) into the research engagement of teachers within English language teaching. This allowed our findings to be placed within something of a wider context of prior research.

The questionnaire was sent out via Headteacher gatekeepers within each school and sixteen teachers responded from across four schools. Six of the sample came from staff that had been directly supported by the research broker (group A) and ten returns were from staff that had not been part of the project but were employed in the same schools (group B). The two groups were analysed and their findings compared.

Section 1

The background of the teachers were as follows:Group A Group B

Highest qualification 5 (BA/BSc/Bed)1 masters

5 (BA/BSc/Bed)3 PGCE1 masters1 PhD/Ed

Length of service 19 years (mean)range: 1-30 years

14 years (mean)range: 1-38 years

Mean number of schools employed

2 6

The majority if the staff in the sample had had no additional academic qualification since the completion of their Initial Teacher Education; there were no significant differences in

12

this regard between the group of teachers involved in the research project and those outside. The mean length of service in Group A was longer but the mean number of schools in which they had been employed was significantly less than that of Group B. This sample of teachers was broad in both groups in terms of the range of time in the classroom and the range of schools in which they had employment experience.

Section 2

In this section the two groups were asked to assess the level of importance of research characteristics. There was much greater variability in response from Group B than Group A but both groups had the greatest agreement that the objectivity of the researcher was the most important feature overall with 100% of Group A identifying it as important; Group B had the most agreement on this feature but only 50% identified it as the most important with 40% identifying it as less important. The two groups had similar opinions on questions that explored methods, sample size and generalisability of findings; for example ‘a variety of methods is used’ Group A 50% rate as more/most important and 40% of Group B gave this the same rating.

Similarly Group A all identified the topic being of interest to teachers and the results giving teachers ideas they can use as being of more/most importance however in Group B 60% of staff rated the topic being of interest at moderate to no importance. They were more positive about the research being of practical use to teachers with 60% identifying it as more/most important but 40% also rated this as moderate to no importance. 83% of Group A felt research conducted by practising teachers was important but 80% of group B rated this as of moderate to no importance.

Group A were far more likely to see all the categories as having importance than Group B; Group A gave no returns against any of the categories as unimportant, Group B had 12 responses in this category.

Overall Group A’s lowest ratings were around the size of the sample and the variety of methods used whereas Group B’s lowest ratings were for the researchers being practising teachers and the topic being of interest to teachers.

Section 3

This section explored the place of research within their own school. In Group A 100% of the teachers felt they were encouraged to do research in school, for Group B this figure fell to 70% with 30% of staff unsure or disagreeing. Group A felt that teachers talk about research in their setting (100%) and again this was high in Group B (80%). Time allocated to research within schools was the area both groups highlighted as an issue; they disagreed that there was time built into the school year to undertake research (Group A 66%, Group B 40%) but Group A highlighted this issue more than Group B. In

13

both groups there was a split between access to books and journals with 50% in each feeling they had this and 50% disagreeing.

Section 4

The majority of staff in Group A had been involved in a research proposal with most citing a project facilitated by the research broker as the example. In contrast 90% of Group B had had no involvement in research in the previous two year period. Those in Group B who had been involved in research had done so as part of a course requirement (their undergraduate dissertation and NPQML projects were cited as examples). A majority of both groups however had read published research however with 80% of Group B and 84% of Group A answering yes to this question. The most common reason in both groups for undertaking the research was to find better ways of teaching (100% Group A and 80% Group B). The two groups also agreed that on the lack of expectation regarding research from school senior leaders; only 10% of Group B and 33% of Group A identified senior leadership support as a key factor in undertaking research.

The barriers to developing research identified in the survey were time (66% Group A, 30% Group B) and expertise (33 % Group A and 40% Group B). Group B felt that their colleagues in school do not do research (40%) whereas only 16% of Group A felt this was the case. None of the staff involved felt that employers thought research was unimportant but one teacher (Group A) did raise the issue that Ofsted set the agenda for school priorities more than anything else.

14

Discussion and recommendations

ConclusionsThe purpose of this project was to tackle what Helmsley-Brown and Sharp (2003) identify as one of the main barriers to teacher engagement in research, the ‘institutionalised culture that does not foster learning’ (p.46) by providing leadership at an institutional level in the form of a research broker. The specific aims of the project were to:

develop research champions in each school

connect staff with research relevant to their setting and practice

support staff to engage with their own research by connecting them to funding streams and supporting methodologies

build research knowledge and capacity across a range of teachers and schools

develop effective dissemination tools to share findings within and beyond the participating schools.

The quantitative data suggests that the project has been successful in increasing the number of schools and teachers engaged in research with greater numbers involved in both type 2 research, participating in projects designed and run externally and type 3 research, participating in projects designed and implemented internally. To this end there has been some progress made against the aims listed above. Staff within the project are connected to relevant research; more were involved in primacy research and reading published reports than in the staff that were not part of the project. However this finding also shows that the first and last aims of the project are still some way from being achieved. Whilst the staff involved in the research may be building their own capacity this is clearly not yet being disseminated to their colleagues; 40 % of staff Group B, working in the same schools as those in Group A, believe their colleagues are not engaged in research despite the presence of project work within the school. It is therefore difficult to argue at this stage that the staff in Group A are ‘research champions’ in the sense of raising the profile of research within their own schools.

Staff who worked with the research broker were far more likely to see the benefits and applications of research in the classroom than those who did not. As shown in section 2, 83% staff in group A felt that an important feature of research was the involvement of practising teachers and 84% felt research conducted in real classrooms was highly important in contrast to 30% in Group B. This latter result correlates with Borg’s (2006) findings in which his respondents placed real classrooms at the lower end of importance with 51% rating it as important. It does therefore seem that the experience of

15

undertaking primary research for themselves increases the likelihood of teachers valuing research being done in classrooms by practitioners; the correlation between the group of teachers here was stronger than their level of qualification, length of experience or number of schools suggesting that working with the research broker may have been the most important factor in this finding.

The survey data also matches previous findings in the drivers and barriers to teachers undertaking research. Both groups identify time as an important barrier to research as did Borg’s (2006) sample but Group B were more likely to identify expertise and support as things holding them back than Group A suggesting that the confidence of staff who have been working with the research broker is greater. Worrall (2004) and Barker (2005) also found that a lack of external pressure and personal dispositions were factors holding back the research agenda; for the former both Group A and Group B’s findings suggest that there remains a lack of pressure. None of group B identified senior leader expectations as a factor in undertaking research and only 33% of Group A reported senior leader expectations as a driving force and whilst this is more positive it still is a much smaller driver than the desire to ‘find better ways of teaching’ which was the most important driving factor in both Groups A and B and in Borg’s (2006) study.

The research questions were: Can a dedicated research broker support research engagement across a teaching school alliance?

Can a research broker raise the number of staff involved directly in primary research?

Can a research broker develop staff perceptions of the role and relevance of research to their daily practice?

The data suggests that the role of the research broker did have a positive impact in the research engagement in the TSA. The numbers of staff and schools involved in type 2 and 3 research did grow over the project’s lifetime and for those staff that worked with the research broker their perceptions of the role of classrooms and teacher was more positive than other staff. They had greater access to reading, felt more confident around methodology and felt more encouraged to undertake research than their colleagues.

Next steps

There are still aspects of the project however that are not as strong. The staff in Group A have not yet begun to act as champions for this approach with their colleagues in schools; we have not yet disseminated enough of their own findings or the ways in which the projects were undertaken and this remains part of the ongoing action plan for the TSA.

Even with the allocation of a research broker to support staff there remains a perceived issue with time and workload in Group A that, while less than that of Group B, has not

16

been eradicated as part of this project. The successful allocation of funds has played a key part in releasing staff and it should also be recognised that where funding was not found; i.e. the phonics project, the research has not yet been begun. As a result the TSA has allocated a budget of £4,000 for this current academic year to allow staff and schools within the alliance to bid for small pots of money to support research going forward.

The research broker has been one step towards altering the ‘institutionalised culture’ but there is clearly some work remaining to be done with senior leaders in school. Most staff across both groups did not feel that there was an expectation from senior leaders that they engage with research to support practice. The reasons for this were not part of the remit of the questionnaire but in additional comments some staff suggested that both the ‘Ofsted agenda’ and the lack of research experience/understanding within senior leadership teams may be factors in their reluctance to promote research the performance management process or within job descriptions.

Recommendations beyond the TSA

Our findings would suggest that the process of allocating a research broker may have something positive to offer other settings in increasing staff engagement in research. There are however some aspects that would have to be in place for this to happen:

The time allocated to the research broker role was significant across the year (38 days or 0.2 of a full time contract in term time). The research broker undertook the bulk of the work in finding funding, pulling together a team of interested staff, supporting the writing of the bid and finding existing research. We suggest this is a strong factor in the success in gaining funding and seeing the projects through from the planning to the operational level.

The research broker needs to have some training or prior background in research and access to academic journals is a key benefit. This saved considerable time when pulling together bids and helped to increase the confidence of staff in Group A that they were going to be able to carry out the projects successfully.

Systems to support the staff in Group A to act as research champions within their own schools need to be put in place if impact is to be seen in the whole school setting beyond the life of the project in which they are involved.

17

References

Barker, P. (2005) Research in Schools and Colleges. National Educational research Forum Working paper 7.2

Beauchamp,G., Clarke, L., Hulme, M., and Murray, J. , (2014) Policy and Practice within the United Kingdom. BERA (online) (accessed February 6th 2015) BERA online

BERA/RSA, (2014) Research and the Teaching profession. Building the capacity for a self-improving education system. London: BERA

Borg, S. (2006) ‘Research engagement in English language teaching’ Teaching and Teacher Education 23 pp 731-747

Cohen, L., Manion, L. and Morrison, K. (2011) Research Methods in Education. London: Routledge

Gibson, S., Oliver, L., Dennison, M. (2015) Workload Challenge: Analysis of teacher consultation responses London : CooperGibson Research/DFE

Goldacre, B. (2013) Building evidence into education. Bad Science (online) accessed on February 6th 2015) Bad Science online

Hemsley-Brown, J., & Sharp, C. (2003). ‘The use of research to improve professional practice: A systematic review of the literature.’ Oxford Review Of Education, 29, 449–470.

Leat, Lofthouse and Reid, 2014 ‘Teachers’ views: perspectives on research engagement’ Oxford Review of Education, 41 (2), pp 270-286

Stenhouse, L. (1981) ‘What counts as research?’ British Journal of Educational Studies, 29 (2), pp. 103-114.

Timperley, H.S. & Parr, J.M. (2007) ‘Closing the Achievement Gap through Evidence-based Inquiry at Multiple Levels.’ Journal of Advanced Academics, 19(1), 99-115.

Worrall N. (2004) ‘Trying to build a research culture in a school: Trying to find the right questions to ask.’ Teacher Development, 8, 137-148

18

Crown copyright 2017

The views expressed in this report are the authors’ and do not necessarily reflect those of the Department for Education.

Any enquiries regarding this publication should be sent to us at: college.research@education.gov.uk or www.education.gov.uk/contactus

19