RESEARCH AND EVALUATION

Research and evaluation 3

3

Research and evaluation 2

2

The work of the Centre for Evaluation and Monitoring (CEM) is rooted within the tradition of rigorous research. Within CEM we have a dedicated group of experts in evaluation and support staff who make up the CEM Research and Evaluation (CRE) team. Our vision is to help educators to improve educational outcomes by creating world leading assessments, empowering teachers, and promoting evidence-based practices and policy. As well as conducting our own research, CEM’s Research and Evaluation team regularly work with education organisations across the world on a variety of projects. These range from one-off specialist reports, to substantial research and evaluation commissions that span several years and involve large-scale data collection and processing. As one of the largest educational research units in a UK university, CEM publishes a range of papers and reports that aim to improve pupils’ outcomes.

Research and Evaluation ExpertiseCEM promotes Evidence-Based Education (EBE) and believes that educational policy and practice should be informed by the best possible evidence about the likely effects of different choices on outcomes of importance. This means that specific intervention strategies and policies should be rigorously evaluated before they are advocated and/or required. Where this is not possible, interventions can be ‘adopted’ on an experimental basis so that their impact can be evaluated.

CEM has extensive expertise in a wide variety of research and evaluation methods.

• RCTs - Examples of CEM’s work on RCTs include: Every Child Counts, Fife Peer Tutoring, Durham Shared Maths, Screening and Interventions for Inattentive, Hyperactive and Impulsive Young Children

Research and evaluation

• Pilot and feasibility studies - CEM is currently running a pilot study evaluating the Innovation Unit’s development of a Project Based Learning intervention, and also a pilot evaluation of the ‘SHINE in Secondaries’ intervention.

• Qualitative and process evaluations

CEM also has extensive expertise in widening participation (WP) programmes which derives from both evaluating WP programmes and reviewing existing evidence. For example, at present we are collecting data and evaluating a wide range of existing WP programmes including: Sutton Trust UK Summer Schools (including e-mentoring); Pathways to Law; Teachers Summer Schools; UCL Sutton Scholars project; and the Kent Academies Trust WP project.

• Regression Discontinuity Design (RDD) - In conjunction with Professor Carole Torgerson (School of Education, Durham University) and Dr Adetayo Kasim (Wolfson Research Institute for Health and Wellbeing, Durham University), CEM is applying the RDD methodology within education research.

• Quasi experimental design – When RCTs are not feasible, these techniques have been used with statistical modeling for projects such as the Sutton Trust Effects of Selective Education Project.

• Data tracking - CEM is currently developing an evaluation framework for the Sutton Trust which is being used to monitor and evaluate a wide range of university access/widening participation programmes.

Research and evaluation 4 Research and evaluation 5

4 5

Access GrammarThis project seeks to look at ways access to grammar schools for highly able children from non-privileged backgrounds can be improved. The project will identify potential target cohorts in the study areas for a range of outreach interventions and will look to evaluate these activities. For this project, the CEM Research and Evaluation team are working in collaboration with the Sutton Trust, Grammar School Heads Association, King Edwards Foundation and the Department for Education.

Current research and evaluations

Embedding Research-Based Interventions for Improving Literacy into Teachers’ Everyday PracticeThis project will evaluate a series of approaches designed to encourage teachers to incorporate research-based advice and information from pupils’ assessments (InCAS) into their everyday practice by assessing the impact on pupils’ educational outcomes at Key Stage 2. Within the context of a randomised control trial, teachers will receive one of three levels of encouragement to incorporate research-based advice into their everyday practice, including an advice booklet, twilight group discussion sessions, peer observations and pupil assessment data. This project is funded by the Education Endowment Foundation (EEF).

Hallè SHINE on Manchester EvaluationThe Hallè SHINE on Manchester intervention is an academic and cultural Saturday school programme targeting disadvantaged pupils in primary schools in the Manchester area and is funded by the EEF and SHINE Trust. Together with the School of Education at Durham University, the CRE team have been commissioned by the EEF to evaluate the programme to find out how well it improves maths, literacy and attitudes to music for children invited to participate, using a clustered randomised controlled trial.

SHINE in SecondariesThis is an independent pilot evaluation of the impact of the SHINE Saturday School programme for underachieving Year 7 pupils which uses

regression discontinuity design. The primary outcome is literacy, with secondary outcomes of mathematics and attitudes.

Impact of School Inspections on Teaching and Learning:This is a European Commission funded research project as part of the Lifelong Learning Programme and involves working alongside partners from the Netherlands, Ireland, Austria, Czech Republic, Sweden, Norway and Switzerland. The project primarily makes use of survey data from head teachers and teachers from a sample of schools that received a school inspection in 2009/10. For further information on this project, please visit: http://schoolinspections.eu/

Research and evaluation 6 Research and evaluation 7

6 7

Project Based LearningProject Based Learning (PBL) involve students taking a structured ‘project’ approach to learning, with a start and an end, and working in a co-operative and cross-curricular way. Although PBL approaches have been used in school teachings for many years, its success and impact on educational outcomes has not been evaluated in the context of the English education system. CEM, in collaboration with York Trials Unit, are conducting an independent evaluation of a PBL intervention. The first year includes a pilot study, where CEM will advise on the development of the intervention and in the second and third years, the impact of the intervention on attainment and engagement in school will be evaluated through a randomised controlled trial. The PBL project is funded by the EEF.

Shared MathsThis project aims to improve mathematics attainment in primary schools via cross-age peer tutoring which involves older pupils tutoring younger pupils in the classroom. Evidence shows that by working together using the Shared Maths process, children can understand maths more fully and remember what they have learned more easily. The project is funded by the Education Endowment Foundation and is evaluating the impact of Shared Maths when rolled out in four local authorities through a randomised controlled trial that involved 82 schools. More information on the Shared Maths project can be found at: http://www.sharedmaths.org/

The Lions ProjectLions is a nursery based in China, providing bespoke programmes for families which aims to enhance child development and life accomplishment. Specialist teams of Lions’ staff design, enable and administer learning to take place in the family home. Lions assess, monitor and report child development via an in-house system. CEM is evaluating this assessment.

The Sutton Trust Programme Evaluation FrameworkThe Sutton Trust develop and fund a wide range of programmes with the aim of increasing participation in higher education for young people from less advantaged backgrounds. CEM has been commissioned to develop and operate a Common Evaluation Framework, initially including five separate programmes and some additional initiatives within those programmes. The project involves evaluating outcomes through comparisons of longitudinal data (for example from the National Pupil Database and UCAS), repeated cohort surveys and qualitative observation and interviews.

Research and evaluation 8 Research and evaluation 9

8 9

PROFESSOR ROBERT COE is the Director of CEM. Rob’s research interests include: evaluation methodology; debates around Evidence-Based Education, the nature and justifications of causal claims in educational research; the involvement of practitioners in research; school effectiveness and improvement. He was one of the authors of the publication of the Sutton Trust’s Toolkit (Higgins, Kokotsaki and Coe, 2011) which reviewed the international literature on effective school-based strategies. Rob has expertise in general conventional statistical analysis, interpretation and limitations of statistical hypothesis tests, effect size statistics, meta-analysis, multilevel modelling, item response models, growth models, propensity score matching, bootstrapping and generalisability theory. To find out more about Rob’s work, visit: https://www.dur.ac.uk/directory/profile/?id=614

DR CHRISTINE MERRELL is the Director of Research at CEM, and Reader of Education at Durham University. Christine is an experienced academic and her research into educational assessment, monitoring and interventions, early childhood development and the academic progress of severely inattentive, hyperactive and impulsive young children are considered to be world class. Christine’s expertise has led to membership of Ofqual’s Early Years Advisory Group and membership of the NICE Guideline Development Group for the ADHD Guidelines. Christine coordinates the research within CEM to improve the educational outcomes of children by developing effective assessment and monitoring systems, and evaluating educational interventions. To find out more about Christine’s work, visit: https://www.dur.ac.uk/directory/profile/?id=2582

DR ANDY WIGGINS is Associate Director of Research and Evaluation at the CEM. Andy has been at CEM for about twelve years and has led and contributed to a wide range of educational research and evaluation projects for a variety of organisations - including the Department for Education, Scottish Government, and the National College. Currently Andy is working with the Sutton Trust on the Access Grammar and Programme Evaluation Framework projects, leads the Shared Maths (peer tutoring) project and is a member of the Education Endowment Foundation Evaluation Advisory Board. To find

out more about Andy’s work, visit: https://www.dur.ac.uk/directory/profile/?id=4309

DR HELEN CRAMMAN is Research Contract Manager for CRE. Helen’s research interests include the evaluation of education interventions, career aspirations and gender equality in STEM subjects and science outreach. She is project manager on a range of projects across CEM and sits on CEM’s Executive Committee. To find out more about Helen’s work, visit: https://www.dur.ac.uk/research/directory/staff/?mode=staff&id=3401

KAREN JONES is a Research Associate at CEM. Karen is currently working on an EU-funded project on the ‘Impact of School Inspections on Teaching and Learning’ project. Karen’s research interests are varied, ranging from school inspection and accountability to pupil attitudes in STEM subjects and grammar schools. Karen specialises in the analysis of large datasets and quantitative research design and analysis. To find out more about Karen’s work, visit: https://www.dur.ac.uk/sass/phd/profile/?mode=staff&id=7315

DR BERNARDINE KING is a Reading Researcher specialising in early reading and dyslexia. Bernardine has led randomised controlled trials evaluating pedagogic and technological interventions designed to help struggling readers. Bernardine has audited assessments of general cognitive ability and of Key Stage 3 science. Her own research is into auditory and visual deficits in dyslexia and her publications include her book ‘The Key to Dyslexia?’

DR DIMITRA KOKOTSAKI is a Researcher at CEM and a lecturer at the School of Education at Durham University. Dimitra is currently working on the evaluation of the ‘Restorative Approaches’ initiative in County Durham, the Hallè SHINE on Manchester project, and a project funded by the Nuffield Foundation which aims to improve the primary-secondary transition in music education in the North East of England. Dimitra’s

research interests include the identification and improvement of the educational, behavioural and socio-psychological conditions in schools with a specific focus on pupil creativity and engagement. To find out more about Dimitra’s work, visit: https://www.dur.ac.uk/education/staff/profile/?id=2225

DR JOHN LITTLE has worked as a statistician at Durham University for over 10 years. His specialist areas include Rasch models, simulation and data modelling of complex systems.

ANDREW LYTH is a statistician at CEM. His specialist areas are developing national norms for assessment results, the prediction methodologies used in CEM’s feedback to schools and Rasch theory.

VICTORIA MENZIES is a Researcher and Trial Coordinator at CEM. Victoria is currently coordinating several projects, including the ‘Shared Maths’ project and the evaluations of ‘Project Based Learning’, ‘Hallè SHINE on Manchester’ and ‘Hallè SHINE in Secondaries’. Victoria’s research interests include education evaluation, higher education access, peer tutoring, pilot trial methodology and randomised controlled trials. To find out more about Victoria’s work, visit: https://www.dur.ac.uk/education/staff/profile/?id=10745

DR FRANCIS NDAJI is an experienced member of the CEM Research and Evaluation Group. Francis has been involved in a variety of evaluation contracts and has strong skills in quantitative research methods. Francis also has experience of managing one of CEM’s monitoring systems, producing feedback for participating schools on an annual basis, which utilises his skills in programming and data analysis.

About the CRE team

Research and evaluation 10 Research and evaluation 11

10 11

DR LYN ROBINSON is a Research Associate at CEM. Lyn’s expertise includes the development of novel data collection methods, randomised controlled trials (RCT) and the translation of research results for use in everyday practice. Lyn is currently project managing an RCT funded by the EEF to investigate how research-based interventions to improve literacy can be embedded into teachers’ everyday practice. Lyn’s research interests include investigating how environmental and behavioural factors impact on childhood education, health and wellbeing. To find out more about Lyn’s work, visit: https://www.dur.ac.uk/research/directory/staff/?mode=staff&id=5802

DR HELEN WAREHAM is working as a Research Support Assistant for CEM. Her most recent research experience is in the utilisation and analysis of large secondary datasets. Helen is currently working on projects funded by the Sutton Trust. To find out more about Helen’s work, visit: https://www.dur.ac.uk/school.health/staffmembers/staffprofile/?username=srnw54

KIRSTY YOUNGER is a Research Assistant at CEM, and is currently working on an Evaluation Framework project with the Sutton Trust. Since joining CEM in 2012, Kirsty has also worked on randomised controlled trials and the ‘SHINE in Secondaries’ evaluation. Her interests include the relationship between research and policy-making, research methodology, and high education access interventions.

Areas of specialism and interestCEM Research and Evaluation have a variety of specialisms and areas of interest. These include, but are not exclusive to:

• Classroom-based interventions to help severely inattentive, hyperactive and impulsive young children

• Maths education

• Science education

• Music education

• Widening participation to Higher Education

• Dyslexia

• Comparably of GCSE and A Level examinations

• Grammar schools and selection

• Environmental and behavioural impacts on education, health and wellbeing outcomes

• School transitions

• Analysis of large-scale data sets

• Accountability and school inspection

CEM maintains one of the largest educational data banks in the country; a substantial and invaluable volume of statistical data which has been collected for over two decades from our own in-house monitoring systems.

Collaborate with usWe work collaboratively with departments and groups internal to the University (such as the School of Education, the Wolfson Research Institute for Health and Wellbeing), external organisations, both academic, non-academic and commercial. We are also commissioned by external organisations to conduct evaluations and research, for example; the Scottish Executive, the Institute of Physics, Save the Children, the Wellcome Trust, the Sutton Trust and the Education Endowment Foundation.

EventsWe run a range of research based events at CEM including:

• Conferences – CEM’s first Research Conference – ‘30 years of Evidence in Education’ - will reflect on the current position of evidence-based educational policy in the UK, from an international perspective. The conference will be held on 23rd September 2014, in Central London. For more information, visit: http://tinyurl.com/lzq25t5

• Seminar Series – In association with Durham University’s School of Education, CEM’s Seminar Series allows academics, researchers and PhD students from around the world to present their findings and discuss their research interests. For information on upcoming seminars, visit: https://www.dur.ac.uk/education/research/seminars/

• Research Café – The Research Café is an internal event whereby teams within CEM regularly meet to identify and discuss internal collaborations and the utilisation of existing CEM data.

• Reading Group – CEM’s Reading Group is an internal event, held monthly, where staff come together to discuss topical education literature and research.

If you would like to attend any of CEM’s events, please contact us, details below. Further information on CEM’s events can be found http://www.cem.org/training-and-support/events/eventlist

Contact detailsLead contact:

Dr Helen Cramman (Research Contract Manager)

Email: research@cem.dur.ac.uk

Phone: +44 (0) 191 334 4240

www.cem.org