ResearchEDTime for a reality check?
Robert CoeResearchED Research Leads Network Day, 13 December 2014
∂
ResearchED
Engagement with educational research– “ …the start of something wonderful”– Or just another fashion?
And what does it mean anyway? What is a reasonable hope/expectation for
research and evidence to make a difference to practice?
2
∂
Dimensions of great teaching
1. Quality of instruction
2. (Pedagogical) content knowledge
3. Classroom management / behaviour / control
4. Classroom climate / relationships / expectations
5. Beliefs (theory) about subject, learning & teaching
6. Wider professional elements: collegiality, PD, stakeholder relationships
3
RESEARCH?
∂
The problem with ‘What works?’
What you want to know:What will work for me
What research/evidence tells you:Something seems to have ‘worked’ for researchers
4
Research is always equivoca
l
Is it clear what it is?
Past tense
Does it transfer to ‘real life’ in
my context?
What outcome
s are used /
valued?
∂
A choice between two or more approaches that are– Well-defined (clear exactly what you must do – even if
one approach is ‘continue normal practice’)– Feasible (cost, time, acceptability, ethics)– Replicable (anyone who wants to can do them)
A context that is generalisable – The choice and context have to be similar enough to
those faced by others
Agreement about criteria for success– What outcome matters, and how measured?
5
Requirements for evidence-based practice
∂
Requirements for evidence-based practiceAre the same as for prediction or giving advice
Teacher: I want my pupils to learn to read. Should I use synthetic phonics or real books?
Researcher: Each context is unique, research can’t tell you that
Teacher: OK, should I give them books at all, or just do dancing?
Researcher: Each context is unique, research can’t tell you that
6
∂
Why might research help (1) Knowledge of research
Some pedagogical practices are better supported by evidence than others– Aim for what is actually effective, not just fashionable
Some interventions are better supported by evidence than others– Go with the best bets, based on evidence
Evidence can help in challenging & developing teachers’ existing theories and beliefs about learning– Effective professional learning requires this
(Timperley, et al, 2007)
7
∂
Should we challenge teacher beliefs?
Beliefs about– Why some students do not learn
• Ability, social background, effort, motivation• Their fault vs my fault
– What does it mean to learn (mathematics)• Connectionist / Transmission / Discovery
orientations (Askew et al, 1997)
– How does feedback work• Correction vs accommodation
8
∂
Impact vs cost
Cost per pupil
Eff
ect
Siz
e (
mon
ths
gain
)
£00
8
£1000
Meta-cognitive
Peer tutoringEarly Years
1-1 tuitionHomework (Secondary)
Mentoring
Summer schools After
school
AspirationsPerformance pay
Teaching assistants
Smaller classes
Ability grouping
Most promising for raising attainment
May be worth it
Small effects /
high cost
Feedback
Phonics
Homework (Primary)
CollaborativeSmall gp
tuition Parental involvement
Individualised learning
ICT
Behaviour
Social
www.educationendowmentfoundation.org.uk/toolkit
∂
Why might research help (2)
Research Mindset
Monitoring ongoing effectiveness– “Know thy impact” (Hattie)
Evaluating impact of changes– What is working for you
Critical perspective– Show me the evidence– Devil’s Advocate– “Tolerance of ambiguity” (Sherrington)
10
∂
‘Distributed Research’“Central to the philosophy is the concept of ‘distributed research’, the idea that the recipients of the feedback (i.e. teachers in schools and colleges) are themselves active researchers in the process, analysing and interpreting the data, rather than simply passive recipients. The research is seen as a collaborative process. The CEM Centre has skills in test design, the creation of monitoring systems and takes advantage of economies of scale in its management of the process. It also provides the basic structure of the analysis, using the collective dataset. After that, however, interpretation is a local matter and cannot be done from the centre, though considerable support is offered to help schools and colleges to interpret the data for themselves. The important thing is that the data are fed back to the smallest unit of management that can take responsibility for acting on the feedback (Fitz-Gibbon, 1996, p. 161).”
11
Tymms & Coe, 2003
∂
Clear, well defined, replicable intervention
Good assessment of appropriate outcomes
Well-matched comparison group
EEF DIY
Evaluatio
n Guide
Key elements of good evaluation
What could
you evaluate?
∂
Do we know a good lesson when we see one?
13
∂
Lesson Observation
1. Two teachers observe the same lesson, one rates it ‘Inadequate’. What is the probability the other will agree?
a) 10% b) 40% c) 60% d) 80%
2. An observer judges a lesson ‘Outstanding’. What is the probability that pupils are really making sustained, outstanding progress?
a) 5% b) 30% c) 50% d) 70%
14
www.cem.org/blog
∂
‘Improvement’ often isn’t School/college would have improved anyway
– Volunteers/enthusiasts improve: misattributed to intervention– Chance variation (esp. if start low)
Poor outcome measures– Perceptions of those who worked hard at it– No robust assessment of pupil learning
Poor evaluation designs– Weak evaluations more likely to show positive results – Improved intake mistaken for impact of intervention
Selective reporting– Dredging for anything positive (within a study)– Only success is publicised
(Coe, 2009, 2013)
Top Related