http://miblsi.cenmi.org

Unpacking Hattie’s Effect Sizes

Laura Colligan & Kim St. Martin November 20, 2014

Gots and Wants DataGots Wants

Background info on Hattie’s synthesis of research (Visible Learning-2009)

Strategies to bring info back to staff (how much, how little, balance) – template, protocol, etc.

Conditions necessary for learning More table talk time / including time to plan to bring this back to staff

Contributing factors for cognitive overload More on the role of inquiry learning and unpacking its effect size

“Store Theory” for learning Role of feedback

Refresher on importance of prior knowledge

Explaining why some effect sizes are so small (retention and class size)

Book and “good reading time” Reading chapter suggestions: readings either done ahead of time (3 wants); people who finish reading first begin conversation by talking quietly until others can finish reading)

Good discussion with colleagues Central office administrators to hear this

3

1.0 Review from previous session

2.0 Unpacking a few effects• Direct Instruction• Problem-based learning

3.0 Planning to bring information to staff• What is the goal?• What is your compelling why?• Action plan

Agenda

4

1.0 Review from Previous Session

5

• Hattie’s synthesis in Visible Learning (2009) was the first of its kind

• Focus on PreK-12

• 138 “innovations”

• 6 Domains: child, curricula, home, teaching, teacher, school

• Used an effect size so we can compare the effects of one innovation to another

The “Barometer”

6

• .40 is the benchmark (average everything together)

• We don’t want to compare the effectiveness of innovations to zero

– If we did, 95-97% of innovations would be classified as working because they are above zero

• “Student maturation” innovation had an effect of .15

– That is considered the cut-off for “disasters”

The “Barometer” (cont.)

Teacher as Activator vs. Facilitator (updated list)

Teacher as Activator Teacher as Facilitator

Teaching students self-verbalization / questioning (.76)

Inductive teaching (.33)

Teacher Clarity (.75) Simulation and gaming (.32)

Reciprocal teaching (.74) Inquiry-based teaching (.31)

Feedback (.74) Smaller classes (.21)

Metacognitive strategies (.67) Individualized instruction (.22)

Direct instruction (.59) Web-based learning (.18)

Mastery learning (.57) Problem-based learning (.15)

Providing worked examples (.57) Discovery methods in math (.11)

Providing goals (.50) Whole language (.06)

Frequent / effects of testing (.46) Student control over learning (.04)

Graphic / Behavioral organizers (.41)

8

We have some understanding about the “what” but why those things?

How Information Gets to Long-Term Memory

Multi-store theory

Iconic (Sensory Memory or ultra

short-term memory)

Short-term memory

(Working memory)

Long-term memory

Things we see and hear lasts between less than a second or up to 3 seconds.

• Work bench of our conscious minds.

• Bench needs to stay active or things fall off.

• Problems:1. Capacity – only

can hold a few things

2. Time which things can fall off (be lost)

• Archival • Time does not impact

LTM• Problems:

1. Difficult to load information to LTM

2. Need for efficient strategies to code new information to existing info

3. Retrieval strategies to easily access info.

10

• Goal is to to get information to long-term memory…but that is challenging

– Challenging due to the limitations of our working memory (short-term memory

• Short term memory is like a workbench– Too many things means they fall off– The things on our workbench need to be active

(our minds have to do something with the information) so they don’t fall off

Unpacking Knowledge Acquisition

11

• Conditions needed for learning

• Factors that impact our ability to remember

• Things that cause us to hit overload

• Good news: – We have control of many of these things– The things harder to directly control (e.g.,

motivation and effort) we can still design lessons in such a way to increase motivation and effort

How to Get Information to LTM

12

Partner 1: You are going to review and then teach to your partner, the “store theory” AND the conditions needed for learning

Partner 2: You are going to review the factors that impact memory retention and sources of overload

• Partners review for a few minutes, jot down key points want to communicate

• 1’s go first

• Partners ask clarifying questions and / or add to any information your

Partner Activity

13

Now that we have an idea about the conditions are minds need to acquire knowledge we will talk about what we can do as educators to align instructional methods with how our minds learn…

14

2.0 Unpacking a Few of Hattie’s Effects

15

Pairing Terminology

Direction instruction, explicit instruction, principles of effective instruction, and fully guided instruction are all terms used interchangeably

Their core features are the same

All terms are used in the literature so pairing is necessary to avoid confusion

“Direct Instruction” Effect and Other “Activators”

Teacher as Activator Teacher as Facilitator

Teaching students self-verbalization / questioning (.76)

Inductive teaching (.33)

Teacher Clarity (.75) Simulation and gaming (.32)

Reciprocal teaching (.74) Inquiry-based teaching (.31)

Feedback (.74) Smaller classes (.21)

Metacognitive strategies (.67) Individualized instruction (.22)

Direct instruction (.59) – includes elements of all in red!

Web-based learning (.18)

Mastery learning (.57) Problem-based learning (.15)

Providing worked examples (.57) Discovery methods in math (.11)

Providing goals (.50) Whole language (.06)

Frequent / effects of testing (.46) – also known as “retrieval practice”

Student control over learning (.04)

Graphic / Behavioral organizers (.41)

You could also add the following from the barometer:Formative evaluation (.90)Spaced vs. mass practice (.71)Worked examples (.57)

17

• Read the Direct Instruction research summary from Visible Learning (2009).

• What addition effects from the “Teacher as Activator” slide can you fit into the steps of direct instruction?

Activity

18

This article ties many of the pieces together (Hattie’s effect sizes, what we know about knowledge acquisition, and how to begin crafting the “why” in teacher-friendly language.)

19

Primary Sources of Information for Article

Research compiled from three different sources:

1. Cognitive science

2. Review of what master teachers do

3. Cognitive supports (effective procedures to help people move information to LTM)

20

Principles of Effective Instruction

1. Begin a lesson with a short review of previous learning2. Present new material in small steps with student

practice after each step3. Ask a large number of questions and check the

responses of all students4. Provide models5. Guide student practice6. Check for student understanding7. Obtain a high success rate8. Provide scaffolds for difficult tasks9. Require and monitor independent practice10.Engage students in weekly and monthly review

21

Activity

• Read the Problem-Based Learning research summary from Visible Learning (2009).

• What are your thoughts about how both direct / explicit instruction and problem-based learning might work together to ensure information gets stored in LTM?

22

• How does inquiry / discovery learning or problem-based learning fit with direct instruction?

Common Question to Address

23

It is not an either or – but a when

Direct / Explicit Instruction and Inquiry

Direct / Explicit Instruction

Inquiry / Discovery

Little or no background knowledge

A great deal of background knowledge in the domain (subject or concept)

History of difficulty, or failure

History of success

24

Integrating Inquiry Learning into the Explicit Instruction Format

“Principles of Effective Instruction” (Rosenshine) after #6 and #9:

After you “obtained a high success rate“Require and Monitor Independent Practice”

Explicit Instruction (Archer) #16: “Provide Distributed and Cumulative Practice”

Direct Instruction (Visible Learning, Hattie) #3: “Build Commitment and Engagement” and #7 “Independent Practice”

25

Safe Bets for Selecting a “What”

Effective instruction is the foundation of effective MTSS models (remember Hattie’s effect size for RtI…1.07)

Effective instruction = Direct / explicit instruction

I integrate inquiry and problem-based learning into the steps of direct / explicit instruction

Tiers 1-3 (programs that are used, teaching methods in the classroom, etc.) use the steps of direct / explicit instruction

26

Do not forget the integration between academics and behavior!

“Classroom Management” innovation = overall effect size of .52

BUT…look at the differences when you tease out what encompasses “classroom management”

27

Classroom Management

Teacher attributes for having well-managed classrooms:

“With-it-ness:” ability to quickly identify and act on behavioral problems (1.42) AND was able to retain emotional objectivity (.71)

Disciplinary interventions (.91) including classroom contingency strategies requiring specific students to reach a level of appropriate behavior

28

Classroom Management (cont.)

Clearly defined rules and procedures (.76)

Teacher/student relationships were moderators of classroom management (.87)

Tangible recognition for students who demonstrate appropriate behavior (.82)

Intervention that involved direct and concrete consequences for misbehavior (.57)

Chapter 3 of your new Hattie book unpacks this!

29

PBIS Connection

Big ideas of PBIS:Define expectationsTeach expectations Monitor students to ensure expectations

are followed Acknowledge students for meeting

expectations Correct students who are not

demonstrating the expectations by using a pre-established continuum of consequences

30

3.0 Planning for Back at the Building

31

1. How to systemically pull this off so the work spreads across the whole district

2. Need for a compelling why (Golden Circle)

3. Impact of cognitive load

Three Things to Consider

Leadership at Two Levels

• Mantra: The role of the district is to standardize the process while the role of the buildings is to customize implementation

33

Defining the Support Structure

Three entities that serve a distinct purpose to support implementation of any initiative, practice or program within the district

1. Individual(s) who have executive leadership

2. School Administrators (Administrative Team)

3. Implementation Team

34

Defining the Support Structure

• Each overlap within one another because there is a great deal of communication and coordination occurring between them

• The overlap occurs because of a person who has executive leadership authority and is the conduit between the entities

• There is also someone providing coordination of efforts and by doing so, works closely with the executive leader

35

1. How to systemically pull this off so the work spreads across the whole district

2. Need for a compelling why (Golden Circle)

3. Impact of cognitive load

Three Things to Consider

36

The Importance of “Why”

Golden Circle

Why

What

How

Never be afraid to ask “why.”

37

• People have a limit in terms of their ability to learn new things at any particular point in time.

• Conditions can be adjusted to minimize cognitive load and increase learning

– People learn better and can handle more when they:• Have adequate background knowledge• See how things connect.

The Critical Value in Understanding “Cognitive Load”

38

• How are you tying the pieces together…what is your big umbrella?

– District priorities– Hattie stuff– Next steps

• What is your compelling why for having people focus cognitive energy on Hattie learning?

• What actions will you take based on the learning from our last two sessions?

Planning to Bring this to Staff

39

Linking Initiative

s:

USA Schools

Holt Public School’s Pre K-12 Priorities under the MTSS Framework

READING3-4 Reading Street K-2 RS Fidelity Checks5-6 Developing Curr.5-12 Reading ApprenticeshipK-6 Common Assessments

DISTRICT WIDEDistrict-wide Building Data ReviewDistrict Data ReviewImplementing a Illuminate EdDistrict Assessment CalendarK-12 Intervention ContinuumCommon AssessmentsScope, Sequence and Pacing

MATHK-5 Essential Skills/Report Card Alignment6-8 Math Continuum6-12 Math PST Work5-12 Lesson StudiesK-6 Common Assessments

WRITINGA Year Curriculum Study K-6

INSTRUCTIONAL STRATEGIES & STUDENT

ENGAGEMENTTLT Model and Lesson Study

At the K – 12 LevelExplicit Instruction Visible Learning

K-5 Core Math PD

MTSS K-12 Framework integrated into the MI School Improvement Planning Process

BEHAVIORK-6 SWPBIS/Second Step Tiers 1,2,37-12 Academic CultureEMERGING:Science and Social StudiesEarly ChildhoodIntegrated Arts & KineticsHealth Education

41

Activity

Think about what information from this session and the previous one your staff need to hear.• What is the goal of brining Hattie’s

information to staff?

• What is your compelling why?

• Review the ideas provided and develop your own action plan for brining this information to staff (use chart paper and display)?

42

Reading Assignment

Please read the following before our next session: • Chapter 9: Acquiring complex skills

through social modeling and explicit teaching

Today’s work is further complimented by the following chapters from Visible Learning and the Science of How We Learn (2013)• Chapter 3: The teacher-student relationship• Chapter 7: Teaching for automaticity in basic

academic skill