Feedback: Time to Start Getting It

Raymond J. McNulty, President

@ray_mcnulty

SUCCESS BY DESIGN NOT BY CHANCE

Our work is about “Learning.”

“In too many cases, testing is used as the measure to judge whether change has occurred rather than as a mechanism to further enhance and consolidate learning by teachers or students…

It is the feedback information and interpretations from assessments, not the numbers or grades, that matter.”

-- John Hattie

Competitive Business Model “Feedback”

•Know how you are doing•Fix things that are wrong•Get new ideas•Evaluate your ideas•Build better relationships

Automaster

Hotels

Airlines

Businesses

HospitalsCustomer satisfaction surveys

Motivation is a key ingredient

for success in learning.

What Works Best?• REQUIRE

• MANDATE

• FORCE

• EXCITE

• EMPOWER

• MOTIVATE

Essential Design Elements

1. EXCITEMENT2. PASSION3. MOTIVATION4. EMPOWERMENT

Where are these elements in your school improvement plans

and in the classroom?

Talking with kids…

It’s not us against them!

25th Percentile

75th Percentile

1. CARE: My teacher in this class makes me feel that s/he really cares about me

40 73

2. CONTROL: Our class stays busy and doesn’t waste time.

36 69

3. CLARIFY: My teacher explains difficult things clearly. 50 79

4. CHALLENGE: My teacher wants me to explain my answers – why I think what I think.

59 83

5. CAPTIVATE: My teacher makes learning enjoyable. 33 72

6. CONFER: My teacher wants us to share our thoughts. 47 79

7. CONSOLIDATE: My teacher takes the time to summarize what we learn each day.

38 67

Proportions of students scoring in each decile of the MCAS 8th grade ELA distribution

Proportions of students scoring in each decile of the MCAS 8th grade Math distribution

MCAS math gains 8th to 10th grade, compared to others from the same 8th grade decile

(School Rank Percentile)

MCAS ELA gains 8th to 10th grade, compared to others from the same 8th grade decile

(School rank percentile/100)

CULTURE DRIVES STRATEGY

To get at culture issues you must use qualitative data,

follow up with great questions, and take thoughtful action.

Pride and Involvement

Empowerment

Collaboration

Idea/Reflection Time

Fun

Risk Taking

Critical Conversations

Idea Support

Conflict/Tension

10 innovative organizations

School of excellence

5 stagnant organizations

What do you value?

Teacher – Student ComparisonsT – I am aware of my students’ interests outside of school.

84%

S – My teachers know my interests outside of school.

28%

A Teacher’s Story“As a middle and high school English teacher, I can say unequivocally that

nothing has made me more uncomfortable than having a supervisor sitting in my

classroom, scribbling away on his or her legal pad as I teach,” said Alexis Wiggins in a thoughtful Education Week article.

Sometimes the feedback was helpful, but a very stressful process.

A Teacher’s Story• She tried getting student feedback

and learned some lessons:– End of course feedback wasn’t as

helpful as mid course feedback– Not all students were open with her in

the hand-written mid-course feedback– An “aha moment” occurred when her

principal asked for feedback on a teacher mentoring program

A Teacher’s Story• She concluded that she would use

computer feedback mid term to get student feedback– She was amazed at the feedback she

got… concerns about grading, favoritism and reading level of material

– “I realized that I had been kidding myself all these years,” she said. “I hadn’t been getting truly honest feedback, just slightly honest feedback.”

“If we want to be serious about students’ learning, we need to be serious about our own learning. We need to continually seek and accept ideas, help, and criticism. Feedback works.”

Alexis Wiggins in Education Week, Oct. 20, 2010 (Vol. 30, #8, p. 19)

School attendance is mandatory until age _____

12

4 WE™ Surveys = 4 Perspectives on Teaching and Learning

• WE Lead– Coherent Vision, Empowerment, Culture of Learning,

School Management, Community Partnerships• WE Teach

– Rigor, Relevance, Relationships• WE Learn

Rigor, Relevance, Relationships• WE Support

– Rigor, Relevance, Relationships

STRUCTURE of the SURVEYS

1. I2.WE3.THEY4.COMPANION INDICATORS

The mission statement promotes high expectations.

30%

I understand the importance of our mission statement.

68%

Total in Agreement

We Lead - Whole Staff Survey

Coherent Vision

Total In Agreemen

t

Full-Time Dept. Chairs

Classroom

Teachers

Support

Staff

Staff morale at this school is high.

50% 53% 50% 52%

We Lead - Whole Staff Survey

Years working in schools

Coherent Vision

1st year

2-5 years

6-10 years

11-20 years

Over 20

Staff morale is high at this school.

69% 57% 52% 48% 49%

We Lead - Whole Staff Survey

I am expected to make students passing the state test my number one priority.

58%

I spend too much time re‐teaching what students should already know.

48%

Total in Agreement

We Teach - Instructional Staff Survey

When I graduate from school, I hope I will be prepared forcollege.

86%

Total in Agreement

We Learn - Student Survey

My teachers make me aware of different career choices.

50%

I am required to demonstrate my understanding in a varietyof ways (i.e. speaking, writing).

65%

Total in Agreement

We Learn - Student Survey

Think about how this can help with the CCSS.

Teacher – Student Comparisons

T – Students can apply what I am teaching to their everyday lives.

92%

S – I can apply what I learn to my everyday life.

59%

California's new "parent trigger" law allows parents of children at a troubled public school to "trigger" one of four school intervention models simply by signing a petition.

Similar legislation is pending in 14 states…

Having a community survey in place and knowing their concerns in advance could be very helpful to the system.

Enable parents who could gather a majority at any persistently failing school to either:–fire the principal–fire 50% of the teachers–close the school–or turn it into a charter school.

Academics are a priority in the school/district.

79%

The schools in our community prepare students academicallyfor success in life.

69%

Total in Agreement

We Support - Community Survey

“Each of the three schools administered the We Teach and We Learn surveys to all

teachers and students in the fall of 2008. This was truly an “ah-ha!” moment for the

majority of committee members and served to spark meaningful dialog and

commitment to positive change prior to planning for change.”

Geraldine Sawrey, Assistant Superintendent of School Improvement

Cabell County School District, WV

Student work + Student test scores = Teacher evaluation

What about student voice?

WE Surveys vs. iKnowSurvey everyone that stays at

hotel

vs.

Survey those with rooms near elevator

WE Surveys vs. iKnow

Big picture/school

SummativeCurricularStructural

“How did we do?”

Classroom/student level

Granular

Instructional

“How would I adjust to this data?”

"Students are the ones that need the best teaching - it's their life, it's their

opportunity to be taught at the best level, and they know what kind of teaching works

best because they are in the classroom all day every day, five days a week,”

- Adam Fischer, president, Boston Student Advisory Council The Boston Globe (Boston, MA) | May 6, 2010

“Kids know effective teaching when they experience it,”

- Dr. Ronald Ferguson, Harvard University

The New York Times | December 10, 2010

Why student perception data?

• Ensure that students are engaged in class

• Assess the conditions that inspire students to learn

• Inform decisions with data

What it measuresStudent perceptions of:•oneself in the learning

process•of the course overall•of the teacher and instruction

QuestionTotal in

AgreementGender

Male Female

This class challenges my

thinking.76% 73% 80%

In this class, we discuss

issues that are interesting to me.

62% 61% 62%

QuestionTotal in

Agreement

The teacher knows my hopes and dreams. 35%

The teacher expects me to be

successful.89%

7th 8th 9th 10th 11th 12th

The teacher cares if I am

absent from class.

47% 52% 66% 73% 81% 85%

Question Total in Agreement

If you get a chance visit

Follow us!

• Ray McNulty: @ray_mcnulty

• International Center for Leadership in Education:@rigorrelevance

Feedback: Time to Start Getting IT

Raymond J McNulty, President

@ray_mcnulty

Using student survey responses in one section to predict learning gains in another.

Predicted difference per school year*

On state math tests 4.8 months

On the Balanced Assessment in Math 3.7 months

On state English Language Arts (ELA) tests 2.3 months

On the Stanford 9 Open Ended ELA 2.9 months

*Based on Table 9, p. 26: Bill and Melinda Gates Foundation, “Learning about Teaching: Initial findings from the Measures of Effective Teaching Project.” December 2010.