Chile Measuring Teacher Effectiveness Rec

8/12/2019 Chile Measuring Teacher Effectiveness Rec

1/14

1

Formative Teacher Evaluation with Ensea Chile

Final Report

Delivered by Mitch Kochanski and Stephen Zerfas

Outline

1. Executive Summary ................................................................................................................ 2

2. Analysis and Communication of Student Survey Data ............................................................. 4

3. Feedback Tools/Techniques ................................................................................................... 7

4. Evaluating Teaching vs. Analyzing Learning ........................................................................... 9

5. Prior Recommendations ........................................................................................................ 12

6. Next Steps ............................................................................................................................ 14


2/14

2

Executive Summary

The purpose of this report is to: 1) Provide recommendations on the three action points

identified after our initial report, and 2) wrap up our engagement by providing a short overview

of prior recommendations as well as suggestions for next steps.

Analysis and Communication of Student Survey Data

We offer five recommendations for the analysis and communication of the student survey data,

as well as for all future data dissemination processes:

1. Ensea Chile should build a one-page feedback sheet. The format should be similar to

that used in the MET and should combine student survey data with classroom

observation scores (and, once available, student learning data). This is a quick, effective

way to disseminate the most important information.

2. Distinct versions of the one-page summary should be given to teachers and tutors:

teachers should receive numerical results, while tutors should receive relative rankings

of the teachers. This would prevent unwanted competition but maximize the feedback

utility of the data.

3. Each tutor should meet with their teachers in a one-to-one setting to hand over the

summary for the first time. The one-to-one setting will allow tutors to emphasize they are

there to help, work alongside teachers in developing goals and milestones to translate

feedback into action, and dig deep to understand what makes exemplary teachers so

extraordinary.

4. Tutors should follow up on the customized questions teachers used in their surveys,

identify the ideas, and incorporate them in the next survey.

5. Ensea Chile should draw attention to outstanding teachers who are passionate about

measures. These champions can serve to generate buy-in across the network.

Specific Feedback Tools/Techniques

A comprehensive list of the feedback tools and techniques we have encountered throughout our

research is compiled in Section 2 for future reference. This section does not contain specific

recommendations; it is simply intended as a feedback toolkit.

Evaluating Teaching vs. Analyzing Learning

Ensea Chile has expressed an interest in reforming their teacher evaluation processes to

better focus upon student learning (as opposed to teacher practice). The analysis of student

learning requires collection of student learning data, therefore the task at hand is to develop

effective data collection processes. Speaking from a broad level, we recommend that Ensea

Chile consider this task as requiring two simultaneous approaches: top-down and bottom-up.

The top-down approach is concerned with developing standardized, organization-wide student

achievement data collection processes. The bottom-up approach is concerned with training

teachers to gather, analyze and interpret student data on their own (on-the-spot assessments,

quizzes, tests, etc.).


3/14

3

Prior Recommendations

We believe several of the insights from our initial report are worth repeating, specifically

because their adherence demands the ongoing attention of Ensea Chile leadership:

1. All metrics need to be self-validated in student outcomes.

2. Use student achievement gains, teacher observations, and student surveys together,

especially when evaluating teachers.3. Engage teachers in design and implementation of feedback/evaluation systems.

4. Language is essential (Formative >> Evaluative), and it all comes down to

relationships.

Next Steps

In reflecting on the engagement, we have identified several next steps:

1. Build one-page feedback sheet modeled after MET.

2. Develop robust student achievement data collection processes that, at a minimum, use

standardized tests.

3. Validate teacher observations and student surveys by observing how they correlate with

student achievement gains.

4. Share feedback toolkit with tutors and discuss which opportunities seem most

interesting and useful.

5. Revisit Ensea Chiles existing teacher training practices and consider how well we are

training our teachers to gather, analyze and interpret student learning data on their own.

Investigate how other teacher-training organizations train their teachers in these

methods.


4/14

4

Analysis and Communication of Student Survey Data

How can we make this information valuable for teachers?

We suggest Ensea Chile focus on five things in the dissemination process:

1. Develop a one pager to visually represent statistical results

2. Provide absolute scores to teachers but relative ranks of teachers to tutors3. Deliver results for the first time in a one-to-one meeting with teachers

4. Translate data into action for each teacher by setting specific goals and dates for

progress reports

5. Seek out top-performing and enthusiastic teachers as examples for the network

1. Developing a One Pager

We suggest Ensea Chile develop a one-page summary of allevaluation/feedback data

available on individual teachers, not just the student survey, modeled after the MET project.

This comprehensive one pager will allow Ensea Chile to integrate delivery of (1) student

survey results with (2) teacher observation feedback using TAL and, eventually, (3) student

achievement gains once Ensea Chile has obtained the right data.

The sample one pager provided by MET can be found in an attached document, but it roughly

takes the three-part format illustrated below. Each section can use any number of graphics to

represent statistical results:

Overall Score

Student Achievement Gains(Not currently available; Ensea Chile needs to

develope this)

TeacherObservation

Student SurveyResults

The benefits to putting all three measures on the same page are that it (a) succinctly provides

feedback of every type and (b) graphically reinforces the breakdown of the measurement

system. Ensea Chile does not currently have helpful data on student achievement gains, which

typically come in the form of standardized test scores, but needs to secure these as soon as

possible anyway in order to validate their teacher observation and student survey results. Once

obtained, these should be included in the one pager.

2. Provide Absolute Scores to Teachers but Relative Rankings to Tutors: Solving the Feedback-

Competition Tradeoff

Ensea Chile has the option of delivering absolute scores or relative rankings to its teachers

and tutors. Relative rankings are always much more helpful because they give meaning to the

otherwise largely arbitrary numbers. For example, if a teacher were to receive an average of 4

out of 5 on a particular student survey question, she might think she did very well on that aspect


5/14

5

of her teaching. If, though, the average for the entire network was a 4.2, she could very well be

one of the lowest performers, indicating she should work to improve. On the other hand, relative

rankings are inherently competitive. Given that Ensea Chile is challenged with creating a

culture of cooperation and candid relationships and soliciting buy-in from anyone, the

organization needs to be sensitive to the pitfalls of competition. The last thing Ensea Chile

wants is to see its efforts to provide feedback to teachers backfire because its teachers now feelthey are isolated and even more inadequate when constantly compared to everyone else.

In solution to this problem, we think Ensea Chile can get the informative benefits of relative

rankings without the dangers of excessive competition by providing relative rankings of each

teacher (easily displayed with boxplots as shown in the attachment) to every tutor and providing

absolute scores in each category to every teacher (easily displayed numerically). This will allow

a teacher to immediately see what he or she can work on according to the current scales (which

we have found elsewhere to be successfully tied to student achievement) without pitting each

teacher against one another in too competitive of an environment. At the same time, each tutor

will can correct for misleading high or low scores where appropriate. Consider a situation in

which a teacher receives an average score in both keeping students busy and keeping them

interested, for example. If the entire network of teachers averaged high scores on keeping

teachers busy but low scores on keeping students interested, a tutor might encourage that

specific teacher to spend more effort on keeping students busy rather than keeping students

interested because she is a relatively low performer in the former but a relatively high performer

in the latter.

Note: we think providing relative performance information to tutors and absolute information to

teachers should be considered for all three types of feedback, not just student surveys.

3. Maximizing a One-to-One MeetingWe recommend Ensea Chile deliver each teachers student survey results in person in a one-

to-one meeting. This could easily be done in the same visit as the monthly teacher observation,

especially since the proposed one pager incorporates both teacher observation and student

survey results. This will allow tutors to deliver the results while pursuing four additional goals:

personally emphasize this is about helping and not evaluating teachers, pay special attention to

which questions teachers added to the survey and why, ask teachers what else they would like

to know or gather data on, and identify Champions and their questions.

1. Tutors can never have enough opportunities to demonstrate that this process is about

helping teachers first. They can and should deliberately stress this in the meeting.

2. Three blank spaces were added for teachers to write in additional questions to allow us

to cast a thousand seeds and allow good ideas we missed to rise to the surface. We

need to finish the job and capitalize on this opportunity. Each tutor should question the

motivation behind each teachers custom questions and dig deep when necessary.

Dialogue among tutors should be encouraged, and truly good ideas need to be

incorporated into the next survey.


6/14

6

3. In the same way the student survey asked for teacher input in order to solicit buy-in and

sow a thousand seeds, tutors should ask teachers what else they would like to know in

a one-to-one meeting.

4. Tutors should be on the look out for teachers that scored very highly on the survey (and

TAL rubric) and how they used their blank questions. These outstanding teachers can

serve as Champions (see Incorporating Champions in the Process below).

4. Translate Teacher Feedback into an Action Plan

The attached Diocese of Memphis teacher-input form (Appendix A) is a tool that pulls teachers

into the goal-setting and evaluating process. During the one-on-one meeting, tutors could use

this form to work together with teachers to translate the student survey data into actionable

goals. This is the final step in the feedback process; it is both essential and complex, and for

both of these reasons it is ideal that the teacher have the tutors direct support. This would

create a partnership out of a mentor-mentee relationship and customize the process to each

teacher. This would both a powerful way to encourage teacher buy-in and an effective way to

track success.

We believe the opportunity to transform the mentor-mentee relationship into a partnership using

this tool (or something similar) is so great an opportunity that it should be used in response to all

forms of feedback, not just the student survey.

5. Incorporating Champions in the Process

Exemplary teachers are not just assets for students, they are role models for other teachers in

the Ensea Chile network. Ensea Chile should be on the lookout for outstanding teachers to

(a) ask them what else they would like to know in the measurement process and (b) serve as a

testament to other teachers how the measurement process is helpful. Exemplary teachers who

are excited about measurement are likely to think of things we have not yet considered, makingtheir input especially valuable. Those who serve as role models could also go a long way in

getting other teachers or even tutors excited about the student survey progress. Quotes or

stories about how these teachers used measurement tools to aid in their success should be

shared with everyone associated with Ensea Chile. This could be done via a newsletter, a

webpage, reference in one-to-one meetings, detailed in handouts, or any number of creative

ways.

Resources

Gates, F. (2010). Learning About Teaching: Initial Findings from the Measures of

Effective Teaching Project

Gates, F. (2012). Gathering Feedback for Teaching: Combining High-Quality Teacher

Observations with Student Surveys and Achievement Gains

Gates, F. (2013). Feedback for Better Teaching: Nine Principles for Using Measures of

Effective Teaching


7/14

7

Specific Feedback Tools/Techniques

What are interesting options for teach feedback between tutor class observations?

This section presents a simple list of the many different tools/techniques for teacher feedback

that we have encountered during our research. We do not present specific recommendations in

this section; we simply mean to supply a toolbox of ideas (and their sources) that may be ofinterest to Ensea Chile either now or in the future.

Microphone/Video Recordings (Tom Kane): In situations where frequent classroom visits are

unfeasible, teachers might record a lecture to later share with their tutor and review together.

Tom Kane suggests that a teacher can learn some things from watching him/herself teach than

he/she cannot learn from a tutors feedback.

Teacher Mentor (ACE): Every ACE teacher is paired with a teacher mentor" in the school

where they work. The mentor is typically a veteran teacher on the school faculty, and the

purpose of the relationship is to create a candid and accessible resource of teaching wisdom for

the young ACE teacher.

Written Teacher Reflections (ACE): Every two weeks, ACE teachers write and submit a short

self-evaluative reflection in response to a prompt from their tutor. Examples of useful prompts

include: How did you feel about your classroom management this week? or What specific

improvement do you hope to make next week? These reflections create a frequent flow of

conversation between the teacher and tutor, and allow the tutor to respond directly to the

teachers most pressing concerns.

One Minute Essay (Light, 2001):after every lesson students are given one minute (or a

comparable amount of time) to answer the question What was the main idea of the lesson?The essays are anonymous and collected by the teacher. There are two major benefits:

1. Teacher receives immediate feedback on how well students grasp the main idea.

2. Students areforced to think in a synthetic way. Over time, it seems students expecting

to write the one minute essay think differently as the lesson unfolded!

Formal Time Logs (Light, 2001):students are asked to track on a hour-by-hour basis what

they do each day for up to two weeks. An advisor or teacher then sets up a small group or

individual meeting with a student (if individual, no more than 10 min is necessary) to review the

log. Three key things happen in this meeting.

1. Teachers very quickly get to know their students quite well, including their life outside the

classroom. You learn an enormous amount about someone by observing how they

spend their time!

2. Teachers are able to assess students time management skills and see over time how

they have influenced students time management skills.

3. Teachers are able to instruct students on how to improve time management where

necessary and suggest extracurricular involvement (there is lots of research

emphasizing the importance of connecting life inside and outside the classroom).


8/14

8

Solicit Student Critique of a Teachers Work (Light, 2001):A student(s) is asked how they

might improve a teachers work. This is most easily done in an advanced class with a teachers

own essay, article, or work in process. This is a powerful shift from usual exercises because it

engages the student in collegial partnership (this can still be done with young grades). We see

three major benefits: (1) when done right, the student often puts enormous effort into the critiquebecause they feel the teacher is counting on them for support, (2) the teacher gets to see

employ the creativity of new minds, and (3) the teacher gets to see which areas the student has

mastered in application.

Testing Teachers on Pedagogical Knowledge (MET): teachers take a written test that

assesses their knowledge about the art of teaching. This appears to have potential in testing

how well a teacher has internalized or is able to articulate critical principles or tactics such as

those that underlie the TAL rubric. However, the MET project deemed success on this test was

not associated with student achievement gains and did not report in its final results.

Real-Time Earpiece Feedback (TFA): Teach for America uses a subtle earpiece to deliver

real-time feedback to teachers, especially for teachers that are struggling with controlling the

classroom and dealing with gross behavior problems. The equivalent of the tutor sits in the back

of the class and speaks into a mouthpiece that is linked to the teachers earpiece. The teacher

can then receive real-time instruction on how to handle the situation.

Learning from the Best (TFA): TFA is experimenting with how to effectively use their very best

teachers (truly remarkable and inspiring) to help instruct other teachers. TFA has used video

material because it is easy to disseminate, but is also experimenting with live teacher

observations (only this time, the observer is learning from the example rather than evaluating).

So far, they have faced two major challenges. First, how can TFA codify these very best teacherpractices? Second, how can you use one truly talented teacher as an example without making a

beginner or less effective teacher feel inadequate?

Resources

Light, R. (2001). Making the Most of College, Harvard University Press.

This resource has great tools for effective teaching at the advanced level. There is less

direct focus on feedback.


9/14

9

Evaluating Teaching vs. Analyzing Learning

How can we incorporate student learning into our teacher evaluation practices?

We all agree that an effective system of teacher evaluations should look beyond teacher

practice and focus on student learning as well. Ensea Chile currently focuses more upon the

former than the latter; what steps should be taken to strike a better balance? The analysis ofstudent learning requires the collection of student data, but what data is most helpful and how

should it be gathered? Ensea Chile should equally consider two alternative approaches to

obtaining and utilizing feedback on student learning: top-down and bottom-up. The top-down

approach involves developing the large-scale, long-term processes needed in order to

administer standardized assessments. The bottom-up approach involves equipping teachers

with the tools and techniques they need to autonomously gather feedback on student learning

on a day-to-day basis within the classroom.

Top-Down Approach

Ensea Chile needs to obtain helpful standardized test data as fast as possible. This is a must

for several reasons. First, student achievement outcomes are the ultimate measure of success,

and every other measure has merit only insofar as it can be traced back to improving student

outcomes. Second, measures of success are only useful if they can be compared.

Standardizing these measures is critical if Ensea Chile hopes to compare its results across

teachers and against other organizations. Finally, while some argue non-academic outcomes

are even more important than academic outcomes, there is (a) consensus that academic

outcomes are important and (b) much previous work on standardized academic tests relative to

standardized non-academic measures. Ensea Chile should continue to be a pioneer in looking

into non-academic outcomes, but it needs to move to secure helpful standardized testing data

as fast as possible to validate all of its efforts with teacher observations, student surveys, or any

other metric. Everything will have been for nothing if these measures cannot be traced back tosome form of student success.

Bottom-Up Approach

The most effective feedback mechanisms for analyzing learning and improving teaching are not

large-scale processes that can be administered by Ensea Chile or the tutors; the most effective

mechanisms for analyzing learning and improving teaching are small-scale processes that must

be administered by the teacher inside the classroom. Ensea Chile should be concerned with

training its teachers to analyze student learning themselves. Listed below are recommendations

from Trav Johnson and Kim Marshall for how to help teachers gather feedback on student

learning.

Consider Student Learning in Classroom Observations. Teacher evaluation systems can begin

to incorporate student learning even before standardized tests are in place. Johnson

recommends that classroom observations and evaluations shift their focus to consider student

learning. Johnson recommends the following framework for helping teachers think about how

they achieve student learning:

1. Learning Goals: What are the most important things students are expected to learn?


10/14

10

2. Learning Activities: How are the learning activities designed to achieve these goals?

3. Learning Assessments: How do the assessment seek to evaluate the goals?

4. Learning Outcomes: What evidence exists to show achievement of learning goals?

Use On-The-Spot Assessments. Kim Marshall suggests that on-the-spot assessments are

essential to giving teachers crucial feedback for adjusting their teaching methods and lessonplans.Examples of effective on-the-spot assessments include:

Think-Pair-Share: After initial instruction, have students 1) reflect for a moment about a

question, 2) talk to an elbow partner (the teacher circulates and listens in), and 3)

share thoughts with the whole class

Cold-Calling. By randomizing which students are called on as opposed to selecting

from volunteers, you get a more accurate sample of the entire classs level of

understanding.

Quick-Writes/Quizzes.Ask students to write a summary, or to respond to a few key

questions. Should not take more than five minutes, and the data should be assessed

immediately.

Exit Tickets. Gauge student understanding of one or two key points at the end of every

lesson.

Use Interim Assessments.Kim Marshall suggests that short-term on-the-spot assessments

must be complemented by long-term interim assessments. Interim assessments are typically

administered every 4-9 weeks.

Evidence Supporting the Formative Utility of Assessments

Inside the Black Box (1998) by Black and William: Voluminous evidence of the impact

of in-class assessments on student learning

Visible Learning (2008) by John Hattie: Hattie suggests that one of the mostdeterministic factors of student learning (more powerful than parental involvement or

socioeconomic status) was the effective use of formative, in-class assessments

Growth Mindset by Carol Dweck. Regular and frequent assessments send the

message to students that they should be learning something new every single day.

Resources

http://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms

.pdf

Contrasts between teacher-centered and learner-centered paradigms

http://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-

evaluation

Framework for learner-centered teacher evaluation

http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-

goals/22423#.U1Uz_HddVv0

Student-centered teacher evaluations focus on learning goals
http://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdf


11/14

11

http://cte.udel.edu/sites/cte.udel.edu/files/u7/v17n8.html

PDSA Framework (for teachers): developing learner-centered assessment

http://www.slideshare.net/chesleyjr/student-centered-assessment-presentation

Various methods for measuring student learning

Gates, F. (2010). Learning About Teaching: Initial Findings from the Measures of

Effective Teaching Project(Tech. Rep.). Gates, F. (2010). Learning About Teaching: Initial Findings from the Measures of

Effective Teaching Project(Policy Brief).
http://cte.udel.edu/sites/cte.udel.edu/files/u7/v17n8.htmlhttp://cte.udel.edu/sites/cte.udel.edu/files/u7/v17n8.htmlhttp://www.slideshare.net/chesleyjr/student-centered-assessment-presentationhttp://www.slideshare.net/chesleyjr/student-centered-assessment-presentationhttp://www.slideshare.net/chesleyjr/student-centered-assessment-presentationhttp://cte.udel.edu/sites/cte.udel.edu/files/u7/v17n8.html


12/14

12

Repeated Recommendations from Prior Report

The purpose of this section is to briefly echo the most essential recommendations which were

made in the two initial progress reports. Each of the recommendations listed below are were

chosen to be repeated for two reasons: 1) they are crucial to success, and 2) their

implementation requires ongoing attention

1. All metrics need to be self-validated in student outcomes. Success in teaching is

measured by the student outcomes, and any attempt to measure teacher effectiveness via

student surveys or teacher observations is merely an intermediate attempt to measure the

process, not the outcome. The process, in turn, only has value if it is tied to the outcome, so all

Ensea Chiles hard work will have been for nothing if their efforts can not be proven to have

been associated with improvement in student outcomes. Literature from other contexts with their

own unique situations is not good enough. Ensea Chile must self-validate its metrics with its

own student outcomes.

2. Use student achievement gains, teacher observations, and student surveys together,

especially when evaluating teachers.Each ones weaknesses are offset by the others

strengths in terms of feedback potential, predictive power, and reliability. MET emphasizes over

and over again that using these together is most important when evaluating teachers and

making high-stakes decisions. When providing merely formative feedback, however, it is fine if

Ensea Chile spends more effort on strong feedback measures.

3. Engage teachers in design and implementation of evaluation system.Any formative

evaluation is a tool for teachers; our task is not only to design a good tool, but to ensure that

teachers value this tool. No evaluation system will impact student learning unless teachers

respect the system and appreciate its results; as such, it is essential that we constantly seek tofoster teacher buy-in. One approach to fostering teacher buy-in is to engage them as much as

possible in its design and implementation. Several specific actions that might be undertaken to

foster teacher buy-in are listed below:

Invite teachers to design their own student survey questions

Invite teachers to participate in focus groups or conference calls

Regularly gather feedback from teachers - either formally or informally - about the how

well the system is working and how useful the results are.

4. Language is essential. Ensea Chile tutors should be careful and deliberate in their use of

language when communicating with teachers regarding evaluations. In every interaction, tutors

should use language that cultivates an attitude of collaboration, candor andcommitment to

learning.Avoid use of the word evaluation whenever possible, especially in regard to feedback

instruments that are exclusively formative in nature. Additional thoughts/examples of the ideal

use of language are listed below:

Formative vs. Evaluative

Im here to collaborate with you vs. Im here to evaluate you

What do you think? vs. Heres what I think?


13/14

13

How can we improve? vs. How can you improve?

This new system will have a positive impact on teachers everyday lives

How can we learn from whatyoure doing well, and support you in places where youd

like to improve

5. It all comes down to relationships. If teachers have good relationships with theirtutors/principals/mentors, then teachers will be more open to constructive criticism and feedback

where they need it most.

6. Many, unannounced short observations >> few, announced long observations. Most

everyone agrees that six 10-minute observations are more useful for teachers than one 60-

minute observation. Here is the rationale:

Teachers need feedback more frequently than once or twice a semester

Once-a-semester observations produce long lists of feedback, but teachers can work

better with just one or two suggestions at a time

Teachers are nervous about having an entire semesters teaching evaluated in one

session; with many sessions, the stakes are lower and the observation is more

candid/authentic.

Resources

Rethinking Teacher Supervision and Evaluation, by Kim Marshall

Is Supervising the Heck out of Teachers the Answer?

http://www.edweek.org/ew/articles/2008/05/07/36marshall_ep.h27.html

Designing and Implementing the Next Generation of Teacher Evaluation Systems:

http://ccsr.uchicago.edu/publications/designing-and-implementing-next-

generation-teacher-evaluation-systems-lessons-learned
http://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learned


14/14

14

Next Steps

In reflecting on the engagement, we have identified several next steps:

1. Build one-page feedback sheet modeled after MET. This is detailed in the

recommendations, but needs to be done to move forward with the dissemination process.

2. Develop robust student achievement data collection processes that, at a minimum, use

standardized tests.As mentioned earlier, Ensea Chile needs to self-validate to generate

credibility for all of its feedback measures. Using standardized tests is a major part of that

process. Ensea Chile should take advantage of standardized tests required by the state, but

should not stop until it is content with the quality and amount of data. There are many types of

standardized tests that Ensea Chile could get a hold of and implement with its teachers at the

classroom level, if not the school level (i.e. MET uses the Stanford 9 Assessment). Whether

Ensea Chile uses tests currently used by the state, implements its own, or mixes both, Ensea

Chile should definitely use tests with a vertical scale that transcends grades. This will allow

results that reveal improvement for students that are far behind grade level and would simply fail

tests with narrow focus limited to the current grade.

3. Validate teacher observations and student surveys by observing how they correlate

with student achievement gains.Once tests are implemented, analysis needs to be done to

validate teacher observations and student surveys. Feedback from these tools is a only a

guess, and a very imperfect one at that, until they are validated with student outcomes.

4. Share feedback toolkit with tutors and discuss which opportunities seem most

interesting and useful.This takes the toolkit out of the office and gets it into the hands of

teachers who are in the best position to identify when and where to use a new tool.

5. Revisit Ensea Chiles existing teacher training practices and consider how well we

are training our teachers to gather, analyze and interpret student learning data on their

own.This takes the benefits of measurement and evaluation the Training and Support Team

has deemed so valuable and enables each teacher to reap the benefits of their own micro

measurement and evaluation processes. Ensea Chile should investigate how other teacher-

training organizations train their teachers in these methods. A possible first step would be Kim

Marshalls book, which has much more information regarding on-the-spot and interim

assessments.

Chile Measuring Teacher Effectiveness Rec

Documents

Transcript of Chile Measuring Teacher Effectiveness Rec