Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and...
-
Upload
arnold-little -
Category
Documents
-
view
221 -
download
0
Transcript of Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and...
![Page 1: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/1.jpg)
Gathering Feedback for TeachingCombining High-Quality Observations with
Student Surveys and Achievement Gains
![Page 2: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/2.jpg)
Closing the effectiveness gap
2
![Page 3: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/3.jpg)
Progressing beyond “The Plateau”
3
![Page 4: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/4.jpg)
Ensuring Reliable & Trustworthy ObservationsEnsuring Reliable & Trustworthy Observations
![Page 5: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/5.jpg)
The First of Three QuestionsHow many distinct levels of teacher performance do you think an evaluation system should recognize?
5
![Page 6: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/6.jpg)
The Second of Three QuestionsWhat performance levels would you assign to what fraction of your district’s teachers on the following competencies?A.Classroom Management (time, behavior, materials)
B.Goals & Tasks (clear, appropriate, rigorous, interesting)
C.Supporting Student Understanding (content depth, feedback, questioning and discussion, instructional dialogue)
6
![Page 7: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/7.jpg)
The Third of Three QuestionsHow closely associated are teaching behaviors with student outcomes (academic growth over time)?
7
Stud
ent P
erfo
rman
ce
Teacher’s Classroom Observation Score
highly
moderately
weakly
![Page 8: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/8.jpg)
8
Validation
Teachers with Higher Observation Scores Had Students Who Learned More
![Page 9: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/9.jpg)
Observation Score DistributionsFramework for Teaching
9
![Page 10: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/10.jpg)
PLATO Prime, CLASS and MQI Lite
10
![Page 11: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/11.jpg)
Observation Score Distributions UTeach Observation Protocol
11
![Page 12: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/12.jpg)
12
![Page 13: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/13.jpg)
13
![Page 14: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/14.jpg)
14
![Page 15: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/15.jpg)
Measure Predictive power ReliabilityPotential for
Diagnostic Insight
Value-added
Student survey
Observation
Measures have different strengths …and weaknesses
15
![Page 16: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/16.jpg)
16
The Importance of Multiple Measures
![Page 17: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/17.jpg)
Compared to What?
Compared to MA Degrees and Years of Experience, the Combined Measure Identifies Larger Differences
17
… on state tests
![Page 18: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/18.jpg)
Compared to What?
…and on low stakes assessments
18
![Page 19: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/19.jpg)
Compared to What?
…as well as on student-reported outcomes.
19
![Page 20: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/20.jpg)
The Measures of Effective Teaching ProjectThe Measures of Effective Teaching Project
Two school years: 2009–10 and 2010–11 >100,000 students Grades 4–8: ELA and Math High School: ELA I, Algebra I and Biology
Participating Teachers
![Page 21: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/21.jpg)
Research PartnersResearch Partners
![Page 22: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/22.jpg)
22
The MET Project is ultimately a research project. Nonetheless, participants frequently tell us they have grown professionally as a result of their involvement. Below is a sampling of comments we received.
From Teachers:•“The video-taping is what really drew me in, I wanted to see not only what I’m doing but what are my students doing. I thought I had a pretty good grasp of what I was doing as a teacher, but it is eye opening … I honestly felt like this is one of the best things that I have ever done to help me grow professionally. And my kids really benefited from it, so it was very exciting.”
•"With the videos, you get to see yourself in a different way. Actually you never really get to see yourself until you see a video of yourself. I changed immediately certain things that I did that I didn't like.”
•“I realized I learned more about who I actually was as a teacher by looking at the video. I learned of the things that I do that I think that I’m great at I was not so great at after all.”
•“Even the things I did well, I thought, ok that's pretty good, why do I do that, and where could I put that to make it go farther. So it was a two-way road, seeing what you do well, and seeing the things that have become habits that you don't even think about anymore."From Raters:•“Being a rater has been a positive experience for me. I find myself ‘watching’ my own teaching more and am more aware of the things I should be doing more of in my classroom.”•“I have to say, that as a teacher, even the training has helped me refine my work in the classroom. How wonderful!”•“I have loved observing teachers, reflecting on my own teaching and that of the teachers teaching in my school.”
What the Participants Said….
![Page 23: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/23.jpg)
MET Extension: A Library of Teaching Practice
Additional Data Collection•Subset of 360 MET Teachers (disproportionately highly effective)•50 lessons taped (18,000 total lessons)•100% teacher & parental consent (allowing for broader public use)•Cheaper cameras with potential for scale
Library of Practice•Searchable database (tagged by Common Core standards, teaching practices, etc.)•Tagging to be done in partnership with schools of education (w/ teachers-in-training)
Potential Uses:•Rater training, certification, and calibration•School districts – professional development•Teacher training institutions – teaching methods•Observation instrument developers – validation of new & existing tools
![Page 24: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/24.jpg)
MET Logical SequenceMET Logical Sequence
24
Measures predict
Measures predict
Measures fairly reflect teacherMeasures fairly reflect teacher
Measures combine
Measures combine
Effective Teaching Index
Effective Teaching Index
Teaching Effectiveness Dashboard
Teaching Effectiveness Dashboard
Research Use
Measures reliable
Measures reliable
Measures improve
effectiveness
Measures improve
effectiveness
Measures stable under pressure
Measures stable under pressure
Measures communicated
effectively
Measures communicated
effectively
![Page 25: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/25.jpg)
Validation EngineValidation Engine
System picks observation rubric & trains raters
Raters score MET videos of instruction
Software provides analysis of:•Rater consistency•Rubric’s relation to student learning
![Page 26: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/26.jpg)
26
Four Steps
Four Steps to High-Quality Classroom Observations
![Page 27: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/27.jpg)
27
![Page 28: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/28.jpg)
Step 1: Define ExpectationsFramework for Teaching (Danielson)
28
Four Steps
Uns
atisf
acto
ry
Yes/no Questions, posed in rapid succession, teacher asks all questions, same few students participate.
Basic
Some questions ask for student explanations, uneven attempts to engage all students.
Profi
cien
t
Most questions ask for explanation, discussion develops/teacher steps aside, all students participate.
Adva
nced
All questions high quality, students initiate some questions, students engage other students.
![Page 29: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/29.jpg)
Step 2: Ensure Accuracy of Observers
29
Four Steps
![Page 30: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/30.jpg)
Step 3: Monitor Reliability
30
Four Steps
![Page 31: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/31.jpg)
31
Four Steps
Multiple ObservationsLeads to Higher Reliability
NOTES: The numbers inside each circle are estimates of the percentage of total variance in FFT observation scores attributable to consistent aspects of teachers’ practice when one to four lessons were observed, each by a different observer. The total area of each circle represents the total variance in scores. These
estimates are based on trained observers with no prior exposure to the teachers’ students, watching digital videos. Reliabilities will differ in practice. See the research paper, Table 11, for reliabilities of other instruments.
![Page 32: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/32.jpg)
32
Students with Most Effective Teachers Learn More in School
![Page 33: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/33.jpg)
Student Perceptions
33
CareCare
ControlControl
ClarifyClarify
ChallengeChallenge
CaptivateCaptivate
ConferConfer
TestPrepTestPrep
ConsolidateConsolidate
CareCare
• My teacher makes me feel that s/he really cares about me
• My teacher seems to know if something is bothering me
• My teacher really tries to understand how students feel about things
• My teacher makes me feel that s/he really cares about me
• My teacher seems to know if something is bothering me
• My teacher really tries to understand how students feel about things
ControlControl
• Students in this class treat the teacher with respect
• My classmates behave the way the teacher wants them to
• Our class stays busy and doesn’t waste time
• Students in this class treat the teacher with respect
• My classmates behave the way the teacher wants them to
• Our class stays busy and doesn’t waste time
ClarifyClarify
• If you don’t understand something , my teacher explains it a different way.
• My teacher knows when the class understands, and when we do not.
• My teacher has several good ways to explain each topic that we cover in the class.
• If you don’t understand something , my teacher explains it a different way.
• My teacher knows when the class understands, and when we do not.
• My teacher has several good ways to explain each topic that we cover in the class.
ChallengeChallenge
• My teacher asks students to explain more about the answers they give.
• My teacher doesn’t let people give up when the work gets hard.
• In this class, we learn to correct our mistakes.
• My teacher asks students to explain more about the answers they give.
• My teacher doesn’t let people give up when the work gets hard.
• In this class, we learn to correct our mistakes.
CaptivateCaptivate
• My teacher makes learning enjoyable• My teacher makes learning interesting• I like the way we learn in this class
• My teacher makes learning enjoyable• My teacher makes learning interesting• I like the way we learn in this class
ConferConfer
• My teacher wants us to share our thoughts• Students get to decide how activities are done in
this class
• My teacher wants us to share our thoughts• Students get to decide how activities are done in
this class
ConsolidateConsolidate
• My teacher takes the time to summarize what we learn each day
• The comments that I get on my work in this class help me understand how to improve
• My teacher takes the time to summarize what we learn each day
• The comments that I get on my work in this class help me understand how to improve
TestPrepTestPrep
• I have learned a lot this year about [the state test]• Getting ready for [the state ] test takes a lot of
time in our class
• I have learned a lot this year about [the state test]• Getting ready for [the state ] test takes a lot of
time in our class
![Page 34: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/34.jpg)
Student Perceptions Top 5 Correlations
34
Survey StatementSurvey Statement CategoryCategoryRankRank
11
22
33
44
55
• Students in this class treat the teacher with respect• Students in this class treat the teacher with respect
• My classmates behave the way my teacher wants them to• My classmates behave the way my teacher wants them to
ControlControl
ControlControl
ControlControl• Our class stays busy and doesn’t waste time• Our class stays busy and doesn’t waste time
ChallengeChallenge• In this class, we learn a lot every day• In this class, we learn a lot every day
ChallengeChallenge• In this class, we learn to correct our mistakes• In this class, we learn to correct our mistakes
3333 • I have learned a lot this year about [the state test]• I have learned a lot this year about [the state test] Test PrepTest Prep
3434 • Getting ready for [the state test] takes a lot of time in our class• Getting ready for [the state test] takes a lot of time in our class Test PrepTest Prep
![Page 35: Gathering Feedback for Teaching Combining High-Quality Observations with Student Surveys and Achievement Gains.](https://reader038.fdocuments.net/reader038/viewer/2022110206/56649f495503460f94c6ab61/html5/thumbnails/35.jpg)
Dynamic Trio
Combining Measures Improved Reliabilityas well as Predictive Power
35
Note: For the equally weighted combination, we assigned a weight of .33 to each of the three measures. The criterion weights were chosen to maximize ability to predict a teacher’s value-added with other students. The next MET report will explore different weighting schemes.
FFT alone
Tripod alone
VA alone
Combined(Equal Weights)
Combined(Criterion Weights)
.05
.1.1
5.2
.25
Diff
ere
nce
in M
ath
VA
(T
op
25
% v
s. B
ott
om
25
%)
0 .1 .2 .3 .4 .5 .6 .7Reliability
Note: Table 16 of the research report. Reliability based on one course section, 2 observations.
The Reliability and Predictive Power of Measures of Teaching: