Multiple Measures of Teacher and Principal Effectiveness

Teacher Evaluation Training for Evaluators

2014-15Rosanne Javorsky and Dr. Patricia DiRienzorosanne.javorsky@aiu3.netpatricia.dirienzo@aiu3.net

11Collaborative CodeSilence electronic devicesMinimize the use of those devices during presentations and activitiesObserve the time frames and signalsLimit sidebar conversationsKeep your materials organized22Project Development - GoalTo develop a teacher effectiveness model that: will reform the way we evaluate teachers

and identify critical components of teacher training and professional growth

332010-2011 Phase IThe following were pilot sites for Phase I:

Allentown School DistrictCornell School DistrictMohawk School DistrictIU5 Northwest Tri-County Intermediate Unit442011-2012 Phase IIParticipation Levels

85 Districts11 CTCs11 IUs8 Charters552012-2013 Phase III293 LEAs (including school districts, CTCs, charter schools, IUs)1,255 schools1,972 supervisors20,350 teachers in PSSA-tested subjects3,771 teachers in non-PSSA-tested subjects66Act 82 of 2012Addresses educator evaluations in the Commonwealth

Includes multiple measures to be used in evaluations:Observation/practiceSchool building data*Teacher-specific data*Elective data*(* Must be related to STUDENT performance/achievement)7Act 82 of 2012Pertains to 3 groups of educators:TeachersNon-teaching ProfessionalsPrincipals

Went into effect on July 1, 2013 (for teachers; will go into effect for non-teaching professionals and principals on July 1, 2014)

89Math, Reading, Writing and Science teachers in grades 4-8 and Biology, Literature, and Algebra I teachers of the course that triggers the Keystone Exam. (Not third grade teachers, however.)

Law: School Building Data Regulations said SPP. Use the most current data is the current school years data which isnt available until the fall of the next year.

Teacher Specific Data: Will not go into effect until the 15-16 school year, because it is based on a three year average. Cant be used for evaluation until you get a three year average. If you have teachers looping in a HS, (Alg. I then Alg II, teachers wont have 3 consecutive years, will not get a PVAAS specific report. Elementary teachers can get PVAAS even if they change grades as long as it is between 4-8 grade.

Danielson will be worth 50% and 35% for Elective data since there is no teacher specfic data.

When you are working with Level I teachers, when you do the first evaluation of the year, if that teacher was in your building or district the year previously, you would use the last years PVAAS. 9Teacher Effectiveness System in Act 82 of 2012

Teacher Observation/Practice

Planning and PreparationClassroom EnvironmentInstructionProfessional Responsibilities

The rating of a classroom teacher for effectiveness in teacher practice shall be based on classroom observation or other supervisory methods.Teacher Effectiveness System in Act 82 of 2012

Building level data will be represented using the academic score determined via the Pennsylvania School Performance Profile (SPP). This profile will include data from all in the blue box (below left), including the following when applicable:

PSSA/Keystone ExamsIndustry Standards-Based AssessmentsGraduation RatePromotion RateAttendance RateInternational Baccalaureate (IB)SAT PerformanceACT PerformanceAspire

Teacher Effectiveness System in Act 82 of 2012

Any data used for a rating in this area must be attributable to the specific classroom teacher who is being evaluated and rated.

LEAs shall use the Student Learning Objectives (SLO) process to document, determine, and validate the weight assigned to Teacher Specific Data measures that establish the Teacher Rating where applicable.*

Teacher Specific DataUses all of the following if applicable and available:

Student Performance on Assessments (% Proficient & Advanced) Not more than 5%PVAAS 3-Year Rolling Average At least 10%IEP Goals Progress* - not more than 5%LEA Developed Rubrics* - Not more than 15%

Teacher Effectiveness System in Act 82 of 2012

Must use the PDE Student Learning Objectives (SLO) process template

PDE recommends a separate elective data measure from any LDR used in their Teacher Specific category. However, LEAs may agree to use a single elective data measure (from the PDE list) for both Teacher Specific and the Elective Data pieces of the pie chart in the absence of teacher specific data by the proper %. Observation/EvidenceDanielson Framework DomainsPlanning and PreparationEducational EnvironmentDelivery of Service Professional Development

Student Performance of All Students in the School Building in which the Nonteaching Professional Employee is Employed District Designed Measures and ExaminationsNationally Recognized Standardized TestsIndustry Certification ExaminationsStudent Projects Pursuant to Local RequirementsStudent Portfolios Pursuant to Local Requirements

Non Teaching Professional Employee Effectiveness System in Act 82 of 2012 Effective 2014-2015 SY14Goes into effect in 2014-2015.

Observation & EvidenceThe framework is different with some different domains. If you have an itinerant in this category, then each school they are in will be prorated from each building.

14Who are non-teaching professionals?Those working under an Educational Specialist certificate (school nurse, guidance counselor, H&SV, IT specialist, school psychologist, dental hygienist)

Those working under an instructional certificate who do NOT provide direct instruction to students

Those holding a Supervisory certificate (Supervisor of Curriculum & Instruction, Pupil Services, Single Area, Special Education, Vocational Education)1515Observation/ PracticeFramework for Leadership Domains

Strategic/Cultural LeadershipSystems LeadershipLeadership for LearningProfessional and Community Leadership

Building Level Data/School Performance ProfileIndicators of Academic AchievementIndicators of Closing the Achievement Gap, All StudentsIndicators of Closing the Achievement Gap, SubgroupsAcademic Growth PVAASOther Academic IndicatorsCredit for Advanced Achievement

Correlation Data/ Relationship Based on Teacher Level Measures

Elective Data/Student Learning ObjectivesDistrict Designed Measures and ExaminationsNationally Recognized Standardized TestsIndustry Certification ExaminationsStudent Projects Pursuant to Local RequirementsStudent Portfolios Pursuant to Local RequirementsPrincipal Effectiveness System in Act 82 of 2012

6/26/2014Goes into effect 2014-20

15 using a different framework. The framework has been adapted somewhat from the original. The technical advisory committee is working on this.

A Performance Level Chart is being created for a conversation for the correlation piece

16Multiple Measures of Teacher Evaluation Observation Evidence (50%)Principal/Supervisor classroom observations, including evidence that demonstrates behaviors associated with improving student achievement

Domains (Danielsons Framework for Teaching)Planning and preparationClassroom environmentInstructionProfessional responsibilities

Our focus these two days

1717Framework for Teaching Rubric4 Domains22 Components

4 Levels of Performance for each componentFailingNeeds ImprovementProficientDistinguished 1818Questions or Comments?19Up to three questions or concerns you hope will be addressed. share at your table value add report out.19Introduction to the DomainsOBJECTIVES: Participants will learn

How their thinking about good teaching compares to the framework we will use.That the Framework represents good common sense, and much that we already know, about teaching.The form and content of Domains 1, 2, 3 and 4.Show: Slide with learning intentions. Let participants read silently (30 sec)

20Having an ImpactIf we want to impact student achievement and growth

Then we must impact teaching and learningAnd of course, we know what good teaching is2121Worksheet #1Participant Materials

Lets begin by collecting our thinking about good teaching.22

Distribute: Worksheet #1 or direct participants to their packet of participant materials- page 3

22Wisdom of Practice

What are the qualities of teaching most tightly tied to student learning? 23(@10 minutes, total)

The goal of this activity is to elicit what the audience knows about good teaching. In the subsequent activity, we will connect this schema to the research.

Give Directions: Tell participants to work alone and list on the left-hand side of the sheet the qualities of teaching that they know impact student learning.

Make sure participants work on the left-hand side of the paper.Allow 3 minutes: no talking or collaborating during this timeStop: After 3 minutes, ask participants to end their writing and to take a quick look at the list of an elbow partner, comparing for similarities and differences.Ask: After 2 -4 minutes, ask how many people had identical lists. (Same words, same order, etc.)When no one raises their hand, explain that this is why a common definition of practice is beneficial.

Next, ask the audience to raise their hand if their list was highly similar to their partners. Many participants will raise their hand, so point out that this wisdom of practice if reflected in the Framework for Teaching. 23The DomainsPlanning and Preparation

The Classroom Environment

Instruction

Professional Responsibilities24(5 minutes total)The goal of this activity is to connect participants qualities of good teaching to the domains of the Framework.

Give Directions: Show the slide The Domains and explain that these are the four domains of the Framework. Tell participants to put a #1 beside any item on their list that is related most to Domain 1, a #2 beside any items that most relate to domain 2 and so on for all four domains of the FrameworkRemind: Only one number per item on the list; decide on best fit.Allow @ 3 minutes.

24A Framework for Teaching:Components of Professional Practice25Domain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesParticipating in a Professional CommunityGrowing and Developing ProfessionallyShowing ProfessionalismDomain 3: InstructionCommunicating with StudentsUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSetting Instructional OutcomesDemonstrating Knowledge of ResourcesDesigning Coherent InstructionDesigning Student Assessments Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical SpaceThe goal of this activity is to expand participants knowledge of the Framework domains to include the 22 components they contain.

Show: Slide, The Framework for Teaching and invite participants to adjust the numbers on their list if they need to, based on this additional information.Allow 3 minutes

Say: How many had number four as the least frequently occurring item on the list? Why do you think this is so? (because most of us think of the items in Domain 4 as just things we have to do and not components that are connected to student learning.

Explain that the items on the right-hand side of the chart are the on-stage components of the Framework, those that we see when we observe in a classroom. On the other hand, Domains 1 and 4, on the left, are the off-stage domains of the Framework. They occur outside the act of teaching.

Tell participants that, just like any other content, the Framework has its own vocabulary. There are four domains, and 22 components. Ask them to try to use the correct terms when referring to these concepts going forward.

Note: if the Framework for Teaching is to be used for teacher evaluation, point out that the off-stage domains are important, and because they are behind the scenes, we have to make sure to collect information about them before and after the observed lesson. 25Worksheet #2 Participant Materials26Identifying the Domains

Extending our learning of the domains(10 minutes)

The goal of this activity is to conclude the initial learning about the Framework by having participants match scenarios to the correct Domain of the Framework. Now, invite participants to extend their learning about the Domains of the Framework, and direct them to Worksheet #2.

Instruct them to work as a table group and to place the number of the Domain in the space beside each statement, indicating which Domain is most closely related to that statement.

Only one domain number per items is permissible.

Allow @5 minutes, then process chorally: facilitator says letter of the item and the class calls out the letter. If there is dissonance, stop and discuss.

Answers to WORKSHEET #2:A. 2B. 3C. 1D. 4E. 1F. 2G. 4 H. 1I. 3J. 4K. 2

1A Framework for Teaching:Components of Professional PracticeDomain 4: Professional Responsibilitiesa. Reflecting on Teachingb. Maintaining Accurate Recordsc. Communicating with Familiesd. Participating in a Professional Communitye. Growing and Developing Professionallyf. Showing ProfessionalismDomain 3: Instructiona. Communicating with Studentsb. Using Questioning and Discussion Techniquesc. Engaging Students in Learningd. Using Assessment in Instructione. Demonstrating Flexibility and ResponsivenessDomain 1: Planning and Preparationa. Demonstrating Knowledge of Content and Pedagogyb. Demonstrating Knowledge of Studentsc. Setting Instructional Outcomesd. Demonstrating Knowledge of Resourcese. Designing Coherent Instructionf. Designing Student Assessments Domain 2: The Classroom Environmenta. Creating an Environment of Respect and Rapportb. Establishing a Culture for Learningc. Managing Classroom Proceduresd. Managing Student Behaviore. Organizing Physical Space27The goal of this slide is to illustrate that not all components of teaching can be seen during a lesson.

Show: The slide of the Framework for Teaching and remind participants that it contains four domains and 22 components, all of which describe the qualities of teaching that are tied to student learning.Remind: That the brown Domains, 1 and 4, are the off-stage domains. That is, they dont happen during teaching, they happen before or after.

The green domains are the on-stage domains that are directly experienced during the act of teaching. This is important because teaching is much more than what we can see during the teaching of the lesson.

It also suggests that teacher evaluation is much more than just watching a lesson; it includes gather information about a teachers planning, and also about the professional responsibilities that the teacher regularly performs. Say that we will see how this concept of on and off stage is played out in the Pennsylvania teacher evaluation process.

271Features of A Framework for Teaching28Generic: applies to all grade levels, content areas

Not a checklistEvidence based/reflective

Not prescriptive: tells the what of teaching, not how

Comprehensive: not just what we can see

Inclusive: Novice to Master teacher

Show: The slide containing the features of the Framework and explain them as follows: (2 3 minutes)

Generic: Its useful to a district to have ONE definition that applies to all teaching situations, all grade levels and all content areas.Not a checklist: Teaching is too complex to be captured in a checklist. Well see in a moment that the Framework has rubrics to describe it.Not prescriptive: Doesn't tell teachers how to do their work; rather, it describes WHAT qualities of teaching are most likely to produce student learningComprehensive: As we just saw, it contains both off-stage and on-stage aspects of teaching Inclusive: Applies to all teachers, from the newest to the most experienced.

Ask: Participants to turn and talk to an elbow partner for 2 minutes about why these qualities of the Framework are important and useful.

28

29Direct: Participants to their copies of the Framework rubrics, found in the Resource Section of your BinderShow: Slide containing the rubric for component 2a. Invite: Participants, with an elbow partner, to examine their rubrics for the components of the Framework and to discuss what they notice. (4 5 minutes), then, Point out: That the rubrics contain four Levels of Performance: UnsatisfactoryProgressing/Needs ImprovementProficientDistinguished

(Remind participants that this is another vocabulary word: Levels of Performance.)

Also point out that some of the components are shaded. These are the power components, which means that they are especially useful to us in growing our practices. While all components are important, it is sometimes useful to streamline our thinking about them, since 22 seems like a lot. We cant really separate the power components from the others realistically, but if we want to give any components special attention, then these ones would be good choices. Say: That the rubrics are useful in making the Framework more complete, and enable us all to acquire a common vision of teaching at its higher (and lower) levels. It is useful for many purposes to have these levels of performance: self-assessment, use with new teachers, evaluation, etc. It makes the process of observing teaching less subjective. Tell participants that well now do an activity that gives us an additional experience with the Framework.

291Assessing prior knowledgeHow much do YOU know about Charlotte Danielsons Framework for Teaching?

Complete the Framework for Teaching Assessment individually30A Framework for Teaching:Components of Professional PracticeDomain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesParticipating in a Professional CommunityGrowing and Developing ProfessionallyShowing ProfessionalismDomain 3: InstructionCommunicating with StudentsUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSetting Instructional OutcomesDemonstrating Knowledge of ResourcesDesigning Coherent InstructionDesigning Student Assessments Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical Space31Trainers note: participants should be seated in tables of 5 or more participants.

Introduce the activity:

Direct participants attention to the blue domain, Instruction.

Say: Referring to the slide of the framework, explain that today well take a deep dive into one of the domains of the Framework, Domain 3. We chose this on-stage domain because it is what most people think of when they think about teaching.

31132Domain 3 Components3a Communicating with Students

3bUsing Questioning and Discussion Techniques

3cEngaging Students in Learning

3dUsing Assessment During Instruction

3eDemonstrating Flexibility and Responsiveness32Direct participants to their rubric of Domain 3 and give them a moment or two to skim them.

Explain a few points:3a has a lot to do with explaining directions and procedures3b refers to a technique used to advance learning3c is the most important component of the entire Framework, the purpose for which all other components exist3d is about assessing students during the act of teaching so that instruction can be modified as it is delivered, if necessary3e has to do with modifying teaching based on either an appropriate teachable moment, or based on evidence that instruction is not working in some way.

Direct participants to Worksheet #3 in their Participant Materials, found on page 5. 32Worksheet #3a Participant Materials

33Create a specific example of your assigned component: Ms. T says to Joey, You dummy!

Do not restate the rubric: Ms. T is rude to a student.The goal of this activity is for participants to be able to translate the words in the rubric to a specific example that illustrates the level of performance.

Ask participants to letter off around their table, a through e, representing the 5 components of Domain 3. Once each person has been assigned a letter (component) of Domain 3, ask them to reassemble into component-alike groups: all letter as together, bs together, and so on. ** Modification for large group Pick a letter from the supply box on your table. The following letters will work on the following components: 3a A & F; 3b B & G; 3c C & H; 3d D & I; 3e E & I. Find the table with your letter on it.

Once participants are settled in their expert groups, explain (in @ 3 minutes) the process of the activity, in which the goal of the group is to create four very specific examples of their assigned component at each of the four levels of performance. The examples should be created collaboratively and written on the worksheet by all members of the group, not just one recorder.Give a couple more examples in addition to the one on the slide, of specific vs. restating:Specific: 7 students raised their hands to ask for clarification on the directionsRestating: The teachers directions were vague.

Specific: The teacher stopped the lesson, said she could tell students were confused, and regrouped them according to their understandingRestating: The teacher made a smooth, major adjustment to the lesson.Allow: 10 - 15 minutes for small groups to finish the assignment. Note: you will need to check each groups work early to make sure they are creating specific for instances, not restating the rubric and that everyone is writing the generated examples on their own papers.

33Worksheet #3b Participant MaterialsGeneralizing about Levels of Performance 34What do all the examples of Failing have in common?

What do all the Needs Improvement examples have in common?

Proficient?

Distinguished?The goal of this activity is for participants to generalize the characteristics of the four levels of performance of the Framework, from the examples they hear.

Once participants have finished in their expert groups, regroup participants back in their home groups Once they are resettled, explain the activity: Invite participants to go around their new group and have each person read the unsatisfactory example created for their component. While participants are sharing, the rest of the participants in the group should use worksheet 3b, found on page 5 of Participant Materials, to write down the characteristics of the unsatisfactory level that they are hearing as their colleagues share. They are not to write each example; rather, they are to write the characteristics of the level of performance they are hearing.

Explain to participants that they are to repeat this process for the basic, proficient and distinguished levels of performance, going around their group and having each member share their example for that level of performance, while the other members write the characteristics of that level on worksheet 3b.

Allow 12 15 minutes for this activity.

While participants are working, write Unsatisfactory, Basic, Proficient, and Distinguished at the top of a piece of chart paper and display the slide.

34Conclusions: Levels of Performance35Failing: Potential for harm

Needs Improvement: Inconsistent, novice

Proficient: Consistent, competent

Distinguished: Unusually excellent, no one lives here permanently in all components

Conclude: by summarizing each level of performance as follows and inviting participants to add to their charts if necessary: :

Unsatisfactory is dangerous. Harm to students can be done at this level: physical, social, emotional or cognitive harm are possible at this level of teaching performance.

Basic is the novice level. Its hallmark is inconsistency: sometimes the teacher gets it right, and sometimes not. Even experienced teachers can have basic characteristics in their practice for a time if their teaching assignment changes significantly.

Proficient is the level at which we expect most of our experienced teachers to function. This is good, solid, consistent teaching.

Distinguished is the exceptional level of practice. No one lives at this level, nor has every characteristic of this level in all components, because that would mean perfection in teaching, and no teacher is perfect. Distinguished is not just good, or even excellent, teaching. It is teaching that sets itself apart as unusual in its outstanding level of excellence

We can all grow and improve, so while we have excellence in our practices sometimes, it is not a place we live all the time in all components. 35Component C of Domain 3: Engagement3a: Communicating with Student

3b: Using Questioning and Discussion Techniques

3c: Engaging Students in Learning

3d: Using Assessment in Instruction

3e: Demonstrating Flexibility and Responsiveness

36Show: The slide, Components of Domain Three and say:Engaging Students in Learning, is the most important component in the entire Framework.

Ask participants to chat at their table groups about the difference between time-on-task and engagement. Allow them @ 2 minutes to do so.

Then say:

Time-on-task means doing what the teacher asks you to do, regardless of whether the task is challenging or not. . Engagement , on the other hand, is brain sweat, or real, challenging thinking. Many times students are on-task, that is, they are doing what the teacher asked them to, but that task does not require real, challenging thinking by each student. Therefore, the students are on-task, but not engaged, which often explains why teachers teach and students dont learn. Engagement, then, is different from time-on-task, and is about what the brain is doing and whether it constitutes rigor.

Engagement is harder to see than time-on-task. Engagement can be thought of as difficult, in a good way. Students would not describe the assignment as easy, or a piece of cake, when engagement is present. Another way to think if it might be mental tippy toes.

361A Framework for Teaching:Components of Professional PracticeDomain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesParticipating in a Professional CommunityGrowing and Developing ProfessionallyShowing ProfessionalismDomain 3: InstructionCommunicating with StudentsUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSetting Instructional OutcomesDemonstrating Knowledge of ResourcesDesigning Coherent InstructionDesigning Student Assessments Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical Space37Display slide: The Framework for Teaching, Components of Professional Practice, and remind participants that previously we have overviewed the four domains, and have studied Domain 3, an on-stage domain, closely. In this activity, we will learn about an off-stage domain, Domain 1, Planning and Preparation. Point out that while Domains 2 and 3 are observable in the classroom (on stage), Domains 1 and 4 require other ways of indicating a teachers skill (off stage). Artifacts, professional conversation, or portfolios may be used.

Be careful in your language: we cant say that no components of Domain 1 or 4 could EVER be seen during a lesson, but rather, that these components are more likely to be seen in other ways.

371Domain 1: Planning and Preparation381a: Demonstrating Knowledge of Content and Pedagogy

1b: Demonstrating Knowledge of Students

1c: Setting Instructional Outcomes

1d: Demonstrating Knowledge of Resources

1e: Designing Coherent Instruction

1f: Designing Student AssessmentsDisplay slide: Domain 1 Planning and Preparation and tell participants that Domain 1 contains the components of a good lesson plan, and briefly discuss the different components. These are the items that a teacher thinks about when planning a lesson. Mention that certainly teachers know more about content and pedagogy (D1a) than we can see in a given lesson. Likewise for D1b, D1d and D1f. Therefore, we like to talk to teachers from time to time, including before an announced observation, to dig a little deeper and to record some of what the teacher tells us. Ask participants to look over the levels of performance rubric for Domain 1 and ask them to read the distinguished level for each component.

Have them note that at the distinguished level of 1f, students help to design some of their own assessments, which helps to explain why this level is not typical, but is exceptional.

Allow 3 minutes.

38Component 1a: Demonstrating Knowledge of Content and PedagogyTeacher wrote a scholarly articleLesson plans/structure/content/relevanceTeacher explanation of probable Students misconceptionsTeachers answers to student questions during classTeacher presented a workshop to facultyTeacher explains the structure of discipline prior to lessonTeacher tells observer how this lesson fits into the larger unitTeacher adjusts the lesson midstream based on Students misconceptionsTeacher poses different levels of content questions during the lessonTeacher states how this lesson connects to content standards39The goal of this slide is to assist participants to begin to identify the kinds of facts teachers share that can be evidence for Domain1.

Tell participants that Domain 1 is about the back story of teaching. Its about planning, and all the things the teacher knows, but cant show, during the act of instruction. Teachers make hundreds of decisions for each lesson that they plan. We dont expect them to write all this information, but when we talk to them from time to time, we can collect some of it, by writing down what the teacher tells us.

Invite participants to think about the on-versus-off stage concept. Have them read SILENTLY the above 10 statements, and think about which could be seen during teaching (on-stage) and which could not.

Allow a minute or two, but enforce silence while everyone reads and thinks. When everyone is ready, show the next slide.

Participants have this slide as a handout and can circle their answers on the paper.1Component 1a: Demonstrating Knowledge of Content and PedagogyTeacher wrote a scholarly articleLesson plans/structure/content/relevanceTeacher explanation of probable Students misconceptionsTeachers answers to student questions during classTeacher presented a workshop to facultyTeacher explains the structure of discipline prior to lessonTeacher tells observer how this lesson fits into the larger unitTeacher adjusts the lesson midstream based on Students misconceptionsTeacher poses different levels of content questions during the lessonTeacher states how this lesson connects to content standards40When you show this slide, mention that the red-colored statements are related to what would happen during teaching, so they are Domain 3 items, not Domain 1 items.

On the other hand, the rest of the items are things that would NOT occur during the teaching of the lesson, but could happen before or after. They give us more insight into what a teacher knows and thinks about when planning a lesson.

Tell the audience that our next activity will give us the opportunity to review the kinds of questions we might ask a teacher to learn more about Domain 1 in their practice. 1Worksheet #4 Participant Materials41 Read the Domain 1 questions.

Why are they important?The goal of this activity is for participants to learn the kinds of information about the components of Domain 1 that a teacher possesses, but that we dont always see easily.

A second goal is for the audience to understand that unless teachers are given an opportunity to respond to questions about the hidden parts of their practice, we cannot know about them.

Direct participants to the worksheet, found on page 7 of Participant Materials.

Explain to participants that this form contains Domains 1 and 4, and questions to elicit information from the teacher about them.

Review the form and the questions it contains. Say that when we ask these types of questions, we get at some of the teachers knowledge that might not be visible during the teaching of the lesson.

Allow a few minutes for people to discuss the questions, and some of the kinds of answers teachers might give. Tell participants to jot dot (bullet, briefly) some of the kinds of information a teacher might provide in response to the prompts for Domain 1 ONLY. (We will study Domain 4 later)Allow 10 minutes.

Walk around as participants work to see if they are providing the kinds of background info about that component that might not be seen during a lesson. For example, for component 1b, teachers know more about students than we might see during one lesson. What kinds of things are these? How does the teacher get information about the students? How is this used in planning lessons?Activity: Collecting responses

If time permits, invite participants to get up, move about the room, and collect some teacher responses for one or more of the components of Domain 1 from someone in the room with whom they dont usually work.

Allow 5 minutes.

41Lesson Plan Data Collection Tool Worksheet #4

Practice Evidence Collection

Write what is said - not what you think about it/your interpretation of it.42

The purpose of this slide is to let participants know that observers can record teachers responses to the questions on page 7 of participant materials

Tell participants that we will be using this tool for collecting evidence of Domain 1 to collect some facts about this shy domain from a pre-observation conference. Explain that the pre-observation conferences purpose is to elicit some of this otherwise invisible information.

Say that we will now watch a short video clip of a pre-observation conference for a middle school math teacher. In it, we should hear some facts from him about his students, his lesson, the unit in which it is situated, etc. We will write down some of this information as we hear it, in the correct component box on pg. 6.

Tell participants that they are to write what they hear, not editorial commentary, advice or opinion.

Now, show the pre-observation conference clip of High School Social Studies Pre-Observation video from the ASCD/Danielson/Framework video. (Be sure to show ONLY the pre-observation conference, not the lesson) and ask participants to write as they watch. (You should collect evidence also.)

At the conclusion of the pre-observation conference, invite participant to share the evidence they collected and to write down any evidence they hear from others that they did not hear or have time to collect.

Close this activity by reminding that Domain 1 is off stage and that when a teacher talks about it and we capture some of that information, it becomes more visible. This is the reason for the pre-observation conference.

You will have other opportunites to practice collecting evidenceYou cant hear cant 1A Framework for Teaching:Components of Professional PracticeDomain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesParticipating in a Professional CommunityGrowing and Developing ProfessionallyShowing ProfessionalismDomain 3: InstructionCommunicating with StudentsUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSetting Instructional OutcomesDemonstrating Knowledge of ResourcesDesigning Coherent InstructionDesigning Student Assessments Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical Space43Display slide of Components of the Framework. Remind participants that they have already studied two domains of A Framework for Teaching, Domains 3 and 1 respectively. Explain that we are now ready to explore the second on-stage domain of the framework, Domain 2, and that we will do so using a graphic organizer structure and by answering a focus question in a modified jigsaw fashion.

431Domain 2: The Classroom Environment2a: Creating an Environment of Respect and Rapport

2b: Establishing a Culture for Learning

2c: Managing Classroom Procedures

2d: Managing Student Behavior

2e: Organizing the Physical Space44Discuss each component briefly, mentioning that Domain 2 is sometimes thought of as the affective domain of the framework, because the level of the teachers performance in components 2a, Creating an Environment of Respect and Rapport, and 2d, Managing Student Behavior, contribute significantly to how students feel about school and about themselves. Component 2b: Establishing a Culture for Learning, also speaks to the teachers beliefs about students abilities to learn; therefore, Domain 2 contributes significantly to students emotions toward school and learning.

Other points to be made about the components:2a is not only about how the teacher treats the students and vise versa, but also about how students treat each other.Another way to think about 2b is that its all about rigor and setting high expectations2c has to do with extracting maximum time for teaching/learning, by managing procedures seamlessly2d is done best when its invisible2e is related to making the physical space match the learning goals

44Worksheet #5 Participant MaterialsBrowse Domain 2 of your Rubric

Reflect and answer questions on Worksheet #5Independently

Table Share45Browse Domain 2 of your RubricReflect and answer questions on Worksheet #5IndependentlyTable Share

45A Framework for Teaching:Components of Professional PracticeDomain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesParticipating in a Professional CommunityGrowing and Developing ProfessionallyShowing ProfessionalismDomain 3: InstructionCommunicating with StudentsUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSetting Instructional OutcomesDemonstrating Knowledge of ResourcesDesigning Coherent InstructionDesigning Student Assessments Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical Space46Explain to participants that the red-colored components are those we have already studied. We are now ready to investigate Domain 4, its relationship to student learning and its importance to novices and to experienced teachers. Remind participants that Domain 4 is the other off-stage domain of the framework. Its safe to say that Domain 4 is virtually never visible during the act of teaching, yet each component is connected to student learning in specific ways.

46147 Domain 4: Professional Responsibilities 4a: Reflecting on Teaching4b: Maintaining Accurate Records4c: Communicating with Families4d: Participating in a Professional Community4e: Growing and Developing Professionally4f: Showing ProfessionalismBriefly discuss each of the components of Domain 4, making the following points:

The role of reflection in adult learning is well-documented. However, many teachers do not reflect in the ways that produce learning.Recordkeeping is designed to inform teachers about trends and patterns in learning, both individual and group, so that instruction can be modified appropriately.Notification is not communication as it is intended in this component.Professional learning communities (PLCs) are an important support for teacher learning4f primarily focuses on advocacy for students who are traditionally underserved47Worksheet #4 Participant Materials48Skim the rubrics in Domain 4.Have a table conversation about HOW these components might impact student learning.Develop a Distinguished response for one component.The goal of this activity is for participants to learn the evidence of Domain 4 in a teachers practice.

Ask participants to locate and read their rubrics for Domain 4. Allow about 5 minutes for this study.

Then invite participants to turn and talk to someone near to them about how each component of Domain 4 might impact student learning. Allow 3 minutes, or more, if necessary.

Then direct participants to their Participant Materials, page 7. We have worked on the left-hand side of this page, Domain 1, earlier. Now we will revisit it for Domain 4.

Remind participants that Domain 4 is the other off-stage domain. We would not see these components during teaching, but questions posed to the teacher can elicit information about his/her practice in this Domain.

After getting into expert groups a through f - participants to read the prompt associated with each D4 component of their expert group and write a possible excellent answer to each work as a group. They should reference their rubrics for Domain 4 at the proficient or distinguished level as they do so. ** Modification for large group have participants take a number from their supply box. The following numbers will work on the following components 4a 1 & 7; 4b 2 & 8; 4c 3 & 9; 4d 4 & 10; 4e 5 & 11; 4f 6 & 12.

Allow @ 12 minutes. Select some non-volunteers to share one or more of their responses. Make clear, in closing, that teachers can answer these questions during pre- or post-teaching conferences, to provide more evidence about their practice. 48Uses of A Framework for Teaching49Self-AssessmentReflectionPeer CoachingTeacher EvaluationMentoring and InductionProfessional Growth PlansNow display the slide, Uses of the Framework and remind participants that the Framework articulates what good teaching looks like and what it doesnt look like. It provides us with a common language, a roadmap for the terrain, so to speak. Use of the Framework for Teaching for these various purposes tends to elevate the work .49Benefits of Using a Framework for Teaching50Common languageSimilar vision for good teaching and how it can be improvedGreater validity and reliability in the teacher evaluation processChanges in novice thinkingOpportunities for collaborationInvite participants to consider the slide for 30 seconds or so without discussion. Then invite volunteers to share any observations they have about the benefits of using the Framework within teaching and school practice. 50Doing Teacher Evaluation Right 515 Rules/Best Practices

Evidence

5152Why Evaluate Teaching?Quality Assurance

Professional LearningTell participants that we will be exploring how to use best practices for teacher evaluation, using the PA Teacher evaluation process.

Remind them that there are two purposes for teacher evaluation as state on the slide, but that mostly we focus only on the first one: getting a grade

Best practices research focuses on the second one, professional learning, because most teachers practices are good. Still we all need to grow, since all students can always learn more, so the practices we will experiences are not focused on gotcha, but on how we acknowledge strengths and continue to grow. 5253Doing Teacher Evaluation RightOBJECTIVES: Participants will learn

The role of the Framework in teacher assessment

Best practices in teacher evaluation

How to conduct teacher evaluation to accommodate quality assurance and professional learning, too

How to differentiate for novices, experienced teachers, and at-risk teachers

How to create buy-in for all stakeholders.

Invite the audience to read the goals silently and to mentally identify the one(s) that are most meaningful for them.

In your packet is an article entitled the widget effect that you might wish to use with your teachers. When providing professional development or sharing it highlights the major reasons to improve our evaluation processes and provides suggestions on how to do so. You can find the article in the resource section of your binder.5354The Widget EffectCount off by A,B,C,D at your table

Read the following sections: A p9 and pp 10-14 B p9 and pp 15-18 C p9 and pp 19-23 D p9 and pp 26-30

Then identify the 3 most important facts from your section The goal for this activity is for participants to learn about the flaws in traditional teacher evaluation processes.

Arrange participants in groups of four or more participants, and ask them to count off by a, b, c, and d. As each person to then read the pages noted on the slide. Encourage them to highlight or underline the most important components in their sections, and then select the three MOST important.

Participants are to work alone. Enforce quiet so that all can read and study.

Allow 15 minutes or so for the reading.5455The Widget Effect Share and LearnAt your table, have each person share the three most important ideas culled from the reading.

Mark your own copy accordingly.

Discuss your thinking.Once everyone has finished reading, invite the participants at their tables to share their three most important facts. Ask the other table members to mark their articles accordingly.

After completing this, as them to discuss their thinking about the article and why it is a good decision for Pennsylvania to move toward a more rigorous, research-based process for teacher evaluation.

Allow 15 minutes for this activity.5556Card SortParticipant MaterialsRead your card

Using a Post-It note, identify:DomainComponentElement

Share at your table as instructed; come to consensus

NOTE: Prior to this activity, you will need to distribute the colored Card Sort cards, so that there is one card for each person and that each table group has cards of all four colors.

The purpose of this activity is for participants to revisit the Framework for Teaching and to connect an individual scenario to one domain/component/element and level of performance.

Ask them to work alone for a couple minutes to read their scenario and to assign it to a domain/component/element of the Framework (using p10 of the Participant Materials), and to write this on their post-it note and affix it to their card..

While there may be more than one possible connection, they are to select the best one. Do not let them confer at this time, but rather, work alone. Allow @ 3 5 minutes.

At the end of this time, Tell participants that the color of the cards is significant: Yellow=Domain 1, Blue=Domain 2, Pink=Domain 3, and Green=Domain4. Invite correction on their post-its at this time.

Next, ask participants to share their scenarios with their table group, and develop consensus about the best answer. 5657RewriteSelect one scenario at your table

Determine tentative Level of Performance

Rewrite at higher and lower levels using rubric characteristicsNow, tell participants to select one of the cards, and to think about what level of performance - Unsatisfactory, Basic, Proficient or Distinguished - that scenario would represent if it were typical of that teachers overall practice.

Allow them to have a table conversation about ONE scenario, using their rubrics, to begin to circle around the appropriate level of performance, for about 3 -4 minutes.

Now, tell them that if they found some language in, say, the basic level and some language in the proficient level, that both apply to the scenario, they should feel free to use both: TEACHING BEHAVIOR DOESNT JUST LIVE IN ONE LEVEL OF PERFORMANCE, it is most often a mixture.

Ask them to write on a second Post-It the phrases from one or more levels of performance that match with the scenario they chose, and to affix this to the Post-it.

57585 Rules for Teacher EvaluationDefensible definition of teachingDifferentiation of evaluative processesEvidence-driven processThe role of teacher learningTransparencyInvite participants to notice the five best practices/rules for doing teacher evaluation right.

Explain that we will we working on each of these practices during the rest of our time together. 5859Overarching QuestionsWho does the thinking?

Who does the learning and growing? Tell participants that, in the classroom, we know that the work of learning needs to be done by the students themselves, not by the teacher. That is, we look to see who is doing the real thinking in classrooms, and we know that when students do it, real learning happens.

In teacher evaluation historically, the position of the teacher within the evaluation process has NOT been that of learner: Teachers have tended to be passive: the observer collects the information, analyzes it, and presents the conclusions to the teacher who primarily listens.

If teachers are to grow their already competent practices, then THEY need to do some of that work: collecting, analyzing and concluding. This changed role of the teacher makes some of the traditional teacher evaluation practices look quite different, which we will see as we continue our learning. 5960Rule # 1: Defensible Definition of TeachingStart with a defensible definition of good teaching that is studied, and understood, by all stakeholders.Tell participants that we spent the earlier part of this training experiencing Pennsylvanias definition of good teaching, which is the Framework for Teaching, otherwise known as the research-based standards for teaching.

Now we will experience a quick review of that definition of practice. 6061A Framework for Teaching:Components of Professional PracticeDomain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesContributing to the School and DistrictGrowing and Developing ProfessionallyShowing Professionalism

Domain 3: InstructionCommunicating Clearly and AccuratelyUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSelecting Instruction GoalsDemonstrating Knowledge of ResourcesDesigning Coherent InstructionAssessing Student Learning Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical SpaceRemind participants that we looked at the four domains, and we learned that two of the four domains are off-stage and that, to learn more about these in a teachers practice, we can talk with the teacher, pose questions, examine artifacts, and so on.

6162Rule # 2:Differentiation of Evaluative Process

Differentiate the processes of evaluation for:

novices experienced teachers teachers at riskExplain that teachers are different the same ways that students are different: skill level and experience. Good teacher evaluation systems typically honor these differences by providing closer supervision and support to those teachers who need it and allow greater variation and independence to teachers whose practices show evidence of competence. 62

Differentiated Evaluation SampleNovice/Untenured

Very close observation and assessmentExperienced/Tenured

Presumption of professionalismAt-Risk

Not punitiveFormal and informal observation of teaching is key + teacher interviews + artifacts Structured process 1/3yr. Other years: informals + teacher interviews+ professional goal-settingIntensive, extensive team-based support based on persistent unsatisfactory performance in one or more components2 4 formal times per year; multiple informal observationsProfessional Goal-Setting: Choose from a list of rigorous, approved activitiesClear goals, outcomes, evidence and timelines anchorNo self-directed activitiesActivities produce evidence which is then evaluated Designed for the teacher who can, and wishes, to improve63Briefly mention that that most typical forms of differentiation include frequent novice observations (2 4/year) and experienced teachers being evaluation formally less frequently with alternate evaluations taking the form of Professional Growth Plans.

Say that Pennsylvania is in Phase I of its new evaluation program, so we dont yet know what differentiation will look like in its final form, but that novice must be evaluated more frequently than experienced teachers. This makes good sense, since we must make consequential decisions about these professions that will have a last impact upon school culture.

Make clear to participants that we have some suggestions/ideas for differentiation especially for experienced teachers and using the framework with the intent to reduce the workload for evaluators and increase the learning for teachers but we will hold that for follow up as this is for pilot of the process and we dont know what PAs system will look like at the moment.63Differentiated SupervisionGuidelines provided by PDE

Determine cycle of supervision consisting of Formal Observation and Differentiated Supervision

Recommend 3-4 year cycle

Determine which staff require Formal ObservationsIdentify Differentiated Supervision Modes that can be used in your district

MUST evaluate every teacher every year in each of the 4 Domains64655 Rules for Teacher EvaluationDefensible definition of teachingDifferentiation of evaluative processesEvidence-driven processTeacher learning integralTransparencyShow this slide and tell participants that we have now discussed two of the three best practices for doing teacher evaluation right. Next we will learn about evidence, a keystone of the process. 6566Rule # 3:Evidence Driven ProcessLet evidence-not opinion-anchor the processExplain that typical evaluation systems involve observers watching the lesson and then writing down what they thought about it. They write advice, opinion, suggestions, likes, dislikes, etc.

Say that this is not the practice in an evidence-driven system for two reasons:Evidence, or facts, form the foundation of good decision-making. We always need to collect the relevant facts first, then make decisions.

In a growth-focused model, the evaluator AND the teacher evaluate the evidence separately, and then compare their thinking. In this way, learning is done by the learner, the teacher.

66Evidence or Opinion?The teacher has a warm relationship with the students.The teacher said that the South should have won the Civil War.The table groups were arranged in 2 x 2 pods. The materials and supplies were organized well.Wait time was insufficient for student thinking.The teacher stated that students have learned to add 2-digit numbers in preparation for todays lesson. 6 students, questioned randomly, did not know the days learning goals.67Ask participants to read the statements on the slide SILENTLY and decide which are evidence and which are opinion.

Participants have this slide as a handout, so they can circle their responses before going over the correct answers.

After a minute or so, provide the correct answers:OEEOOEE

If any questions arise, explain, for #2, that it is a fact that the teacher said the words in #2. this makes it evidence.

Say further that words like warm relationship , organized well, or insufficient thinking are opinions, no descriptions of what was seen and heard. So, what made the observer THINK the relationship was warm? What was ssen and heard? What makes a room well-organized? What was seen and heard? 67Evidence or Opinion?The teacher has a warm relationship with the students.The teacher said that the South should have won the Civil War.The table groups were arranged in 2 x 2 pods. The materials and supplies were organized well.Wait time was insufficient for student thinking.The teacher stated that students have learned to add 2-digit numbers in preparation for todays lesson. 6 students, questioned randomly, did not know the days learning goals.68Ask participants to read the statements on the slide SILENTLY and decide which are evidence and which are opinion.

After a minute or so, provide the correct answers:OEEOOEE

If any questions arise, explain, for #2, that it is a fact that the teacher said the words in #2. this makes it evidence.

Say further that words like warm relationship , organized well, or insufficient thinking are opinions, no descriptions of what was seen and heard. So, what made the observer THINK the relationship was warm? What was ssen and heard? What makes a room well-organized? What was seen and heard? 68

Worksheet #6 Participant Materials69EVIDENCE

is a factual reporting of events, may include teacher/student actions and behaviors,may include artifacts prepared by the teacher, students or others,avoids personal opinion or biases,and is selected using professional judgment by the observer and/or the teacher.Allow participants a moment to read this slide, and then say that we will be doing a series of activities now to teach us about evidence and how to collect it.

Direct participants into their Participant Materials, Worksheet #6 -, Evidence vs. Opinion. And invite them to complete it as a table group. (If time is a factor, you may choose divide the assignment.)

Explain that the items in the left column are types of statements an observer might write based upon an observation. Some of them are evidence and some of them are not. They should decide if each statement is evidence or opinion and should write E or O in the column based on their decision.

If the statement is opinion, they should REWRITE the statements to be a statement of evidence.

Finally, for each statement, they should decide which domain and component of the Framework to which each statement refers and write that domain/component (d/c) in the final column.

Allow @ 10 minutes, more if necessary, then review the answers as a whole group.

(Note: there is no absolute correct answer to the domain and component connections for each item, but you should listen carefully for rationale and push for better thinking if an answer seems weak.

Tell participants if they need it when working with others, there is additional practice information found on p4 of Participant Resources.69705 Rules for Teacher EvaluationDefensible definition of teachingDifferentiation of evaluative processesEvidence-driven processTeacher learning integralTransparency7071Professional LearningLearning is done by the learner; it is mental WORK. - Charlotte Danielson

Who does the mental work in your evaluation process? 7172The Nature of Professional Learning:

Mental Work for TeachersReflection on practiceCollaborationSelf-assessmentSelf-directed inquiry (action research)Feedback based upon evidenceTell participants that when teacher evaluation is conducted intentionally to produce teacher learning, it deliberately incorporates the characteristics found on the slide.

Ask participants to discuss with a partner where in the process we have studied each of these characteristics are found.

Say that the methods we have recommended for doing this work are deliberate. They are required to assure that learning happens,. 7273Narrative-FreeEvaluationThe rubric contains the narrative.Select the language that matches the evidence.The teacher participates in language selection.The highlighter is the tool to use!A summative domain statement is optional.Finally, explain that this process avoids the traditional narrative.

A rubric might be thought of as a bank of potential narrative statements. We select the ones that best match with the evidence, by highlighting. Once we have done this, and attached the evidence to the rubric, no narrative is required.

7374Supporting Teachers CorrectlyDirectiveCollaborativeNon-DirectiveFrom evaluator to teacherBack and forthFrom teacher to evaluatorImmoral, illegal, dangerous, cluelessBoth have ideas to contributeThe teacher deserves to take the leadDrowningSwimmingChampionship Swimming74755 Rules for Teacher EvaluationDefensible definition of teachingDifferentiation of evaluative processesEvidence-driven processTeacher learning integralTransparencyIn closing, make it clear that an important aspect of this training is to make the process and the role of teachers clear.

It is expected that this process (or a representative version of it) will be shared with teachers in each school into which this work arrives. 7576Rule # 5: TransparencyTeachers must learn the rubrics and the process.

7677Involving All StakeholdersMany teacher evaluation systems fail due to resistance that comes from the perception that the evaluation system resulted from the secret efforts of an elite few.77Notification is NOT CommunicationCommunication is two-waynot one-way787879Rule # 5: TransparencyTeachers must learn the rubrics and the process.

How might this happen in your setting? 7980Observation-based Assessment: Process and EvidenceProcessEvidencePre-ObservationDomains 1 and 4Standard Lesson Plan with Components of Domain 1- Evidence provided by TeacherObservation:Domains 2 and 3Standard Evidence Collection Document Shared with TeachersPost-TeachingDomains: 1, 2, 3 and 4Teacher Self-Assessment, Rubrics and additions/correction of evidence gatheredCollaborative AssessmentDomains 1, 2, 3 and 4Evaluator Rubric and Teacher Self-Assessment RubricNow tell participants that we will consider evidence in light of the evaluation process.

Remind them that there are two off-stage and two on-stage domains, so we have to use a process that elicits evidence of all these components, not just domains 2 and 3 , which is typically the custom.

Direct participants now to the evidence collection form for Domains 1 and 4 found on p7 of the Participant Materials. Explain that this is a tool to use during the pre-and post-teaching conferences to collect the off-stage evidence as part of an announced observation. Say that good evaluation systems consist of a mixture of announced and unannounced observations, and that for the announced ones, collecting evidence about Domains 1 and 4 is an essential part.

Ask participants to read the questions associated with Domain 1 and to discuss why these questions are helpful. Make it clear that this is NOT a daily lesson plan document, it is intended for an announced observation, to elicit additional information about planning.

Note further that the most efficient process is for the teacher to provide this completed document, (all D4 components need not be answered) to the principal PRIOR to the pre-observation conference and then can be added-to during the pre-observation conference.

8081

Worksheet #7 Participant MaterialsWatch the pre-observation conferenceCollect evidence on your Lesson Plan formDocument evidence only no opinions, please!Prepare to show the video by asking participants to locate their lesson plan form on p13 of Participant Materials. Tell them they will write down what they hear the teacher saying about his upcoming lesson.

Show the video clip of the ASCD Framework video for the 6th grade middle school math lesson, 2 minutes in length.

Afterward, ask participants to compare the evidence they collected. Allow about 3 4 minutes.

Invite them next to discuss what questions they would pose to the teacher if they could speak with him, and point out that were they able to do so, they would record this evidence on this form as well.

Make sure that participants recorded under component 4e the information that the teacher offered about his professional development with the Smartboard.

Say that it is important to use the pre-observation conference to elicit as much information about the upcoming lesson as possible.

81EvidenceObservation-based Assessment: Process and EvidenceProcessEvidencePre-ObservationDomains 1 and 4Standard Lesson Plan with Components of Domain 1- Evidence provided by TeacherObservation:Domains 2 and 3Standard Evidence Collection Document Shared with TeachersPost-TeachingDomains: 1, 2, 3 and 4Teacher Self-Assessment, Rubrics and additions/correction of evidence gatheredCollaborative AssessmentDomains 1, 2, 3 and 4Evaluator Rubric and Teacher Self-Assessment Rubric82Next, direct attention to the evidence collection process once more and point out that after the pre-observation conference, the observation is conducted and evidence is collected during it.. Ask them to review the evidence collection document found on page 18 of their materials.

Say that this is the tool, (or the PA counterpart) that should be used to collect evidence DURING the observation.

We will be asked to stretch ourselves and learn to collect evidence in real time and not to recopy it. Mention briefly that after step 2, this evidence is PROVIDED TO THE TEACHER. It is just facts, not opinion or judgment, and this step is crucial to the process.

Then the teacher uses the evidence to conduct a self-assessment on the rubrics, prior to the post-teaching conference.

We will be learning more about this process now as we practice collecting evidence.

Direct participants to page 14 Worksheet #8 of their participant materials and reinforce that this is the evidence collection tool for the on-stage aspect of this process, the classroom observation. Point out that it contains spaces to write evidence for Domains 2 and 3 and that it is important to write the evidence we see during the observation ON THE DOCUMENT, not someplace else. This practice improves reliability, so ask participants to put away their other materials and to prepare to write on page 14 as we watch the video of the lesson for the middle school math teacher.

8283Types of Observation EvidenceVerbatim scripting of teacher or student comments: Could one person from each table collect materials?

Descriptions of observed teacher or student behavior:The teacher stands by the door, greeting students as they enter.

Numeric information about time, student participation, resource use, etc.:Three students of the eighteen offer nearly all of the comments during discussion.

An observed aspect of the environment:The assignment is on the board for students to do while roll is taken. Say that we are moving now toward collecting evidence8384

Worksheet #8 Participant MaterialsWatch the lessonCollect evidence of what you see and hearIf you arent sure where to write the evidence, just write itThis is practice; relaxAs you prepare to watch the video of the middle school math lesson for which you collected evidence of planning, ask participants to skim the components of D2 and D3, focusing on the priority components which are shaded.

Say that they should collect evidence of the priority components, and, if they wish, any other components for which they are able.

Remind them that evidence is what you see and hear; not what you think about it. We are describing only.

Show the video of the lesson and collect evidence with the participants.

Following the lesson, ask participants to sit quietly for a few minutes reviewing their evidence and to add any evidence that they didn'tt have time to record.

After 3 4 minutes, ask participants to compare their evidence with others sitting near them, and to add any evidence they missed, and to reorganize their evidence (arrows are fine) based on discussion.

Allow @ 10 minutes for this activity.

84Points about EvidenceAll questions are not about 3b

Engagement is about the nature of the work and who does it

Formative assessments should assess whether EACH student met the objectives. 85Help participants to think about the points on the slide.85EvidenceObservation-based Assessment: Process and EvidenceProcessEvidencePre-ObservationDomains 1 and 4Standard Lesson Plan with Components of Domain 1- Evidence provided by TeacherObservation:Domains 2 and 3Standard Evidence Collection Document Shared with TeachersPost-TeachingDomains: 1, 2, 3 and 4Teacher Self-Assessment, Rubrics and additions/correction of evidence gatheredCollaborative AssessmentDomains 1, 2, 3 and 4Evaluator Rubric and Teacher Self-Assessment Rubric86Direct participant attention to the next step in the process: sharing evidence and teacher self assessment. Explain that in a growth model for teacher evaluation, BOTH the teacher and evaluator assess the lesson, and they do so based on the evidence that is shared.

Once the teacher receives the evidence, he/she is invited to add to it, optionally.

Then, using the rubric, the teacher self-assesses the lesson for the priority components and any others for which there is sufficient evidence. And then provides this assessment to the evaluator, IN ADVANCE of the post-teaching conference.

This is a significant departure from traditional practice, so invite participants to turn and talk to someone at another table about this process to check their understanding.

Now, show participants the self-assessment p. 15-19 of the teacher whose lesson we just watched and ask them to react to it in their table groups: where do they agree with the teachers analysis of the evidence? Where do they think differently?

Allow @ 20 minutes for them to discuss their thoughts about the teachers analysis of the evidence. Remind them that they are not invited to share their own opinions that are not supported by the evidence. 86After the ObservationShare a copy of the evidence gathered with the teacher, and give the teacher a copy of the rubric to self-evaluate. The teacher will return these to you prior to the post-observation conference.

Review the evidence and evaluate the teachers lesson using the rubric.

Compare the teachers self-assessment to your evaluation. Use the areas of non-agreement to guide the post-observation conference.

87Tell participants that the next part of the process of evaluation is for the evaluator to review the teachers self-assessment, as we just did, and then to mark the components of agreement only on their own rubric.

They are to work as a table group and to locate their own rubric in their participant materials, pp. 23 26, the priority rubric components. They are to consider, for now, just these components, in their work.

Note: this is very difficult for participants. They seem to automatically want to mark all components, whether they agree with the teachers self-assessment or not. You will need to watch closely to see that participants are doing this work correctly. 8788Conclusions about the ObservationWhere did your group mark the lesson at or above proficient?

Where did your group mark the lesson below proficient?Key points to make about the lesson:

In general, planning for this lesson (or the evidence we have of it) was not especially strong. Assessment was not planned for; nor were the objectives particularly specific.

Overall, Domain 2 was a strength with the exception of 2b.

2b Culture for Learning, and 3c Engaging Students in Learning are NOT strengths in this lesson because:The lesson relied almost solely on volunteers.If a student did not volunteer, s/he could have done nothing in the lesson except review the three division problems in the Do Now.Listening is not learning.The structure of the group activity allowed for one person to do all of the learning.Most of the activity was at the lower end of Blooms Taxonomy.8889Paradigm ShiftWho collects/provides evidence?Both teacher and evaluator

Evaluation is not done TO you; it is done with you and for youPoint out that this growth-focused process of evaluation positions the teacher as an active participant in the process. Ask participants to turn to an elbow partner and describe where, in the process, the teacher provides evidence and elicit the following:The lesson plan and pre-conference Following the observation, teacher may add to the evidenceDuring the post-conference.

89Concluding about the LessonWhere did your group mark the lesson at or above proficient?

Where did your group mark below proficient? 90WRAP UP refer to two resources Flowchart on Pg. 3 Resources and the Teacher Observation Map. of the process in resources pg. 4

Make the key points here about the lesson:

In general, planning for this lesson (or the evidence we have of it) is not especially strong. Assessment was not planned for; objectives were not particularly specific.

Overall, Domain 2 was a strength, with the exception of 2b.

2b, Culture for Learning, and 3c, Engaging Student in Learning are not strengths of this lesson, because:-The lesson relied on volunteers almost solely-If a person did not volunteer, s/he could have done nothing in the lesson except the three review division problems of the Do Now-listening is not learning-The structure of the group activity allowed for one person to do all the learning-Most of the activity was at the lower end of Blooms

90EvidenceObservation-based Assessment: Process and EvidenceProcessEvidencePre-ObservationDomains 1 and 4Standard Lesson Plan with Components of Domain 1- Evidence provided by TeacherObservation:Domains 2 and 3Standard Evidence Collection Document Shared with TeachersPost-TeachingDomains: 1, 2, 3 and 4Teacher Self-Assessment, Rubrics and additions/correction of evidence gatheredCollaborative AssessmentDomains 1, 2, 3 and 4Evaluator Rubric and Teacher Self-Assessment Rubric91Tell participants to revisit the steps in the process. Remind them that:

Teachers get a copy of the evidence immediately following the lessonTeachers may add to the evidenceTeachers use the evidence to complete a self-assessmentTeachers assess the lesson by highlighting the appropriate rubric phrasesTeachers provide this assessment TO THE OBSERVER IN ADVANCE OF THE POST TEACHING CONFERENCEThe observer review the teachers evidence prior to the post.The observer highlights, on his/her rubric the COMPONENTS OF AGREEMENT ONLY prior to the postThe observer LEAVES BLANK the components of difference prior to the post9192The Purpose of the Post-Observation ConferenceTo discuss the components of difference (not yet marked by observer)

To elicit any evidence that still remains to be added about the lesson

To arrive at an assessment on the rubric for components of difference.92Example:Post-Observation ConferenceEvaluatorTeacher93Use two People (Eric and Rosanne) to read the Post-Observation Conference (this is the poor example)93What were your overall impressions of the conference?

What suggestions/feedback did the observer make to the teacher?

How did the teacher respond?Individually:9494What suggestions/feedback did the observer make to the teacher?

How did the teacher respond?

What changes in Teacher Practice might you expect to see as a result of this conference?How would you know?Debrief as a Group:95Facilitators conduct what might be a typical post-conference using the script provided as a framework. USE THE TRADITIONAL POST-CONFERENCE HANDOUT provided

It is CRITICAL that the suggestions in the script be used as they serve as the basis for upcoming slides and discussion points.

95Words NOT to Use in the Post-Observation ConferenceDefendProveArgueConvince

Avoid language that suggests opposition of that might bring about a defensive response9696Language for the Post-Observation ConferenceSay more about. . .Comment on the evidence for. . .Lets look at the rubric for. . . What is the best match for. . .?Whats the backstory for. . .?

9797Framing Suggestions as QuestionsWho does the thinking? Who does the learning and growing?

98The goals of this activity is for participants to learn the skill of turning suggestions into questions.

Point out, again, that this work positions the TEACHER as an analyst of the evidence, of thinker, problem solver and generator of next steps.

Corrections and respectful differing thinking are encouraged on both sides, but evaluators are asked to avoid the telling model, since telling is not the best way to produce learning.

Tell participants that we will now explore ways to elicit FROM THE TEACHER suggestions and next steps.

Direct participants to their participant materials, page 26 Worksheet 11, and tell them change the advice they see into a question.

Work the first example with the group: Instead of Try calling on genders more equally, a question might be, What are some strategies you might use to insure that boys and girls get called upon equally? Invite the group to work with a partner to do the same with the remaining examples.

Process by calling upon non-volunteers for responses.

98Walk-Through999/15/2014Region 599Domains 1 and 4Domains 1, 2 and 3Domains 1, 2, 3, and 4Domains 1, 2, 3, and 4Agreed Upon Areas of Focus100Point out that the Framework for Teaching and the Teacher Evaluation Pilot can provide opportunities to implement change

That change can be in the form of professional development, changes in instructional delivery, classroom management, etcBut with the need to implement change comes the responsibility to monitor its effectiveness Walk-throughs can become a valuable part of this new process.100Strong evaluation systems include a series of brief, focused, unannounced, walk-throughs.The Walk-Through101Explain to participants that in addition to the announced observation process that includes the pre-/observation/post-, a valid evaluation process should include a series of brief, FOCUSED, unannounced walk-throughs.

101Could be based on an agreed-upon area of focus

Quick-take

5 15 minutes

Any Domain or Component

Formative Assessment not necessary to have conversations with teacher pre/post; nor to share documentation with teacher

The Walk-Through102These provide the teacher with a regular source of feedback, and offer the opportunity to compare the quality of teaching when it is an announced event with the quality when the event is unannounced.

The walk-through can be used as a way to provide feedback to the teacher on an agreed area of focus it is highly recommended that the walkthroughs take place after a formal observation and post-conference assessment so that the walk-through has a purpose.

The focus should be the one or two areas identified during the post-conference assessment as areas the teacher would focus his/her learning and growth

The walkthrough is a formative assessment of the progress a teacher is making toward the agreed upon goals/targets established in the post-conference

A couple of examples:

As an evaluator, you and one of your strongest teachers have just completed a post-observation conference and the teacher would like to focus on moving more towards distinguished in Domain 1, Component F Designing Student Assessments (teacher is Proficient)He/She has decided they could do a better job of differentiating assessments for student abilities and would like to attempt having students contribute to the development of assessmentsHe/She might agree to solicit from students what they think they should be able to demonstrate as proof of having accomplished a new learning.This would become the area of focus for your walkthrough

As an evaluator, you and one of novice teachers have completed a post-observation conference and the teacher would like to focus on 3b Engaging Students in Learning (You and the teacher agree the teacher is Needs Improvement/Progressing)He/She has decided they need to improve their questioning skills and ask questions designed to elicit thoughtful responses and questions that are at a much higher level of rigor.This would become the area of focus for your walkthrough

Remind participants:

The Post-Observation/Conference can be used as a means of establishing the area of focus for the walkthroughs.

You can do more than one walkthrough and it is in fact encouraged, if needed for formative assessment of teacher progress but for the purposes of the pilot, you are required to do only one per participating teacher

There is no requirement, only a strong recommendation that you conduct walk-throughs AFTER the formal observation.

This may or may not be the process that is similar to the one youre currently using in your district you may see some similarities and some differences with your process. This is what we are asking for the pilot and will elicit feedback on the process before moving forward with any actual implementation.

If you chose to do so before a Formal Observation, you must conduct a pre-conference interview to determine the agreed upon area of focus and the focus must be in one of the 4 domains.

1021035 Rules for Teacher EvaluationDefensible definition of teachingDifferentiation of evaluative processesEvidence-driven processTeacher learning integralTransparency103104Rule # 4:Teacher Learning IntegralConduct evaluations in such a way that they produce teacher learning.104105Professional LearningLearning is done by the learner; it is mental WORK. - Charlotte Danielson

Who does the mental work in your evaluation process? 105106The Nature of Professional Learning:

Mental Work for TeachersReflection on practiceCollaborationSelf-assessmentSelf-directed inquiry (action research)Feedback based upon evidenceTell participants that when teacher evaluation is conducted intentionally to produce teacher learning, it deliberately incorporates the characteristics found on the slide.

Ask participants to discuss with a partner where in the process we have studied each of these characteristics are found.

Say that the methods we have recommended for doing this work are deliberate. They are required to assure that learning happens,. 106107Narrative-FreeEvaluationThe rubric contains the narrative.Select the language that matches the evidence.The teacher participates in language selection.The highlighter is the tool to use!A summative domain statement is optional.Finally, explain that this process avoids the traditional narrative.

A rubric might be thought of as a bank of potential narrative statements. We select the ones that best match with the evidence, by highlighting. Once we have done this, and attached the evidence to the rubric, no narrative is required.

107108Supporting Teachers CorrectlyDirectiveCollaborativeNon-DirectiveFrom evaluator to teacherBack and forthFrom teacher to evaluatorImmoral, illegal, dangerous, cluelessBoth have ideas to contributeThe teacher deserves to take the leadDrowningSwimmingChampionship Swimming1081095 Rules for Teacher EvaluationDefensible definition of teachingDifferentiation of evaluative processesEvidence-driven processTeacher learning integralTransparencyIn closing, make it clear that an important aspect of this training is to make the process and the role of teachers clear.

It is expected that this process (or a representative version of it) will be shared with teachers in each school into which this work arrives. 109110Rule # 5: TransparencyTeachers must learn the rubrics and the process.

How might this happen in your setting? 110111Involving All StakeholdersMany teacher evaluation systems fail due to resistance that comes from the perception that the evaluation system resulted from the secret efforts of an elite few.111Notification is NOT CommunicationCommunication is two-waynot one-way112112Concluding about the LessonWhere did your group mark the lesson at or above proficient?

Where did your group mark below proficient? 113WRAP UP refer to two resources Flowchart on Pg. 3 Resources and the Teacher Observation Map. of the process in resources pg. 4

Make the key points here about the lesson:

In general, planning for this lesson (or the evidence we have of it) is not especially strong. Assessment was not planned for; objectives were not particularly specific.

Overall, Domain 2 was a strength, with the exception of 2b.

2b, Culture for Learning, and 3c, Engaging Student in Learning are not strengths of this lesson, because:-The lesson relied on volunteers almost solely-If a person did not volunteer, s/he could have done nothing in the lesson except the three review division problems of the Do Now-listening is not learning-The structure of the group activity allowed for one person to do all the learning-Most of the activity was at the lower end of Blooms

113Observation is NOT Evaluation!Evaluation of teaching is the sum of a number of observations, artifacts, and conversations that, together, provide a clear picture of the teaching practice.114Suggested Evaluative EventsFor tenured staff:

Walk-through (September)Announced observation (October December)Walk-through (December January)Unannounced observation (February April)Walk-though (April June)

115Suggested Evaluative EventsFor non-tenured staff Evaluation #1

Walk-through (September)Announced observation (October)Walk-through (November)Walk-through (December)116Suggested Evaluative EventsFor non-tenured staff Evaluation #2

Walk-through (January)Unannounced observation (February)Walk-through (March)Walk-through (April)Walk-through (May optional)117Annotated Framework for TeachingExplore the document and:Note the benefits of using the Framework within teaching and school practice.How does it compare to the rubric itself?What is the value of this document for developing teacher effectiveness?11810 self reflection10 table talk5 minute share out

118Educators with Unique Roles and FunctionsExamples under the Instruction tab at www.pdesas.org

119

119Complete Framework for TeachingSelf-AssessmentReflectionPeer CoachingTeacher EvaluationMentoring and InductionProfessional Growth Plans120120121Building Evaluator ReliabilityReliability refers to similarity of conclusion/consistency

Consistency is a function of consensus-building activities

Evaluators must practice consensus-building activities regularly

Now, tell participants that we will continue to build our own reliability by collecting and comparing evidence for Domains 2, 3.

Make sure participants are using the evidence collection form for D2, D3 found on page 20-22of their materials.

Make sure that they write on this document when they collect evidence for the on-stage components for the elementary teaching lesson.

Show the video and collect evidence along with the group. 121Paradigm ShiftWho Collects/Provides Evidence?Both teacher and evaluator

Evaluation is not done TO you; it is done with you and for you122Point out that this growth-focused process of evaluation positions the teacher as an active participant in the process. Ask participants to turn to an elbow partner and describe where, in the process, the teacher provides evidence and elicit the following:The lesson plan and pre-conference Following the observation, teacher may add to the evidenceDuring the post-conference.

122123To summarize our work:Deeper meanings of the Framework for TeachingThe nature of differentiated teacher evaluationHow to collect accurate evidence of teaching and use it, with rubrics, to assess performanceHow to conduct teacher evaluations appropriatelyHow to make changes in teacher evaluation that reduce suspicion and distrust123Where to find information about educator effectiveness?PDE websitewww.education.state.pa.usClick