Teacher Evaluation for Teachers in
Non-Tested Grades and Subjects
A Pennsylvania PerspectiveCreated by O David Deitz for the PMEA Leadership Conference 7/16/12
(there are still a lot of questions to be asked and answered!)
?
House Bill 1901
• (B) FOR PROFESSIONAL EMPLOYES AND TEMPORARY PROFESSIONAL• EMPLOYES WHO SERVE AS CLASSROOM TEACHERS, THE FOLLOWING
SHALL APPLY:• (1) BEGINNING IN THE 2013-201 4 SCHOOL YEAR, THE EVALUATIO N• OF THE EFFECTIVENESS OF PROFESSIONAL EMPLOYES AND TEMPORARY• PROFESSIONAL EMPLOYES SERVING AS CLASSROOM TEACHERS SHALL
GIVE• DUE CONSIDERATION TO THE FOLLOWING:• (I) CLASSROOM OBSERVATION AND PRACTICE MODELS THAT ARE• RELATED TO STUDENT ACHIEVEMENT IN EACH OF THE FOLLOWING AREAS:• (A) PLANNING AND PREPARATION.• (B) CLASSROOM ENVIRONMENT.• (C) INSTRUCTION.• (D) PROFESSIONAL RESPONSIBILITIES.• (II) STUDENT PERFORMANCE, WHICH SHALL COMPRISE FIFTY PER• CENTUM (50%) OF THE OVERALL RATING OF THE PROFESSIONAL EMPLOYE• OR TEMPORARY PROFESSIONAL EMPLOYE SERVING AS A CLASSROOM
TEACHER• AND SHALL BE BASED UPON MULTIPLE MEASURES OF
• STUDENT ACHIEVEMENT.
Race to the Top
Goal:An effective
teacher in every classroom.
Building Level Data, 15%
Teacher Specific Data, 15%
Elective Data, 20%
Observation/ Evidence, 50%
Teacher Effectiveness System in Act 82 (HB 1901)Observation/Evidence
Danielson Framework Do-mainsPlanning and PreparationClassroom EnvironmentInstructionProfessional Responsibili-ties
Building Level DataPSSA AchievementPVAAS GrowthGraduation RatePromotion RateAttendance RateAP Course ParticipationSAT/PSAT
Teacher Specific DataPSSA AchievementPVAAS GrowthIEP GrowthLocally Developed District Rubrics
Elective Data/SLOsDistrict Designed Measures and Examina-tionsNationally Recognized Standardized TestsIndustry Certification ExaminationsStudent Projects Pursuant to Local Re-quirementsStudent Portfolios Pursuant to Local Re-quirements
5
Charlotte Danielson’s Framework for Teaching
Four Domains 22 Components 50%
Observation/Evidence
Danielson Framework Domains1. Planning and
Preparation2. Classroom Environment3. Instruction4. Professional
Responsibilities
Building Level Data
Building Level DataPSSA AchievementPVAAS GrowthGraduation RatePromotion RateAttendance RateAP Course ParticipationSAT/PSAT
•Building Level Data• PSSA Achievement• PVAAS Growth• Graduation Rate• Promotion Rate• Attendance Rate• AP Course Participation• SAT/PSAT
15 %
Teacher Specific/Elective DataTeacher Specific Data PSSA Achievement PVAAS Growth IEP Growth Locally Developed District Rubrics Elective Data/SLOs District Designed Measures and Examinations Nationally Recognized Standardized Tests Industry Certification Examinations Student Projects Pursuant to Local Requirements Student Portfolios Pursuant to Local Requirements
15%
20%
35%For NTGS: Elective
What are some ways to develop evidence of student achievement in non-tested
grades and subjects?
Student Learning Objectives
Other Assessments
Measures of Collective Performance
Three Approaches to the Challenge
Measures of Collective Performance
School-wide growth measures (like the PA Report Card)
Standardized assessments used to measure the performance of groups of
teachers (like program evaluation)
Other Assessments
Standardized end of course assessments, formative-benchmark-unit assessments
Teacher-developed assessments of student learning or growth
Student Learning ObjectivesContent- and grade/course-specific measurable
learning objectives that can be used to document student learning over a defined
period of time
Can be written to measure growth or mastery
• NY, IN, MA, GA, UT, RI, NH, New Haven CT, Austin TX
Student Learning Objectives
• FL, DE, Pittsburgh PAOther Assessments
• KY, Pittsburgh PAMeasures of
Collective Performance
What are other states/schools using for non-tested grades and subjects?
What model might be most appropriate for PA to develop?
Student Learning Objective Models
Models of Student
Assessment
Measurement of Student
Performance & Teacher
Effectiveness
What is PA planning to do?Student Learning Objectives with
Model Assessments
Why this approach?The “other assessment” approach causes
narrowing of curriculum and is not financially sustainable.
The “measures of collective performance” are already included in the formula (required by Race to the Top) and do not offer NTGS teachers an opportunity to be evaluated on what they actually teach.
SLOs that don’t describe the assessment protocol tend to lack a true learning focus.
The Approach
• Provide a method for teachers to write SLOs.• Provide a method for teachers to design
assessments appropriate to the SLO.• Provide piloted and peer-reviewed models on the
PDE SAS website.• Provide professional development appropriate to
SLO development.The Methodology: AN SLO TEMPLATE
Looking at the components of PA’s SLO Template…..
• Teacher Information• Course Information• Learning Objective• Data and Targets Used to Establish the SLO• Assessment/Performance Task• Administration of the Assessment/Performance Task• Data Collection• Implementation Timeline• Teacher Effectiveness Measure
Teacher Information
Teacher Name
School Name
District Name
…one piece at a time.
Course InformationCourse Title
Content Area
Grade Level(s)
Brief Course Description
Number of students per class/session
Number of minutes per class/session
Frequency of classes/sessions
Total number of classes/sessions
LO (Learning Objective)
(SLO) Student Learning Objective
SLO Rationale Why was this objective chosen? How do you know it is a worthy area of focus? What are the transfer goals? Based upon this objective, how will students be able to independently demonstrate
their learning?
Curriculum Framework Connection What are the big ideas, essential questions, concepts and competencies (as identified
by PA’s Curriculum Framework) to be measured?
Big IdeasEssential QuestionsConceptsCompetenciesStandards Alignment Based upon the big ideas, essential questions, concepts and competencies selected,
identify the content standards that are aligned to this objective.
Data and Targets Used to Establish the SLO
Awareness of student level of preparedness What, if any, are the course or grade prerequisites? What is shown by the current (baseline) data? Which students may need targeted objectives (e.g.
students with IEPs, diverse learners, etc.)?
Targets What is the expected outcome (goal) by the end of the
designated instructional period (i.e., year, semester, course length, etc.)?
Using student baseline data, how many differentiated targets/goals are needed?
Are the targets ambitious yet realistic?
Assessment/Performance Task Assessment/Performance Task Name
Description of the Assessment/Performance Task Describe the assessment/performance task that will be used to measure the learning
objective (LO). Consider any products or artifacts that will be gathered as part of the data collection
process. Identify the developer of the assessments/performance task used (e.g., teacher-made,
district-developed, commercial, etc.).
Assessment/Performance Task Objectives Rationale Describe how the assessment/performance task authentically reflects the learning
objective (LO). How is this assessment/performance task a reliable and valid measure of student
mastery or growth aligned with expected outcomes and goals? How does the assessment/performance task measure the ability of students to
independently apply what they have learned? How is the assessment/performance task free of bias? How do the assessment/performance task outcomes inform instruction? How does the assessment/performance task align with grade, school and district level
curricular goals?
Growth or MasteryDoes this assessment/performance task align with the student achievement goals for the LO?
Growth (change in student achievement across two or more points in time) Mastery (attainment of a defined level of achievement) Growth and Mastery
Administration of the Assessment/Performance Task
Frequency of Assessment/Performance Task Administration How often and when is this assessment/performance task administered? If measuring growth, are multiple assessment windows in place?
Resources Required What unique or specific equipment, technologies, or resources are needed to
complete this assessment/performance task?
Adaptations for Diverse Learners and/or Students with Disabilities What adaptations built into the assessment/performance task are needed
assist diverse learners and/or students with disabilities?
Personnel Can this assessment/performance task be administered by an equivalent
peer? (If not, please explain.) Does a district policy exist with regard to assessment/performance task
administration?
Data Collection
Measuring Student Progress/Preparation How was baseline data collected? (If baseline data was not collected, please explain.) Can baseline data be compared with the results of this assessment/performance task?
Data Collection In what format will data be collected (e.g., database, graphed, portfolio, etc.)? Is a pre-post test being used? (If so, please describe.) How frequently will data be collected?
Rubrics and Scoring Scales How will individual student growth or mastery be determined (defined and scored) using this
assessment/performance task? Include the specific rubric/scoring scale that will be used. Does the rubric and/or scoring scale correlate with the assessment/performance task?
Data Interpretation How can the assessment/performance task results be interpreted in the same way across
equivalent peers? Is there a reliable and valid scoring and interpretive process (i.e., state developed, district-based,
commercial, standardized, etc.) that is associated with the assessment/performance task? If so, please describe.
Implementation Timeline
Date SLO is due to principal
Date(s) for Assessment and Data Collection
Dates to complete Data Interpretation
Date to present Teacher Effectiveness Measure
Teacher Effectiveness Measure
Classroom Objective How will the aggregated scores of individual student assessment results be used to define teacher effectiveness?Failing: few students achieve content mastery or growth
Needs Improvement: less than a significant number of students achieve content mastery or growth
Proficient: A significant number of students achieve content mastery or growth
Distinguished: An exceptional number of students achieve content mastery or growth
Targeted ObjectiveHow will the mastery or growth of targeted student populations be described and used to define teacher effectiveness?
Failing: Did not meet goal, little to no student mastery or growth
Needs Improvement: Did not fully meet goal but showed some student mastery or growth
Proficient: Met goal or otherwise demonstrated significant student mastery or growth
Distinguished: Surpassed goal or otherwise demonstrated significant student mastery or growth
Targeted Population: Mastery and/or growth goal:
Teacher Effectiveness RatingsWhat were the results of the assessments/tasks and how do they relate to the classroom and targeted objectives?Classroom Objective Failing Needs Improvement Proficient Distinguished
Notes/Explanation
Targeted Objective Failing Needs Improvement Proficient Distinguished
Notes/Explanation
SLO Process DimensionSLO –Assessment Developers-ExpertiseWho was involved in establishing the SLO and what is
their expertise?
Exemplars of Student Work
Let’s see how we can apply this template to a music teaching
setting.(Since choir is what I know the best,
I’ll do a choir SLO. With a little reworking, this SLO would be
applicable to other performance situations.)
Teacher Information
Teacher Name O David Deitz
School Name Teacher Effectiveness Demonstration
District Name PA Dept of Education
Course InformationCourse TitleChoirContent AreaMusicGrade Level(s)6-8, 9-12
Brief Course Description
Secondary level music course that provides opportunity to develop and apply the skills, techniques and knowledge appropriate to choral singing. An integral part of the course is the on-going preparation for public concert performances; in and out of school.
Number of students per class/session
30-70
Number of minutes per class/session
40
Frequency of classes/sessions
3 times per week
Total number of classes/sessions
108
LO (Learning Objective)(SLO) Student Learning ObjectiveStudents will develop independent performance skills appropriate to positive musical contribution to a choral performing ensemble.
SLO Rationale Why was this objective chosen? How do you know it is a worthy area of focus? What are the transfer goals? Based upon this objective, how will students be able to
independently demonstrate their learning?
The quality and musicality of a choir is dependent on each individual as a contributor to the collective whole. Individual assessment informs each students of his/her progress and contribution as an independent musician and (hopefully) as a life-long participant in choral singing.
Curriculum Framework Connection What are the big ideas, essential questions, concepts and
competencies (as identified by PA’s Curriculum Framework) to be measured?
Big Ideas The skills, techniques, elements and principles of the arts can be learned, studied, refined and practiced.
Essential Questions
Why is it important to be able to create, recreate and perform music independently?
Concepts
While much of the school-based musical experience happens within a group, it is also important for people to be able to create, recreate, rehearse and perform music independently.
CompetenciesDemonstrate the ability to independently create, recreate, rehearse and perform musical works and explain why this is important.
Standards Alignment Based upon the big ideas, essential questions, concepts and competencies
selected, identify the content standards that are aligned to this objective.
Academic Standards for the Arts and Humanities9.1 Production, Performance and Exhibition of Dance, Music, Theatre and Visual Arts A. Know and use the elements and principles of each art form to create works in the arts and humanities Music elements: • duration • intensity • pitch • timbre Music principles: • composition • form • genre • harmony • rhythm • texture B. Recognize, know, use and demonstrate a variety of appropriate arts elements and principles to produce, review and revise original works in the arts. Music: • sing • play an instrument • read and notate music • compose and arrange •improvise C. Integrate and apply advanced vocabulary to the arts forms
Data and Targets Used to Establish the SLOAwareness of student level of preparedness What, if any, are the course or grade prerequisites? What is shown by the current (baseline) data? Which students may need targeted objectives (e.g. students with
IEPs, diverse learners, etc.)?
There are no prerequisites for this class. No audition is required, but early screening to assign students to a choral singing part can provide the teacher with some baseline information. Students may have previously been in a choral ensemble class, providing a previous year summative assessment. In addition to information provided by IEPs, students with singing difficulties (pitch matching, speech problems, rhythm problems, and vocal production problems) may need objectives tailored to the nature of the difficulty. Four students with pitch match/tonality problems are identified for this SLO.
Targets What is the expected outcome (goal) by the end of the
designated instructional period (i.e., year, semester, course length, etc.)?
Using student baseline data, how many differentiated targets/goals are needed?
Are the targets ambitious yet realistic?
Each student will demonstrate an ability to sing his/her choral part independently in an unaccompanied setting, exhibiting mastery and/or growth in such components as tone quality, expression and style, phrasing, pitch, intonation, rhythm, diction and articulation.
Assessment/Performance Task Assessment/Performance Task NameChoral Music-Individual Student PerformanceDescription of the Assessment/Performance Task Describe the assessment/performance task that will be used to measure the
learning objective (LO). Consider any products or artifacts that will be gathered as part of the data
collection process. Identify the developer of the assessments/performance task used (e.g., teacher-
made, district-developed, commercial, etc.).
Each member of the high school choir will individually perform a teacher-chosen unaccompanied excerpt from the concert music (repertoire). These will be recorded by having students sing either into a microphone or within the visual and audio range of a video camera. Music selected for the second assessment will be of an equivalent grade level, as adjudicated by professional music organizations.
Assessment/Performance Task Objectives Rationale Describe how the assessment/performance task authentically reflects the
learning objective (LO). How is this assessment/performance task a reliable and valid measure of
student mastery or growth aligned with expected outcomes and goals? How does the assessment/performance task measure the ability of students
to independently apply what they have learned? How is the assessment/performance task free of bias? How do the assessment/performance task outcomes inform instruction? How does the assessment/performance task align with grade, school and
district level curricular goals?
Individual assessments are the only way of ascertaining student progress within the context of the larger performance ensemble. Assessments of this type can both be used as an on-going process through formative assessments (making the necessary pedagogical adjustments in instruction and learning, for both the teacher and the student), as well as summative assessments in the final stages of concert preparation.
Growth or MasteryDoes this assessment/performance task align with the student achievement goals for the LO?
Growth (change in student achievement across two or more points in time) Mastery (attainment of a defined level of achievement) Growth and Mastery
Administration of the Assessment/Performance TaskFrequency of Assessment/Performance Task Administration How often and when is this assessment/performance task administered? If measuring growth, are multiple assessment windows in place?
Students will be assessed at the end of the second quarter and again at the end of the fourth quarter.Resources Required What unique or specific equipment, technologies, or resources are needed to
complete this assessment/performance task?
Current choir repertoire, consisting of an eclectic representation of music from various styles, levels of difficulty, and written in various harmonic approaches (homophonic, polyphonic, syllabic, etc.).
Each student will be given an assessment rubric included in their folder, in order to be familiar with the standards and various levels of proficiency.
Basic recording device in order to record student assessments.
Adaptations for Diverse Learners and/or Students with Disabilities What adaptations built into the assessment/performance task are
needed assist diverse learners and/or students with disabilities?
Students may be diagnosed/identified as possessing emotional or mental disorders such as ADHD, ADD, & MR; requiring instructional and assessment adaptations, according to the requirements of IDEA and the student’s IEP. Adaptations may include factors such as: the student’s comfort level in singing alone; student ability to learn, comprehend or retain subject matter; or to perform certain passages with rhythmic or melodic accuracy when singing alone.
Personnel Can this assessment/performance task be administered by an
equivalent peer? (If not, please explain.) Does a district policy exist with regard to assessment/performance task
administration?This assessment could be administered by an equivalent peer; it can also be administered by the student.
Data CollectionMeasuring Student Progress/Preparation How was baseline data collected? (If baseline data was not collected,
please explain.) Can baseline data be compared with the results of this
assessment/performance task?
Assessment from the 2nd quarter will establish baseline data for the 4th quarter.Data Collection In what format will data be collected (e.g., database, graphed, portfolio,
etc.)? Is a pre-post test being used? (If so, please describe.) How frequently will data be collected?
Data will be collected from listening to the recordings and notating the level of student performance on the attached rubric. This process will be administered twice; the first will be considered a pre-test.
Rubrics and Scoring Scales How will individual student growth or mastery be determined
(defined and scored) using this assessment/performance task? Include the specific rubric/scoring scale that will be used.
Does the rubric and/or scoring scale correlate with the assessment/performance task?
Students will be assessed using the following rubric:
COMPONENT 4-Advanced 3-Proficient 2-Basic 1-Below Basic A. Tone Quality Tone is
consistently focused, clear, and centered throughout the range of the voice.
Tone is focused, clear and centered through the normal singing range. Extremes in range sometimes cause tone to be less controlled. Tone quality typically does not distract from the performance.
Tone is often focused, clear and centered, but sometimes the tone is uncontrolled in the normal singing range. Extremes in range are usually uncontrolled. Occasionally the tone quality distracts from overall performance.
The tone is often not focused, clear or centered regardless of the range, significantly detracting from the overall performance.
B. Expression and Style
Performs with a creative nuance and style in response to the score and limited coaching.
Typically performs with nuance and style that is indicated in the score or which is suggested by instructor or peer.
Sometimes performs with nuance and style that is indicated in the score or which is suggested by instructor or peer.
Rarely demonstrates expression and style. Just plays the notes.
C. Phrasing Phrasing is always consistent and sensitive to the style of music being sung.
Phrasing is usually consistent and sensitive to the style of music being sung.
Phrasing is usually consistent and occasionally sensitive to the style of music being sung .
Phrasing is rarely consistent and/or rarely sensitive to musical style.
D. Pitch Virtually no errors. Pitch is very accurate.
An occasional isolated error, but most of the time pitch is accurate and secure.
Some accurate pitches, but there are frequent and/or repeated errors.
Very few accurate or secure pitches.
E. Intonation Exemplary intonation. Practically of pitches are sung in tune.
Slight imperfections, but most pitches are sung in tune, maintaining tonal center (key) throughout.
Regular intonation difficulties, lacking consistent tonal center.
Significant intonation difficulties throughout, resulting in poor tonal center or recognizable melody.
F. Rhythm The beat is secure and the rhythms are accurate for the pitches being sung.
The beat is secure and the rhythms are mostly accurate. There are a few duration errors, but these do not detract from the overall performance.
The beat is somewhat erratic. Some rhythms are accurate. Frequent or repeated duration errors. Rhythm problems occasionally distract from the overall performance.
The beat is usually erratic and rhythms are seldom accurate distracting significantly from the overall performance.
G. Diction Student articulates clearly, and the text of the music is understandable.
Student articulates the words somewhat clearly, and the text can be understood most of the time.
Student sometimes articulates the words, but the text is often not discernible.
Student rarely articulates the words and the text is not discernible.
H. Articulation Manner of articulation is stylistically clear, agile and unified, according to the written musical excerpt, adding a significant degree of clarity and contrast to the music.
Manner of articulation is mostly clear, according to the written musical excerpt. Minor and infrequent inconsistencies do not significantly distract form the musical clarity and contrast.
Manner of articulation is inconsistent and/or awkward and/or incorrect, according to the written musical excerpt. Adding some clarity and contrast.
Manner of articulation is not very evident, according to the written excerpt, resulting in very little to no clarity and contrast.
Data Interpretation How will the assessment/performance task results be interpreted in the same
way across equivalent peers? Is there a reliable and valid scoring and interpretive process (i.e., state
developed, district-based, commercial, standardized, etc.) that is associated with the assessment/performance task? If so, please describe.
An equivalent peer could adjudicate student performance using this rubric by listening to the recordings. This process has a history of reliability, as found through the CT state common assessments. (ctcurriculum.org)
Implementation Timeline
Date SLO is due to principal Prior to the start of the school year.
Date(s) for Assessment and Data Collection
1st assessment: second week in January2nd assessment: 3rd week in May
Dates to complete Data Interpretation 1st week in June
Date to present Teacher Effectiveness Measure June 5, 2013
Teacher Effectiveness MeasureClassroom Objective How will the aggregated scores of individual student assessment results be used to define teacher effectiveness?Failing: few students achieve content mastery or growth
Needs Improvement: less than a significant number of students achieve content mastery or growth
Proficient: A significant number of students achieve content mastery or growth
Distinguished: An exceptional number of students achieve content mastery or growth
Less than 50% of students will achieve proficient or advanced levels in 6 out of 8 components (mastery) or will achieve improved scoring in 3 out of 8 components, with no decrease in the other 5 components (growth).
50% -74% of students will achieve proficient or advanced levels in 6 out of 8 components (mastery) or will achieve improved scoring in 3 out of 8 components, with no decrease in the other 5 components (growth).
70 % -84% of students will achieve proficient or advanced levels in 6 out of 8 components (mastery) or will achieve improved scoring in 3 out of 8 components, with no decrease in the other 5 components (growth).
85% -100% of students will achieve proficient or advanced levels in 6 out of 8 components (mastery) or will achieve improved scoring in 3 out of 8 components, with no decrease in the other 5 components (growth).
Targeted ObjectiveHow will the mastery or growth of targeted student populations be described and used to define teacher effectiveness?
Failing: Did not meet goal, little to no student mastery or growth
Needs Improvement: Did not fully meet goal but showed some student mastery or growth
Proficient: Met goal or otherwise demonstrated significant student mastery or growth
Distinguished: Surpassed goal otherwise demonstrated significant student mastery or growth
Targeted Population: Four students are identified as having pitch match/tonality problems, as determined by history of participation and/or initial screening via audition process.Mastery and/or growth goal: Students with pitch match problems/tonality problems will be assessed with vocal or piano support, on the same voice part, with an expectation that pitch match occur a minimum of 90% of the time.
Teacher Effectiveness RatingsWhat were the results of the assessments/tasks and how do they relate to the classroom and targeted objectives?
Classroom Objective Failing Needs Improvement Proficient Distinguished
Notes/Explanation Literature selected was not comparable between the two assessments.
Targeted Objective Failing Needs Improvement Proficient Distinguished
Notes/Explanation
SLO Process Dimension
SLO –Assessment Developers-Expertise Who was involved in establishing the SLO and what is their
expertise?
SLO/Student Assessment Development Team June 2012 Sheryl Monkelien, Chair, Music Education - Mansfield University Margaret Bauer, Executive Director – PMEA Dennis Emert, Instrumental Music – Fox Chapel SD – Pittsburgh Gerald Savage, Chair, Vocal Department, CAPA – Pittsburgh Ross Whiting, Fine Arts & Humanities Department – Bok Tech HS – Philadelphia Jamie Kasper, Associate Director – Arts Education Collaboration – Pittsburgh.Exemplars of Student Work
Exemplars can be found at ctcurriculum.org.
What is it that the PA SLO process hopes to accomplish?
To meet federal Race to the Top RequirementsTo meet state legislated teacher evaluation
requirementsTo utilize some processes that teachers
already have in placeTo inform principals as to appropriate
processes in content areas with which they may not be familiar
To provide teachers in non-tested grades and subjects an opportunity to be evaluated in their content area
To inform and instruct teachers in non-tested grades and subjects about standards-based education design: 1) Curriculum Frameworks 2) Standards 3) Backward Design4) Transfer of Learning5) PA Standards Aligned System Website
To have an effective teacher in every classroom.
Thanks for all that you do to bring the arts to the
students and communities in Pennsylvania!
OK, I guess this won’t affect my golf game too much!
• Want to develop or pilot an SLO?• Name/email• School/District• Principal Name/email• Specific Discipline in Music• Develop-Pilot-Both
PhaseIII
TeacherEffectiveness
Pilot
Questions? [email protected]
Top Related