Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.
description
Transcript of Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.
![Page 1: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/1.jpg)
CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS
FOR MEASURING TEACHER EFFECTIVENESSAND STUDENT ACHIEVEMENT
Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D. Janice Janz, Ph.D, & Richelle Voelker, M.Ed.
University of New Orleans (UNO)
Kristin A. Gansle, Ph.D.Louisiana State University (LSU)
![Page 2: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/2.jpg)
CONTRIBUTE TO THE NATIONAL DISCUSSION OF THE ACCOUNTABILITY FOR STUDENT LEARNING THROUGH EFFECTIVE TEACHER PREPARATIONS
Purpose
![Page 3: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/3.jpg)
Assessment • Demand for accountability
• Opportunity to examine the relationship between teaching and learning
• Evidence of strengths, needs for improvement, and impact
“The processes of teaching and learning stimulate one another.”
(Confucius)
![Page 4: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/4.jpg)
Agenda
Louisiana Department of Education LA Teacher Preparation Program (TPP) AssessmentValue-Added Teacher Evaluation ModelData and Results
UNO Collaborative Redesign ProjectCollaborative ProcessesResults: In-ProgressSubsequent Plans
Conversations with ColleaguesDiscussion QuestionsOpen Discussion
![Page 5: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/5.jpg)
LOUISIANA METRICS OF TEACHER PREPARATION PROGRAM EFFECTIVENESS
INCORPORATING STUDENT ACHIEVEMENT
Kristin A. Gansle , Ph.D.Louisiana State University (LSU)
![Page 6: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/6.jpg)
LA Teacher Preparation Program (TPP) Assessment 2000-2001 & Following
• How to measure our program completers’ success in terms of the achievement of their students?
• Especially important given statewide redesign of academic programs including SpEd by TPPs
• BoR (Jeanne Burns), Blue Ribbon Commission, LSU (George Noell, Kristin Gansle)
• Have been using Value-Added metrics to evaluate TPPs since 2003
![Page 7: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/7.jpg)
What Do We Do?• Predict achievement of individual students based on
prior achievement, demographics, and attendance (using State achievement tests: HLM, mean approx 300, sd approx 50)
• Assess actual student achievement• Link the difference between predicted achievement and
actual student achievement to their new teachers and the TPPs that taught them
• Act on results
![Page 8: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/8.jpg)
Predictors for LDOE Value-AddedTeacher Evaluation Model (Current Model)
• Prior Student Achievement• Attendance• Gifted Classification• Free/Reduced Lunch Status• Section 504 Status• Disability Status• Discipline Records
• Retained students are included in the analysis
![Page 9: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/9.jpg)
Value-Added Model
• State Achievement Tests (Math, Science, Social Studies, Reading, & Language Arts - Grades 4-9)
• New Teachers: • 1st and 2nd year teachers with regular certificates• Teaching within area of certification• Full academic year with their students
• Experienced Teachers:• 3rd or subsequent year teachers with regular certificates • Teaching within area of certification
• Each year: all districts, 250,000 students, 7000+ teachers, 1300+ schools
![Page 10: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/10.jpg)
Data We Provide to TPPs• (October, 2006) By content areas (Reading, Mathematics, Science,
Social Studies, English/Language Arts)
• (March, 2011) By certification/grade spans within content areas (Grades 1-5, Grades 4-8, Grades 6-12, & Special Education)
• (Fall, 2011) • By student performance subsets within content areas (Low,
Middle, and High)• By special education status (yes/no) for student groups• By free lunch status (free lunch/pay own lunch)• By LEP status
![Page 11: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/11.jpg)
Data We Provide to TPPs(continued)
• (Fall 2011) Deidentified individual teacher data for programs• Value-Added Score for each teacher• Mean content standards scores from standardized testing
program (LEAP, iLEAP) for ELA, Math, Reading, Science, Social Studies
• What does that look like?
![Page 12: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/12.jpg)
Overall Results overall
content Nmean effect sem
ELA 75 1.0 0.7
Mathematics 103 5.1 0.9
Reading 48 -2.2 0.9Science 88 2.7 1.0
Social Studies 66 0.7 1.7
![Page 13: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/13.jpg)
FOR ADDITIONAL INFORMATION
Kristin A. [email protected]
Jeanne M. [email protected]
George H. [email protected]
http://regents.la.gov/value-added-teacher-preparation-program-assessment-model/
![Page 14: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/14.jpg)
Certification Endorsements
Elementary Grades Middle School Secondary Grades Special Education
content n
mean effect
sem n
mean effect
sem n
mean effect
sem n
mean effect
semELA 42 -0.8 1.1 30 3.1 1.6 41 -1.0 1.1 12 1.2 2.8
Mathematics47 1.4 1.3 69 4.2 1.2 28 3.8 1.3 12 1.0 1.8
Reading 40 -0.6 0.9 8 3.2 2.9 9 3.0 2.6 8 3.7 2.9
Science 38 -1.6 1.1 40 2.5 1.6 20 -0.4 1.3 5 -2.9 1.9
Social Studies35 -2.7 1.7 10 2.0 4.2 19 4.1 3.2 5 -5.1 4.2
![Page 15: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/15.jpg)
Achievement Bands Lowest 25% Middle 50% Highest 25%
content n
mean effect
sem n
mean effect
sem n
mean effect
semELA 25 -4.7 1.7 34 1.0 1.0 21 -2.2 1.2
Mathematics24 -5.2 1.9 31 2.1 1.2 21 -0.6 1.5
Reading 17 -1.8 1.6 24 0.3 0.9 17 -0.2 0.9Science 27 -2.4 1.0 29 0.0 1.1 19 -0.9 1.2
Social Studies30 -0.9 1.5 37 0.0 1.4 25 -0.1 2.1
![Page 16: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/16.jpg)
Free Lunch/Not SpEd/no SpEd
Free Lunch Paid Lunch Special Education No Special Education Services
n
mean effect
sem n
mean effect
sem n
mean effect
sem n
mean effect
semELA 50 -0.5 0.8 47 1.3 0.7 40 0.3 1.4 44 -1.3 0.9
Mathematics40 -1.4 1.0 31 2.0 1.1 20 -1.2 1.1 45 -1.9 1.1
Reading 34 -1.9 0.7 43 1.5 0.8 17 -0.9 1.5 38 -2.1 0.9
Science 38 -0.2 1.0 38 0.8 0.8 23 1.0 1.0 32 0.3 1.0
Social Studies45 -1.4 1.1 40 0.7 1.0 29 -0.8 1.5 40 -1.5 1.1
![Page 17: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/17.jpg)
Deidentified Data in Excel File
TPPs Can Use as They Wish
![Page 18: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/18.jpg)
UNO: PROGRAM DATA
Brenda Burrell, Ph.D. Paul Bole, Ed.D.
Janice Janz, Ph.D. & Richelle Voelker, M.Ed.
University of New Orleans (UNO)
![Page 19: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/19.jpg)
Collaborative Processes
•Analyzing and synthesizing professional standards and literature
• Interviews, surveys, and discussions with multiple and diverse stakeholders
• Individual and committee design and development work sessions
![Page 20: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/20.jpg)
Results: Work In-Progress•Response to Louisiana TPP Assessment•Constructive Principles •Concordance of Professional Standards •Revised Conceptual Framework•Answers to Key Questions: Perspectives
![Page 21: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/21.jpg)
Response to LA TPP AssessmentConcerns/Opportunities
• School administrators may not know what to look for when observing special education instruction
Potential Strategies
• Provide professional development opportunities for school leadership personnel
• Develop guidelines aligned with research based special education practices
![Page 22: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/22.jpg)
Response to LA TPP AssessmentConcerns/Opportunities
• It may be difficult to determine how to assign effectiveness scores to special education teachers working in inclusive settings
• Teachers in high-need schools may not be rated similarly as those in more advantaged schools
Potential Strategies
• Research • Disseminate findings• Make recommendations • Facilitate systemic
changes
![Page 23: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/23.jpg)
Response to LA TPP AssessmentConcerns/Opportunities
• It may be difficult for special education teachers to score “highly effective” on some elements of the Danielson framework
• Some teachers may develop Student Learning Targets (SLTs) that are problematic
Potential Strategies• Advocate for revised descriptions
in rubric to align with special education practices and provide corresponding examples to assist principals when observing effective instructional practices
• Review samples of SLTs developed by special education teachers to determine specific issues and develop corresponding training materials
![Page 24: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/24.jpg)
Constructive Principles•Collaboratively Designed• Informed by Multiple and Diverse Resources•Linked to Professional Standards•Useful for Multiple Assessment Points•Yields Data to Inform/Improve Teacher Education
![Page 25: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/25.jpg)
Concordance of Professional Standards• Danielson Framework for Teaching
• Council for Exceptional Children (CEC) • International Reading Association (IRA)• National Council of Teachers of English (NCTE)• National Science Teachers Association (NSTA)• National Council for the Social Studies (NCSS)• National Council of Teachers of Mathematics (NCTM)
• Louisiana Components of Effective Teaching • UNO Roles and Responsibilities of Teachers
![Page 26: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/26.jpg)
Revised Conceptual FrameworkRoles & Responsibilities of Effective Teachers
•Manage Classroom Contexts/Environments
•Design Curriculum and Instruction
•Deliver Instruction and Assess Learning
•Participate in Professional Responsibilities (e.g., Advocacy, Collaboration, Using Data to Improve Practice)
![Page 27: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/27.jpg)
Answers to Key Questions: PerspectivesQuestions
• How should IHE faculty assess teachers’ use of evidence-based practices and teacher effectiveness
• How should IHE faculty assess program completers’ impact on student learning and student behavioral progress?
Sample Responses• Video Sample of Teaching
• Surveys/Interviews
• Informal Observations
• Progress Monitoring
• Student Work Samples
• Student/Teacher Portfolios
![Page 28: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/28.jpg)
Subsequent Plans• Develop our own teacher evaluation rubric
• Use same teacher evaluation rubric across concluding semesters of preparation and during initial years of teaching after program completion (induction); using evaluation to provide support
• Conduct field testing and research
• Make recommendations to the Louisiana DOE regarding rubric currently used for State teacher assessment and/or training provided to evaluators
![Page 29: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/29.jpg)
Conversations with Colleagues
Question: Set 1
• What data related to teacher effectiveness and student achievement should and can be collected by faculty in teacher education programs?
• What data will represent authentic and meaningful measures of academic and behavioral performance and growth of students with high incidence disabilities and is also available to university faculty?
![Page 30: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/30.jpg)
Conversations with Colleagues
Question: Set 2• Are administrators and other teacher evaluators sufficiently informed to assess teachers with students with special needs in an inclusive setting or other settings?
• What professional development opportunities would be most helpful and how could or should they be delivered?
![Page 31: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/31.jpg)
Conversations with Colleagues
Question: Set 3• How can IHEs assess positive teacher impact on students who do not demonstrate academic or behavior progress?
• Should teacher performance be aligned exclusively with teacher effectiveness, as indicated by student performance?
![Page 32: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/32.jpg)
Conversations with Colleagues
Open Discussion
![Page 33: Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D . Janice Janz , Ph.D , & Richelle Voelker , M.Ed.](https://reader036.fdocuments.net/reader036/viewer/2022062315/568161cc550346895dd1b556/html5/thumbnails/33.jpg)
Contact InformationUniversity of New Orleans (UNO)Department of Special Educationand Habilitative Services
Brenda Burrell Janice [email protected] [email protected]
Paul Bole Richelle [email protected] [email protected]