WASHINGTON STATE FOCUSED SITE VISITWASHINGTON STATE FOCUSED SITE VISITAn Orientation to AccountabilityAn Orientation to AccountabilityAn Orientation to AccountabilityAn Orientation to Accountability

Office of Research, Evaluation, and Assessment

April 19, 2008

The CTL Comprehensive Web-based Assessment System

The CTL Comprehensive Web-based Assessment System

- Multiple Measurements Across Five Transitions- Multiple Measurements Across Five Transitions

- - Fairness, Consistency, and Accuracy of MeasuresFairness, Consistency, and Accuracy of Measures

- Standards Alignment- Standards Alignment

- Formative and Summative Evaluation- Formative and Summative Evaluation

- Data Management, Accessibility, and Utility- Data Management, Accessibility, and Utility

- Analyses Confirm Candidate Performance and Success- Analyses Confirm Candidate Performance and Success

- - The Professional Culture of AccountabilityThe Professional Culture of Accountability

Data are regularly and systematically collected, compiled, summarized, analyzed, and reported publicly to multiple

audiences for the purpose of improving candidate performance, program quality, and unit operations .

THE ASSESSMENT SYSTEM

DATA MANAGEMENT, ACCESSIBILITY, AND UTILITYDATA MANAGEMENT, ACCESSIBILITY, AND UTILITY

Summaries are also shared, discussed, and reported in numerous ways at each level of the unit's governance

organization, and are made public through the CTL/OREA web sites. Feedback is sought at every level:

candidate, faculty, PEAB, administration, as well as the public.

Consistent with the conceptual framework's constructivist philosophy, the CTL assessment system has

dynamically evolved over the past decade as a result of relevant and meaningful experiences.

The latest edition of the system is comprehensively designed to be purposely redundant in the measurement of

standards, flexible enough to meet specific program requirements, and robust enough to provide unit-wide

analyses for the purpose of improving program and unit (CTL) operations.

Multiple assessment measures are collected using a transitional timeline. Data are collected, aggregated, and

reported at both the program level and unit level for both initial and advanced programs using CTL, state, and

national standards. Data reports and summaries are used by individual faculty to assess their own instruction; by candidates to

assess their own learning and development; by program faculty and coordinators to assess program

effectiveness; and by the OREA to assess unit operations.

MULTIPLE MEASUREMENTS ACROSS FIVE TRANSITIONS:MULTIPLE MEASUREMENTS ACROSS FIVE TRANSITIONS:MULTIPLE MEASUREMENTS ACROSS FIVE TRANSITIONS:MULTIPLE MEASUREMENTS ACROSS FIVE TRANSITIONS:

A comprehensive assessment system systematicallyA comprehensive assessment system systematically

produces data on applicant qualifications, candidateproduces data on applicant qualifications, candidate

proficiencies, and the performance of graduates…proficiencies, and the performance of graduates…

Assessment System Measures by Transition

Aligned with Standards

There is a clear explicit connection between the assessment of candidates and professional, state, and

institutional standards for the role in which the educator is being prepared.

Standards are Aligned in Measurements

Standards are Aligned in Courses, Artifacts, Rubrics, and Reports

Standards are Aligned in Summary Data

Standards are Aligned in Aggregated Data

STANDARDS ALIGNMENT

State, Professional, and Institutional Standards are Aligned

Standards are Aligned in Program Reports

Standards are Aligned in Unit Report and Assessment Evaluations

ANALYSES CONFIRM CANDIDATE PERFORMANCE AND SUCCESS

Teacher Residency

Principal and Program Administrator

Professional and Pedagogical Knowledge and Skills -vs- Grades (Pro. Sequence)

Summary of Candidate Success (Pro. Sequence)

Summary of Candidate Success

Pre-Autumn Self Evaluation

Mid Quarter Assessment of Knowledge and Skills

(Student Teaching)

Data show the clear relationship of assessments to candidate outcomes.

Documentation of candidate performance verifies learner expectations have been

met by all candidates.

Professional Growth Plan

FORMATIVE AND SUMMATIVE EVALUATIONFORMATIVE AND SUMMATIVE EVALUATION

Formative Measures

- GPA (Full and Conditional Admissions)

- Student Teaching GPA

- Disposition Inventory

- Knowledge and Skills on LiveText

- Informal and Formal Student Teaching Observations

Summative Measures

- WEST B

- Faculty Referrals

- Final Student Teaching Evaluation

- Performance-based Pedagogy Assessment

- First and Third Year Teacher Surveys

- EBI Principal and Teacher Surveys- Placement Surveys

- WEST E

- Certification Checklist

- Admissions Checklist

Teacher Residency

Data are regularly and systematically collected, compiled, summarized, analyzed and publicly

reported using the web for the purpose of improving performance, quality, and operations.

9CBC85D8

FORMATIVE AND SUMMATIVE EVALUATIONFORMATIVE AND SUMMATIVE EVALUATION

Formative Measures

- ISSLC Quarterly Assessment

- Mid Quarter Assessment on Knowledge and Skills

- Knowledge and Skills on LiveText

Summative Measures

- Professional Growth Plan

- Administrators Certification Follow-up Survey- Intern Meeting Evaluation

Principals and Program Administrators

- Professional Growth Plan

Data are regularly and systematically collected, compiled, summarized, analyzed and publicly

reported using the web for the purpose of improving performance, quality, and operations.

- Thesis, Project, Exam

- Intern Meeting Evaluation

FAIRNESS, CONSISTENCY, AND ACCURACY OF MEASURESFAIRNESS, CONSISTENCY, AND ACCURACY OF MEASURES

EVALUATIONEVALUATION

Inter-rater Summaries

Disposition Validation Study

Courses

Artifacts Rubrics

Rubrics and Grades Analysis

Statistical Evidence of Validity and Reliability

WEST-B

WEST -E

EBI Principal

EBI Teacher

Content Validity

The extent to which measurement

content represents the academic

discipline or domain being taught.

GPA and WEST E Pass Rates Analysis

Standards

The unit regularly searches for stronger relationships in evaluations, revising both the

underlying data systems and analytic techniques. The regular questioning of validity has led

to modifications and changes in assessment technique as well as technology.

Aggregated Data for Standards Analysis

THE PROFESSIONAL CULTURE OF ACCOUNTABILITYTHE PROFESSIONAL CULTURE OF ACCOUNTABILITYWeb-based Presentation of Data

Program Reports

Unit Reports

Public Survey

Assessment Evaluation

Changes the Academic Conversation

Data are shared with faculty and regularly reviewed by PEABs. The underlying data systems and analytic techniques are

evaluated and revised. Recommendations for improvement are based on data-based recommendations from faculty,

CTL governance committees, PEABs, and the public. Accountability has become a regular part of faculty conversation.

WASHINGTON STATE FOCUSED SITE VISITWASHINGTON STATE FOCUSED SITE VISIT

Assessment System

QUESTIONS?