Wayne State University

College of Education

TEAC: Inquiry Brief Proposal Presentation

Presented by the TEAC Advisory Committee

April 24, 2012

Overview Program Overview

Claims & Rationale

Methods of Assessment

Results, Discussion, Plan Claim 1

Claim 2

Claim 3

Auditor’s Comments

Program Overview Background and History

Distinguishing Features of the Program

Requirements for Admissions & Graduation

Demographics of Initial Certification Candidates

Description of Faculty

Logistics of Program

Claims & Rationale Claim 1: Initial Certification

Completers understand subject matter knowledge they will teach (Learning to Learn, Multicultural, Technology).

Claim 2: Initial Certification Completers have pedagogical knowledge (Learning to Learn, Multicultural, Technology).

Claim 3: Initial Certification Completers know how to teach in a caring and effective matter (Learning to Learn, Multicultural, Technology).

Methods of Assessment

Sources of Measure

Surveys (performance assessments, WSU surveys, MDE surveys, observations)

Course Grades (methods & total GPA)

Ratings of Capstone Conversation

Scores on MTTC

Research Design

Reliability and Validity Factors

Claim 1:Subject Matter

Knowledge

Subcommittee: Janet Andrews, Mariane Fahlman, Chavon Jackson, Jo-Ann Snyder, Cheryl Somers,

Initial Certification Completers understand

subject matter knowledge they will teach.

Learn and understand the subject matter they hope to teach.

Competencies & Data Sources

Competencies: 1, 2,

Data Sources:•Unit Plan

•University Supervisor Final Eval. Observation

•Cooperating Teachers Final Eval. Observation

•WSU Student Exit

Survey•MDE Supervisor Survey •GPA

•MTTC

Performance Assessments and Observation Data

Claim 1: Findings

Finding 1 – based on unit plan assessment, 95% of students demonstrated knowledge of subject matter at the proficient or exemplary level

Finding 2 – both cooperating teachers and college supervisors rates 93% proficient or exemplary in knowledge of subject matter

Claim 1: Findings

Finding 3 – all teacher candidates receive a grade of C or better in all subject matter courses.

Finding 4 – all teacher candidates successfully complete the MTTC.

Findings of CCT: LEARNING TO LEARN

Receiving passing grades in 100 % of content area classes is evidence that strategies for learning were at their command

Successful completion of the MTTC test demonstrated their mastery of content material again evidencing their ability to learn

Discussion of Content Knowledge (claim 1)

The data we examined suggests that our candidates have learned and do understand the subject matter when they leave us.

Our data sources for cross-cutting themes appeared to be too narrow.

Additionally we have data we have not examined for cross-cutting themes (e.g. mc course, tech course).

Implications for Content Knowledge (claim 1)

Need better documentation of how students use technology to develop concepts in their discipline.

We need to include other sources that demonstrate evidence of this such as:DIVERSITY:University

Foreign Culture requirement,

COE Multicultural requirement,

COE inclusion requirement,

TECHNOLOGYComputer

literacy requirement,

Technology course in every curriculum

E-PortfoliosBlackboard

LEARNING TO LEARN:

What else can we do?

Claim 2:Knowledge of

Pedagogy

Subcommittee: Oscar Abbott, Abby Butler, Bev Schneider, David Whitin, Phyllis Whitin,

They organize for effective instruction by establishing a safe and orderly environment that is conducive to learning and creative and critical thinking.

They utilize school, community, and technological resources and integrate content across the curriculum.

They use a range of assessment strategies to inform ongoing instruction.

Initial Certification Completers have

pedagogical knowledge:

Competencies & Data Sources

Community Observation

District School Curriculum Analysis

Classroom Management Plan

Lesson Plan Unit Plan Case Study Cooperating

Teacher’s Final Eval Observ.

Univ. Supervisor’s Final Final Eval Observ.

WSU Student Entry Survey

WSU Student Exit Survey

MDE Supervisor Exit Survey

Competencies: 2, 3, 4, 8, 9Data Sources:

Performance Assessments and Observation Data

Percentages are proportions of students rated proficient or higher on the respective competency for each data source. *Indicates assessments given early in the program where a score of basic is considered acceptable.

Competency Performance Assessments Observations

CO* DSC* CMP* LP UP CS CTF USF

2 37% 49% 89% 95% 93% 96%

3 48% 91% 97% 87% 95% 94%

4 42% 90% 95% 95% 96%

8 44% 85% 90% 76% 97% 97%

9 40% 49% 89% 95% 97%

Findings for Pedagogy (Claim 2)

• Higher percentages of proficient or exemplary ratings are found in assessments that take place later in the program.

• Highest ratings (Lesson/Unit Plan) and Cooperating Teacher and Supervisor assessments indicate that candidates exit the program pedagogically well prepared.

• There are lower ratings for CO, DSC, CMP, and CS. CO, DSC, and CMP reflect introductory experiences. CS is assessed later in the program.

Discussion for Pedagogy (Claim 2)

CO, DSC, and CMP, with only 37 – 48% showing proficient and above performance, are indicative of a beginning level understanding when this content is first introduced. A rating of basic is considered satisfactory for these 3 assignments.

Data from 2010-2011 reflect the work of many candidates who were not exposed to the service learning component that is designed to give candidates experience working with children and community organizations.

The low Case Study scores (87% and 76%) are of more concern since it is assessed later in the program.

Discussion for Pedagogy: Cross-Cutting Themes (Claim

2) Learning to Learn is reflected in five performance

assessments and two observations: CO, DSC, CS, LP, UP, and CTF, USF.

Multicultural is reflected in three performance assessments and two observations: DSC, LP, UP and CTF, USF.

Evidence for these two themes is integrated throughout students’ experiences.

Technology is only reflected in two assessments: DSC, LP.

Evidence for Technology theme is thin.

Implications for Pedagogy (claim 2)

Closely monitor assessments with lower ratings, and implement changes to strengthen the program.

These results, particularly for CS, suggest modifications to our program, e.g. to incorporate into methods classes additional experiences involving formulating instructional decisions based on ongoing observational assessments.

Modules developed by SED may be useful to all initial certification candidates in their work on CMP.

Revise assessment rubrics that reflect technology in candidates’ teaching.

Initiate conversations to examine evidence of technology-integrated pedagogy at the program level for each content area.

Claim 3:Caring and Effective Teaching

Subcommittee: Tom Edwards, Sharon Elliott, Sharon Sellers-Clark, Bo Shen, Marshall Zumberg,

Initial Certification Candidates are Caring and

Effective Caring Teaching: Reflects interest

in, and concern for, students.

Effective Teaching: Results in student learning

Competencies & Data Sources

Competencies: 3, 4, 5, 6, 7, 10

Data Sources:

Community Observation (CO), Classroom Management Plan (CMP), Reflective Journal (RJ),Teaching and

Learning Statement (TLS), Lesson Plan (LP),Unit Plan (UP),Case Study (CS).

Percentages are proportions of students rated proficient or higher on the respective competency for each data source.

*Indicates assessments given early in the program where a score of basic is considered acceptable. Note:Community Observation (CO), Classroom Management Plan (CMP), Reflective Journal (RJ), Teaching and Learning Statement (TLS), Lesson Plan (LP), Unit Plan (UP), Case Study (CS)

CompetencyPerformance Assessment

CO* CMP* RJ TLS LP UP CS

3 48% 73% 91% 97% 87%

4 73% 84% 90% 95%

5 47% 87% 88% 95% 85%

6 42% 44% 77% 81% 86% 91% 82%

7 35% 44% 74% 86%

10 39% 82%

Competencies & Data Sources Competencies: 3, 4, 5, 6, 7, 10

Data Sources: (Con’t.)

Capstone Conversation (CC), WSU Student Entry Survey (WSEn), WSU Student Exit Survey (WSEx) MDE Student Exit Survey-MDE (MSEx), MDE Supervisor Survey (MSS), University Supervisor Final Observation

(USF), Cooperating Teacher Final Observation

(CTF).

Survey and Observation Data

Percentages are proportions of students rated proficient or higher on the respective competency for each data source.

*Indicates assessments given early in the program where a score of basic is considered acceptable.

Note: CC=Capstone Conversation; WSEn=Student Entry Survey; WSEx=Student Exit Survey; MSS=MDE Supervisor Survey; USF=University Supervisor Final Evaluation Observation; CTF=Cooperating Teacher Final Evaluation Observation.

Findings for Caring & Effective (Claim 3)

Percentage of proficient and exemplary ratings tends to be higher on assessments that occur later in the program.

Highest percentages of students rated proficient and exemplary on items related to caring and effective teaching occurred in the MSS, USF, and CTF.

There are only small differences in the percentages of students rated proficient and exemplary on the WSEx and MSEx compared to the USF and CTF.

Lowest percentages of proficient and exemplary ratings occurred on the CO and CMP, both of which occur early in the program. (It is also noteworthy that a rating of “basic” is considered satisfactory on those two assessments.)

Findings for Cross-Cutting Themes

in Caring & Effective Teaching Caring & Effective Teaching: Data drawn from 7

performance assessments, 4 surveys, and 2 observations across 6 competencies.

Learning to Learn data was drawn from

Four of the performance assessments (CO, RJ, TLS, LP) across Competencies 7 & 10.

Three (CC, WSEn, WSEx) of the surveys in Competency 6.

Both of the observations in Competency 6 (CTF, USF)

Findings for Cross-Cutting Themes

in Caring & Effective Teaching Caring & Effective Teaching: Data drawn from 7

performance assessments, 4 surveys, and 2 observations across 6 competencies.

Multicultural Perspective data was drawn from

Six of the assessments (CO, RJ, TLS, LP, UP, CS) across Competencies 5, 6, 7, & 10.

All four of the of the surveys across Competencies 5 and 7.

Both of the observations across Competencies 5 and 7

Findings for Cross-Cutting Themes

in Caring & Effective Teaching Caring & Effective Teaching: Data drawn from 7

performance assessments, 4 surveys, and 2 observations across 6 competencies.

Technology data was drawn from

Three of the assessments (TLS, LP, UP) in Competency 4.

Three of the surveys (WSEn, MSS, WSEx) across Competencies 3, 4, and 6.

There was no data addressing issues of caring & effective teaching from either either observation that provided direct evidence of technology.

Findings for Cross-Cutting Themes (CCTs) in Caring & Effective

Teaching

There was data for each of the CCTs embedded in the data for Caring & Effective.

The data that was there supports each theme.

Multi-Cultural Perspective had the most supporting data.

Technology had the least supporting data.

Technology is not addressed directly in either the CTF or USF.

Discussion: Caring & Effective Teaching

Our analysis of the data revealed a clear trend in the growth of the proportion of candidates rated proficient or higher on the performance assessments in areas related to caring & effective teaching.

This analysis suggests that our program completers grow over time in their ability to provide caring and effective teaching.

The vast majority reach proficient or exemplary ratings.

The evidence we have supports the claim that we prepare teachers who are caring & effective.

Discussion: CCTs in Caring & Effective

Regarding the cross-cutting themes, the evidence of a multi-cultural perspective is strongest, because it

comes from more data sources and addresses more of the related competencies.

of learning to learn is adequate ,

of technology is weakest, coming from only a few data sources, one of which is self-reported.

Implications for Caring and Effective Teaching

Revisit the rubrics of those performance assessments with the lowest percentages of proficient and exemplary ratings to ensure that the rubrics are providing the information we seek.

Consider revising the rubrics for all of the performance assessments so they provide more information about our candidates technological proficiency in the service of pedagogy.

Revisit the CMP rubric to learn why no data related to caring & effective teaching from that source supports any cross-cutting theme.

Revise the observation reporting forms to provide information on technology.

Small Group Discussion 1. Do we have the evidence to support these claims?

Why/Why not?

2. What are the strengths of the evidence?

3. What might be missing?

4. Other assessments/tools we should consider to assure: Subject Matter Knowledge Pedagogical Knowledge Caring & Effective Teaching

With the 3 cross cutting themes: Learning to Learn Multicultural Perspectives Technology

Auidtor’s Comments