EEmpowerment Zone: An eLearning System Facilitates the Assessment of Teaching Candidates.
-
Upload
tobias-stafford -
Category
Documents
-
view
226 -
download
0
Transcript of EEmpowerment Zone: An eLearning System Facilitates the Assessment of Teaching Candidates.
Overview
Provide Context Accelerate Online Program eLearning Delivery System
Exploratory Study – very much in progress Third Year Evaluation of Accelerate
Online/OPTIONS a Transition-to-Teach Program at TAMU - Results
Background
We have invested over three years of continuous effort and resources in developing a teacher preparation program that incorporates an on-line delivery system
And have come to view this delivery system as holding significant promise for the continuing education and formal development of teachers following their initial certification and entry into the profession.
Perspectives
The eLearning attributes of accessing reliable and valid multimedia resources 24/7 from a personal workstation are powerful influences for changing how professional development experiences are envisioned.
However, in the arena of eLearning, it is generally accepted that professional development efforts in public schools have not “kept pace with the rapid changes in the quality and quantity of information technology” (Denton, Davis, Strader, Durbin, & Wang, 2004; Moursund & Bielefeldt, 1999).
School policy makers wanting to improve teaching that results in greater student performance nearly always consider teacher professional development as a central component in their school improvement plans (Guskey, 2002).
Our experiences (Denton, Davis, Smith, Strader, Clark & Wang, 2004-05) with technology professional development for teacher educators and classroom teachers are consistent with the literature (Garet, Porter, Desimone, Birman & Yoon, 2001; Joyce & Showers, 2002; Loucks-Horsley, Love, Stiles, Mundry, & Hewson, 2003) regarding professional development experiences that –
emphasize academic subject matter (content), provide opportunities for “hands-on” activities (active
learning), are integrated with ongoing classroom operations
(coherence), and provide many development experiences for an
extended period of time are more likely to produce desired knowledge and
skill changes.
The Context –Accelerate Online:A Teacher Preservice Professional Development Program
Accelerate Online/OPTIONS is a three component program for certifying secondary teachers in Texas.
The Online Curriculum consists of 35 online modules developed to engage the candidate with issues identified as necessary for a beginning teacher by the Texas Board of Educator Certification .
Early Field Experience
The Field Based Experience consists of a 40 clock-hour supervised teaching field experience in a secondary school during the initial phase of the program.
Internship
The final program component is a paid internship where candidates are supported by a trained mentor and a university supervisor, who guide, observe and provide constructive feedback to the interns during their year-long development as a beginning teacher.
eEmpowerment Zone System
Comprehensive, web-based elearning, assessment and professional development system designed for diverse communities of learners (Davis, T., Director and Co-Developer).
Primary delivery system for the Accelerate Online teacher certification program in the Department of Teaching, learning, and Culture
2500 users to date, including approx.110 Accelerate Online students
http://empowermentzone.tamu.edu http://empowermentzone.tamu.edu/portfolios/center
eZone | Online Modules
The eEmpowerment Zone, is a dynamic, on-demand delivery platform developed in-house (Davis & Strader) that enables cohorts/communities of teaching candidates to access instructional modules, as well as integrated resources and tools that support teaching and learning, supervision, resource evaluation and collaboration.
eZone | Resources By organizing instructional web-based modules,
electronic portfolios, resources and tools into an integrated system, teaching candidates can seamlessly complete the online and field-components of the program, while receiving extensive support from university supervisors, mentor teachers, fellow students and program staff.
In addition, the backend eZone database houses extensive performance data on all students related to online instructional module assessments, classroom observations, and ePortfolio assessments.
Welcome VIDEO
eLearning System Design
eZone –eLearning System
Online Modules ePortfolios Classroom Observations Integrated Resources
My eAssessment Center
Cluster 1
Cluster 2
Cluster 3
Cluster 4
Module/Course Assignments
Early Field Experiences
Instructional Artifacts
Domain 1
Domain 2
Domain 3
Domain 4
Domain 5
Community Discussion Boards
Community ePortfolios
Domain 6
Domain 7
Resource Media Center
Break Room
My eAssessment Center
The My Assessment Center in the eZone and underlying database facilitates 24/7 tracking of student progress
The My eAssessment Center serves to affirm candidate program status in terms of completed assignments and module deliverables
A digital monitoring system will return the candidate to the uncompleted section of a module that was exited before the module was completed.
My eAssessment Center As a management resource for program
administrators, this digital monitoring system attends to each candidate’s visits to a module, the elapsed time spent examining the contents of the module and whether items requiring a response have in fact been completed as well as module pretest and posttest performances by the candidates.
Candidate Progress
Candidate progress in completing the modules are reviewed (weekly) to determine whether individual candidates are progressing satisfactorily in the program,
As well as determining whether modules have potential design flaws given the collective performance of the candidates on particular activities as well as overall performance on the module.
Additional Perspectives Three recent reports provide insights about alternative route
program features [Allen (2003), Seftor and Mayer (2003), and Mayer et al. (2003), cited in American Institutes for Research(2005)]
All three reports examined studies on the types and characteristics of alternative certification programs that might prove to be effective in teaching and for student learning
Allen’s literature review generated a set of principles regarding program features which he found to be indicated in successful programs
Among these five were: Solid curriculum that includes instruction in classroom basics
and teaching methods, and As much training as possible prior to teaching assignments
Research Questions
What is the relationship between candidates’ module instructional time and their performance on the Pedagogy and Professional Responsibilities (8-12) exams?
What is the relationship between candidates’ module posttest performance and their performance on the Pedagogy and Professional Responsibilities (8-12) exams?
Module Clusters for Accelerate Online Program
Module Clusters for Accelerate Online Program are organized by the following clusters:
School & Classroom Environment Designing Instruction Promoting Student Learning Professional Roles & Responsibilities Technology Applications Plus 3-4 Content Area Modules (Science,
Mathematics, History and Social Studies, English, Language Arts & Reading)
Pedagogy and Professional Responsibilities ((PPR)
Test Framework
Domain 1: Designing Instruction and Assessment to Promote Student Learning (approx. 31% of the test)
SBEC Standards Assessed PPR Standard I: The teacher designs instruction appropriate for all students that reflects
an understanding of relevant content and is based on continuous and appropriate assessment.
Online Module Alignment SBEC Standards (Knowledge and Skills) Addressed in Module
M4: Instructional Planning Knowledges 1.7K-1.11K Skills 1.6S-1.11S
M5: Performance Objectives Knowledges 1.7-1.15K Skills 1.6S-1.15S
M6: Instructional Strategies Knowledges 1.19K-1.24K 3.12K-3.14K Skills 1.19S-1.23S 3.15S-3.17S
M7: Accommodating Student Diversity in the Classroom
Knowledges 1.1K-1.6K 2.1-2.3K Skills 1.1S-1.5S 2.1-2.3S
M8: Adolescent and Learner Development Knowledges 1.1K-1.6K Skills 1.1S-1.5S
M19: Constructing Tests and Program Assessment
Knowledges 1.25K-1.31K Skills 1.24S-1.29S
M20: Multiple Choice Item Construction Knowledges 1.25K-1.31K Skills 1.24S-1.29S
M21: True False Item Construction Knowledges 1.25K-1.31K Skills 1.24S-1.29S
M22: Matching Item Construction Knowledges 1.25K-1.31K Skills 1.24S-1.29S
M23: Essay Item Construction Knowledges 1.25K-1.31K Skills 1.24S-1.29S
PPR Test Framework – Accelerate Online Module Alignment – Domain I
Domain II
PPR Test Framework Domain II: Creating a Positive, Productive Classroom Environment (approx. 15% of the test)
SBEC Standards Assessed PPR Standard II: The teacher creates a classroom environment of
respect and rapport that fosters a positive climate for learning, equity, and excellence.
M2: Time Management Knowledge 4.3K Skills 3.1S, 3.11S, 3.20S, 4.10S
M3: Discipline Management Techniques
Knowledges 2.6K- 2.23K Skills 2.6S - 2.21S
PPR Test Framework Domain III: Implementing Effective, Responsive Instruction and Assessment (approx. 31% of the test)
SBEC Standards Assessed PPR Standard I: The teacher designs instruction appropriate for all students that reflects an
understanding of relevant content and is based on continuous and appropriate assessment.
PPR Standard III:The teacher promotes student learning by providing responsive instruction that
makes use of effective communication techniques, instructional strategies that actively engage students in the learning process, and timely high-quality feedback.
Technology Applications Standards I-V
M1: Educational Philosophy – Approaches to Teaching
Knowledge 4.3K Skills 3.1S, 3.11S, 3.20S, 4.10S
M9: Diagnosing Learners for Instructional Delivery
Knowledge 3.5K - 3.11K Skills 3.7S - 3.14S
M10: Communication Skills Part I & II Knowledges 2.1K-2.5K 3.1K-3.4K 3.15K-3.16K Skills 2.1S-2.5S 3.1S-3.6S 3.18S-3.20S
M11: eCommunication Tools and Applications Technology Application – Knowledges: 2.1k, 2.3k ; Skills: 2.1s, 2.2s, 2.6s, 3.11s, 3.12s, 4.8s
M12: Basic Productivity Tools Technology Application – Knowledges: 1.3K; Skills: 1.3S, 3.1s, 3.2s, 4.03s, 4.04s
M13: Online Resource Applications Technology Application – Knowledges: 2.1k, 3.2k, 5.2k; Skills: 2.1s, 2.2s
M14: Inductive Instructional Delivery Strategies Knowledges 3.5K-3.11K Skills 3.7S-3.14S
M15: Deductive Instructional Delivery Strategies Knowledges 3.5K-3.11K Skills 3.7S-3.14S
M24: TrackStar I Technology Application –Skills: 2.01s, 2.02s, 2.06s
M25: TrackStar II (Project) Technology Application –Skills: 5.01s, 5.03s
M26: TrackStar III (Project) Technology Application –Skills: 4.08s, 5.03s, 5.07s
M27: Spreadsheets and Databases in the Classroom
Technology Application – Knowledges: 1.1k, 1.2k, 2.2k 3.1k; Skills: 1.5s, 3.2s, 3.3s, 3.15s, 4.2s, 4.9s
Domain IV
PPR Test Framework Domain IV: Fulfilling Professional Roles and Responsibilities (approx. 23% of the test)
SBEC Standards Assessed PPR Standard IV: The teacher fulfills professional roles and
responsibilities and adheres to legal and ethical requirements of the profession.
M16: Code of Ethics for Educators
Knowledge 4.1K-4.8K Skills 4.1S-4.11S
M17: Responsibilities to Educational Partners
Knowledge 4.1K-4.2K Skills 4.1S-4.4S
M18: Professional Growth and Continuing Education
Knowledge 4.9K- 4.12K Skills 4.12S - 4.15S
Data Collection
In general, each time a user moves to a new page, a page time stamp is recorded to indicate that page has been finished.
Page Time Stamp (PTS) –
Module Instructional Time
Module Instructional Time (MIT) – For a given module, every time a given user
accesses a module an estimate of the time spent on that module is calculated by computing the differences between each subsequent page time stamp (estimate of time spent on page). Then all times are added up for all pages associated with that module.
Data Sources – MIT Data Values being recorded in the eZone database: Time = Total MIT in seconds converted to minutes Time Info = Records MIT and total number of pages
visited Time Excluded = Very large PTS values are
excluded from MIT calculations. However, these values are recorded in the database as reference data.
Page Time Stamp (PTS) values 60 minutes are excluded.
(Rationale: Spending 60 minutes or more on one page is recorded as idle time)
Participants PPR Data
PPR Data for all Accelerate Online students was downloaded from the CEHD Data Portal and merged with eZone Data
PPR Total Score as well as the scores across the four domain areas were obtained
N=42 (Module data from 42 students were used)
N=28 (PPR data from 28 students were used)
Procedures
Calculated the total module instructional time and module averages for all students, across all modules
Data Analysis
Using the SPSS application program, correlation methods were employed to explore the relationship between PPR scores and module performance (completion time, posttest scores).
Findings - Preliminary
PPR Score by Total Module Instructional Time
R2 = .000 prob = .999
PPR Score by
Module Average
R2 = .525** prob = .006
Total Module Instructional Time and
Module Average
R2 = .115 prob = .420
Note: This investigation is still in progress
** Correlation is significant at the 0.01 level (2-tailed)
Discussion
The results from this preliminary investigation suggest that the amount of time spent on a module in an elearning environment is not related to the module posttest score or PPR score.
However, the posttest score of teaching candidates is significantly related to PPR scores
Limitations
Small sample size – I would like to complete this analysis in a few months, with a greater n
Initially I wanted to explore relationships between Module Clusters (1-4) and PPR Domains (1-4) – a more complete data set, and a larger sample are needed
THIS IS A WORK IN PROGRESS
Evolving Research Agenda
eZone –eLearning System
Online Modules ePortfolios Classroom Observations Integrated Resources
My eAssessment Center
Cluster 1
Cluster 2
Cluster 3
Cluster 4
Module/Course Assignments
Early Field Experiences
Instructional Artifacts
Domain 1
Domain 2
Domain 3
Domain 4
Domain 5
Community Discussion Boards
Community ePortfolios
Domain 6
Domain 7
Resource Media Center
Break Room
Phase 1
Phase 3
Phase 2
Evolving Research Agenda
How do various candidate performance data: Module Assessments Classroom Observations ePortfolio Assessment - Instructional Artifacts
Relate to key outcome measures (performance on PPR exams, program completion and retention, student performance …).
Results of Implementing Preservice Teacher Development Program
Year 1 Numerator Denominator Success Ratio
Candidates Placed/Total Candidates 10 10 1.00
Teachers Certified/ Total Candidates 8 10 .80
Teachers Retained/ Total Candidates 7 10 .70
Year 2
Candidates Placed/Total Candidates 19 20 .95
Teachers Certified/ Total Candidates 18 20 .90
Teachers Retained/ Total Candidates 17 20 .85
Year 3*
Candidates Placed/Total Candidates 40 46 .87
*The Year 3 Ratios of Teachers Certified and Teachers Retained are not provided because many of the Year 3 candidates are completing their final field experience.
The results from this preliminary investigation suggest that engaged instructional time in an elearning environment within the context of a self-paced instructional module, doesn’t necessarily yield the same results as studies that focus on engaged time and academic learning time in face-to-face environments. Several studies involving f-2-f settings suggest that there is a positive relationship between time and student achievement. I would like to explore the literature more in these areas.