Michael FloryCNA Education

Roland O’Daniel & Jo Ann MosierCollaborative for Teaching & Learning

April 2011

The Effects of a Hybrid Program for Algebra I on Grade 9 Students’ Mathematics Achievement

Innovation Conference December 2011

2

Agenda

Introductions, REL Appalachia at CNA Hybrid InstructionResearch QuestionsThe KYVS hybrid program for Algebra I– Materials– Processes– Video

Research DesignPerceptions from CTL’s Instructional Specialists

3

hybridalgebra.wikispaces.com

4

REL Appalachia at CNA

One of 10 regional labs, funded by the Institute for Education Sciences, USDE.

Major activities– Long-term evaluations– Shorter-term analyses– Field scientist program– Quick turn-around--

Ask-a-REL, Bridging Research to Practice

Research Team

NameName AffiliationAffiliation RoleRole

Dr. Linda Cavalluzzo CNA Education PI

Dr. Deborah Lowther

Dr. Christine Mokher

Education Innovations & University of Memphis

CNA Education

Co-PI

Researcher

Dr. Xitao Fan UVA Methodologist

Implementation Team

NameName AffiliationAffiliation RoleRole

Dr. Linda Hargan/Jo Ann MosierRoland O’Daniel

Collaborative for Teaching and Learning (CTL)

Site recruiter/Instructional Specialists, PD Algebra 1 instruction

Kari Welch Kentucky Virtual High School (KVHS)/KDE

Courseware selection and access; Training for Master Teachers; Access to student data

7

Online vs. hybrid instruction

• Online learning– Teacher of record is at a distance

– Student can be anywhere that has internet access

– Can bring a course to, A school that otherwise could not offer it A student needing to repeat it A student that wants to take an advanced course

• Hybrid learning– Online resources complement instructional experience

Courseware or supplemental materials in the classroom Before or after school access to supplemental materials Use of online communications to address student questions

8

The KYVS Hybrid Program for Algebra I

• Algebra I teachers instruct students using online resources in a face-to-face classroom setting– Teachers have a computer with internet and screen for instruction in

their classroom

– Students have individual access 2 days/week

• Teachers receive sustained Professional Development (PD)– Learn to use technology to facilitate effective instructional practices

9

Research questions

• What is the impact of the KYVS Hybrid Program for Algebra I on grade 9 students’: – Math achievement in pre-algebra/algebra in the fall of grade 10 on the

PLAN assessment?

– Subsequent math course enrollment in the post-intervention year?

• Does the impact of the intervention vary between: – Male and female students?

– Rural and non-rural schools?

10

Study Design and Sample

Randomized control trial

41 schools across KY offering Algebra I to 9th graders (6,000 students)

2 Cohorts– 25 schools in Cohort I, SY 07/08– 16 new schools in Cohort II, SY 08/09– Each school remains in their group for 2 years

11

ALGEBRA I HYBRID RESEARCH SCHOOLS2007-2011

12

Key Internet-based resources

• For teachers: Spotlight on Algebra I

• For students: NROC Algebra I

• LMS: Blackboard

13

The KYVS Hybrid Algebra I Program

3-day F2F orientation and start up

5 Synchronous online summer sessions

1-day end-of-summer F2F capstone

During school year– Monthly online sessions– 2 site visits from instructional specialists

14

Notable Features of the Professional Development

Sustained

Work with a learning community

Active learning

Models the practices teachers will use in their classrooms– Effective practices in a hybrid learning

environment

15

kvhs.blackboard.com

Username: roland.odanielPW: password

“My Courses” tab

16

kvhs.blackboard.com “My Courses” tab

Username: roland.odaniel PW: password

17

18

19

20

21

22

23

24

25

In the classroom

The hybrid program facilitates use of effective instructional practices– Teachers set learning goals for each class– “Bell-ringer” a warm-up activity to access prior

knowledge– Graphic organizers help students monitor

progress through lessons– “Exit slip” a closing activity to assess and

respond to student learning– Mathematical Literacy Strategies

26

In the lab

Students work alone or in pairs

Use a note-taking device to write down what they understand, and what they don’t

Teachers become facilitators of learning– Providing individual assistance as needed, or

teach a mini-lesson

27

28

Hybrid Algebra 1 Classroom Model

Structures and processes

– Transparency of learning goals

– Transitioning from classroom to virtual classroom

– Tracking tool or note taking device

– Learning cycle components

– Intentional whole group reflection (guided discourse)

29

Hybrid Algebra 1 Classroom Model

Typical lessons

– Starter activity – Learning expectations (learning product,

transitioning from classroom to the laboratory) – KYVS lesson (warm-up, new learning with

explorations & use of applets)– Whole group reflections with guided discourse– Assignment– Next steps

30

http://www.hippocampus.org

31

http://www.hippocampus.org

32

33

34

35

Program evaluation

• Randomized Controlled Trial (RCT)• Gold Standard for measurement of impact

– High internal validity

– Causal by design

– If properly randomized, differences between T and C groups are random, except for exposure to treatment

• Measures the Impact of the Intent to Treat, and not the impact of the treatment itself

36

Theory of Action

37

Study Design: Instruments and Data Collection

• PRE-INTERVENTION:– 2 pretests in grade 8– Student Information

Demographics Enrollment records

• DURING INTERVENTION– Classroom observations– Teacher survey

• POST-INTERVENTION:– 10th grade statewide test– 10th grade course– Algebra I end-of-course test

Administered by researchers Could not use in final report due

to low response rates and differential attrition by treatment status

38

Power Analysis—before and after recruiting

• Underlying assumptions to determine MDE– Number of observations at each level, intraclass correlations, cluster-

level covariates, level of significance, potential attrition

39

Attrition – PLAN Outcome

• Consolidated Standards on Reporting Trials (CONSORT)

• PLAN Assessment of Pre-Algebra/ Algebra Knowledge and Skills

• Response rate:

84% Treatment

86% Control

40

41

Perceptions of Instructional Specialists--Strengths

Establishment of a Community of Learners Engagement of teachers as learners while in the distance environment: small group interaction with whole group

report out. reading articles prior to the session – sharing

ideas and intentional connections to the hybrid environment.

designing session topics that deepen understanding of the hybrid environment and are responsive to learning needs.

42

Perceptions of Instructional Specialists--Strengths

Creation of classroom structures and processes supported through modeling and materials.

Extension of content knowledge and best practices in teaching Algebra I within the hybrid model.

Formation of new teaching habits needed within the hybrid approach.

43

Perceptions of Instructional Specialists--Challenges

Ensuring fidelity of implementation.

Maintaining a sense of urgency and need for another approach to learning.

– Transitioning the learner from face-to-face instruction to the virtual independent learning environment.

– Planning instruction that bridges learning from classroom to the virtual environment.

44

Perceptions of Instructional Specialists--Challenges

Accessing the technology that does not always cooperate (e.g., proxy issues).

– Affects classroom and PD sessions

Logging in and engaging in the distance learning session.

45

Status Report

Final Report – Draft report submitted for review, June 2010– In final phase of review– Release of final report expected, Spring

2012– RELappalachia.org

46

For Additional Information… www.relappalachia.org

Linda Cavalluzzo,CNA703-824-2197

cavallul@cna.org

Jo Ann Mosier or Roland O’Daniel, CTL 800-995-3965

jmosier@ctlonline.org or rodaniel@ctlonline.org

Grace YehKentucky Virtual School, KDE

502-564-4772Grace.Yeh@education.ky.gov