Enhancing Academic Achievement and Transition Outcomes

through Technology

2010 OSEP Project Directors’ ConferenceJuly 21, 2010

Margo Vreeburg Izzo, PhDIzzo.1@osu.edu

Alexa J. Murray, MAAlexa.Murray@osumc.edu

http://nisonger.osu.edu/Transition/EnvisionIT.htm

Brief Project Overview

• Background

• Need for project

• Findings

• Discussion Questions

Transition Realities forYouth with Disabilities

The National Longitudinal Transition Study 2 (2004) and Ohio Longitudinal Transition Study (2004) show:

• A large gap between what youth with disabilities say they want and what they actually experience after high school remains.

• SwD attend and remain at postsecondary institutions at much lower rates than their non-disabled peers.

Questions:• How can we improve the high school curriculum to

assist students in transition?

Need for Information andTechnological Literacy

• Defined as the set of skills needed to find, retrieve, analyze and use information.

• Individuals who are both technologically and information literate will be able to use computers effectively to find the information they need.

• Useful for people with disabilities because they enhance communication, learning, writing, and task management.

• Creates equal opportunities in education and aid in transition from school to work.

• Teachers and students appreciate the independent self-directed learning technology can provide.

Need for Academic & Transition Planning Skills

• EnvisionIT integrates information literacy and career/transition skills into English and Technology classrooms.

• Students become more engaged and invested in their learning by using material that is relevant to their lives.

• Examples – Online Interest Surveys, Learning and Personality Assessments.

6

Need for 21st Century Skills

The skills students need to succeed in the 21st Century:

• Life and Career Skills• Learning and Innovation Skills• Information, Media and Technology

Skills• Core Subjects with 21st Century

Themes (Partnership for 21st Century Skills, www.21stcenturyskills.org)

7

21st Century Curriculum

EnvisionIT Curriculum

• Teaches IT skills.

• Help students build a self-directed Transition Portfolio.

• Match students’ interests, abilities, personality to career goals.

• Delivered through CMS

EnvisionIT Curriculum

An online curriculum for ALL high school students that teaches:

• Information Technology (IT) Literacy

• Self-directed Transition Planning

• National and Ohio English/Language Arts and Technology Standards

• Age Appropriate Transition Assessment

Transition Assessment

• How does EnvisionIT meet assist with the transition assessment?• Self-assessments (Unit 2 & 3)• Goal setting (Unit 6)• Self-evaluation of career goals (Unit 6)• Family Feedback (Unit 10)• Transition Portfolio as a whole

Examples of Activities

• General Directory (6.9)

• Using a Career Portal (6.11)

• Career Objective (6.12)

• Goal Setting for the Future (6.13)

• High School Course Schedule (6.14)

• PowerPoint (6.15)

• Bookmarks (6.16)

Unit 10 – Implementation

• Students will be sharing their portfolios with their families for feedback and improvement. Encourage students to:

• Share portfolio with family members at student directed IEP meeting or parent/teacher conference

Share with another teacher or guidance counselor

Invite families to classroom for group presentationHave students take portfolio home for feedback

Have students present their portfolio to the class

Examples of National Information Technology Standards met in

EnvisionIT Curriculum • Students are proficient in the use of technology.

– Unit 7, Activity 1: Students investigate how using the correct operators in a search can affect outcomes.

• Students understand the ethical, cultural, and societal issues related to technology.– Unit 8, Activity 1: Students will answer a list of

questions to evaluate the validity of Web Searches.

Examples of National English/Language Arts Standards

met in EnvisionIT Curriculum• Students employ a wide range of strategies as they write

and use different writing process.– Unit 5, Activity 4: Students write a narrative essay using

their Career narrative outline and previous research.• Students apply knowledge of language structure,

conventions, media techniques, etc, to create, critique, discuss text.– Unit 9, Activity 2: Students learn about online article

databases by searching for articles and cite their sources.

Examples of Ohio English/Language Arts Standards

met in EnvisionIT Curriculum• Students analyze the relationships of pairs of words and infer

word meanings from these relationships.

– Unit 2 Activity 1. Students take at least one of three identified self-assessments to discover their learning style, strength, or personality trait and write a paragraph explaining their results.

• Students apply reading comprehension strategies, including making predictions, comparing and contrasting, recalling and summarizing and making inferences and drawing conclusions.

– Unit 10 Activity 3. Students compare either their top two job or college choices by completing the corresponding Preparation Chart with information they have gathered.

Tools of the Trade

• Teacher’s Guide

• Appendices

• UDL Supports

• Checklists

• Experimental Protocols

• Technical Assistance

Curriculum Intervention StudySteppingstones Phase II

A pretest-posttest control group design was used to answer the following research questions:

1. Did the EnvisionIT online transition curriculum increase student IT literacy?

2. Did the EnvisionIT online transition curriculum increase student transition planning skills?

Methods: Setting & Participants• 15 high schools randomly assigned stratified by

SES• 7 experimental schools• 8 control schools

• Inclusive and Resource Room/Direct Instruction Classrooms

• English/Language Arts classrooms & Technology classrooms

• Year 1 N=287, Year 2 N~268

Data Collection:AIMSweb

• Pre/post• Benchmark: Reading independently at 8th

grade level, no supports necessary.

• Strategic: Reading instructionally at 8th grade level, independent at 6th grade, needs Guided Notes, Review Sheets.

• Intensive: Reading below 6th grade level, needs Guided Notes, Review Sheets, AT.

Dependent Measures: IT Literacy

Information Technology Literacy Survey

• 21-item multiple choice test used as pretest and posttest

• The curriculum-based measurement was found to be reliable (Cronbach’s Alpha =.822, ICC =.676)

• Example Question: Web directories and search engines are created differently. Search engines are made by ________.

Data Collection: IT-Literacy

• Pre/post• 23-item• Curriculum based• Validated• Covers all IT

concepts in Units 2-9

Data Collection: Focus Groups• Student Focus Groups

• Mid-year focus groups• 5-7 students• After post-test is completed

• Teacher Focus Groups:• Conference Calls• June 16, OSU Campus• Compensation, lunch provided

Dependent Measures: Transition Skills

Ohio State Career Survey

• 23 item survey to measure students’ perceptions of their transition skills

• The curriculum based measurement was found to be reliable (Cronbach’s Alpha =.847, ICC = .740).

• Example Question: Please rate your knowledge about finding jobs.

Data Collection: OSU-Career Survey

• Pre/post

• Transition skill assessment

• 23 Multiple Choice Questions

• Demographic Information

Results: Analytic Sample

Year 1 Participants (N=287)

• 59% did not have a documented disability.

• 41% had a documented disability,

• 17% had a learning disability

• 14% had mental retardation

• 11% had other documented disabilities (e.g., autism, multiple disabilities, SED/SBH, Speech, TBI and OHI).

Results: Analytic Sample

• 80% were from rural school districts

• 55% were male

• 87% were of Caucasian ethnicity, 12% are African-American, Hispanic, Asian, or of other decent.

• 47% were 9th graders (65% in experimental group compared to 25% in control group)

• 68% were not reading either independently or instructionally at grade level.

EnvisionIT Year 1 Descriptive StatisticsVariable Experimental (N=153) Control (N=134)

School SettingSuburban 17% 25%

Rural 83% 75%

GenderMale 56% 55%

Female 44% 45%

Ethnicity

White 93% 82%

African-American 3% 8%

Other 4% 10%

Classroom SettingResource Room 34% 47%

Inclusive 66% 53%

Disability StatusNo Disability 59% 57%

Disability 41% 43%

Reading Ability

Benchmark 32% 22%

Strategic Intervention 35% 47%

Intensive Intervention 33% 31%

Grade Level9th Grade 65% 25%

10th-12th Grades 35% 75%

Table 1. Comparison of mean change in Information-Technology Literacy Scores by Student Classification Group.

Experimental Control t value

p value*

Total Sample 3.493 0.571 3.5446 .0005* Disability Status Students with disabilities 1.241 -0.125 1.4265 .1549 Students without disabilities

5.745 1.268 3.6013 .0004*

Grade Level 9th graders 2.883 1.186 0.9573 .3393 10th-12th graders 4.103 -0.043 3.7989 .0002* AIMSweb Classification Intensive Intervention 1.294 -0.573 1.6787 .0944 Strategic Intervention 3.666 1.229 2.3133 .0215* Benchmark Intervention 5.518 1.059 3.4553 .0006* School Location Suburbun 4.506 1.013 2.5605 .0110* Rural 2.480 0.130 3.1184 .0020*

*after controlling for the other predictors in the model, p ≤ .05.

Results: IT Literacy

Transition Results:Knowledge of Finding Jobs

Experimental students with and without disabilities had significant gains in knowledge post-test compared to control students with and without disabilities.

Mean Lower 95% Upper 95%

Students without Disabilities

Experimental .2747 .0402 .5093

Control .1558 -.1073 .4189

Students with Disabilities

Experimental .3226 .0149 .6303

Control .2632 -.1105 .6368

Transition Results:Finding College Information

Compared to control students with disabilities, experimental students with disabilities had greater gains in reported ability to find college information.

Mean Lower 95% Upper 95%

Students with disabilities

Experimental .4677 .1307 .8048

Control .3333 -.0594 .7260

Discussion• Students who completed EnvisionIT:

• Improved their IT Literacy Scores

• Reported increased knowledge of how to find jobs (all students)

• Reported increased knowledge of college information (SwD only)

• Data analysis to continue for 2007-08 school year.

• All schools receive intervention 2008-09.

Fidelity Definition

• “In intervention research, treatment fidelity is defined as the strategies that monitor and enhance the accuracy and consistency of an intervention to ensure it is implemented as planned and that each component is delivered in a comparable manner to all study participants over time.” (Smith, Daunic, & Taylor, 2007)

Fidelity: Year 1

• Site observation

• Dichotomous scale

• Collected at least 2X per year/semester

• Inter-rater reliability

• Teacher rating included

Fidelity: Year 1

Fidelity: Year 2• Site observation

• Refinement of tool

• Change in scale from dichotomous to Likert for greater sensitivity

• Collected at least 2X per year/semester

• Inter-rater reliability

• Teacher rating included

Fidelity: Year 2

Fidelity: Lessons Learned• It is possible for a teacher to meet fidelity

without implementing good instruction (as measured by academic outcomes)

• Challenge is refining the fidelity tool so that good instruction is accurately captured

• Repeated observations are necessary, especially for those teachers not meeting fidelity

• Multiple raters should be used to reduce bias but inter-rater reliability should be assessed

References

• Izzo, M., Yurick, A., Nagaraja, H., & Novak, J. (2010, In Press). Effects of a 21st Century curriculum on students’ information technology and transition skills. Career Development for Exceptional Individuals

• Izzo, M., Yurick, A., & McArrell, B. (2009) Supported eText: Effects of text-to-speech on access and achievement for high school students with disabilities. Journal of Special Education Technology, 24 (3), pp. 9-20

• Smith, S., Daunic, A., & Taylor, G. (2007, November) Treatment fidelity in applied educational research: Expanding the adoption and application of measures to ensure evidence-based practice (Report). Education and Treatment of Children