Identifying Research-based Practices for RtI: Scientifically-based Reading Instruction

Presented at the 2nd AnnualEvidenced-Based Practices Summit

of the Wing InstituteBerkeley, CAApril 26, 2007

Janet S. Twyman, Ph.D., BCBA

Historical Perspective: Reading in America After a century and a half of universal public

education, and despite the highest per-pupil expenditure on public elementary and secondary education in the world,40 percent of U.S. fourth-graders are reading below the minimally acceptable level (NAEP 2006).

For minority students in inner-city schools, the reading failure rate is 65 percent.

Children who don’t read by fourth grade almost always fall behind in all other subjects, frequently require special education programs, and, as adults, have higher rates of drug addiction, incarceration, and welfare dependency.

Legislation to Avert our National Reading Crisis

The No Child Left Behind Act of 2001 commits the nation to ensuring that every child in the country will learn to read by the end of third grade.

Within NCLB, Reading First was first authorized at more than $6 billion to apply scientifically based reading research to strengthen children’s reading skills from kindergarten through third grade. Yet, at $1 billion per year it accounts for only 2 percent of federal education spending.

Reading First – An Overview

As part of NCLB, it mandates that the best scientific research should guide the teaching of reading.

It supports research-based methods to ensure that every child in public schools reads at or above grade level by third grade.

The essential components of reading instruction are defined in the bill as explicit and systematic instruction in phonemic awareness; phonics; vocabulary development; reading fluency, including oral reading skills; and reading comprehension strategies.

Reading First – What it Means to Schools Preparing teachers through professional development

and other support, so they can identify specific reading barriers facing their students and have the tools to effectively help their student to learn to read.

Selecting or developing rigorous diagnostic reading assessments that document (1) effectiveness in improving students' reading and (2) the schools accountability for their results.

Selecting or developing effective instructional materials, programs, and strategies to implement methods that have been proven to prevent or remediate reading failure (including newly identified Response to Intervention Initiatives).

Establishing reading programs for students in grades kindergarten through 3 that are grounded in scientifically-based reading research (SBRR), in order to ensure that every student can read at grade level or above by the end of the third grade.

What the Law says is Scientifically-Based Reading Research

The term `scientifically based reading research' means research that--

(A) applies rigorous, systematic, and objective procedures to obtain valid knowledge relevant to reading development, reading instruction, and reading difficulties; and

(B) includes research that--(i) employs systematic, empirical methods that draw on observation or experiment;(ii) involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn;(iii) relies on measurements or observational methods that provide valid data across evaluators and observers and across multiple measurements and observations; and(iv) has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review.

Source: H.R.1. 2001

“Scientifically-based” - Controversies Abound

There little agreement about what constitutes “evidence.”

NCLB permits both quantitative and qualitative evidence (without clearly specifying the types of questions best answered by each approach).

Only small number of interventions used today have been empirically evaluated by some peer-reviewed standard.

Evidence-based interventions are less likely to be used than interventions for which there is no evidence or evidence showsa lack of impact (Kazdin, 2000).

In many instances educators/practitioners are not aware of evidence-based interventions (Kratochwill, Albers & Shernoff, 2004).

Source: Detrich, Keyworth, & States (2007). Evidence-based Education: It Isn’t as Simple as You Might Think.

What the education field refers to as “Research-Based”

Current uses of the term research–based in federal documents and journal articles (referred to as “evidence” when applied to early reading programs):

(1) the program contains elements that research suggests are effective (see for example Simmons & Kame’enui, 2003)

(2) pretest vs. posttest or simple comparison studies have provided some evidence of effectiveness for an instructional program, or

(3) the program has undergone some form of scientifically controlled study, often involving randomized control groups (Coalition for Evidence–Based Policy, 2003)

However,

…these uses fail to distinguish between the

scientific development of a program,

and the scientific evaluation of

outcomes after a program has been

developed.

Types of “Research” to consider when evaluating programs: Research-Informed

A program based upon best practices identified by research or contain elements that other research suggests are effective.-No requirement that the program itself be scientifically designed.

Research-Filtered (Summative Evaluation)A program, regardless of how it was designed, measured against an alternative form of instruction, or at times, no instruction at all. -No requirement that the program itself be scientifically designed.

Research-Guided (Formative Evaluation)A program that has been scientifically designed and tested during its development (most often guided by previous research results). In the research–guided approach, formative evaluation is intertwined into the instructional design protocols, and at its most thorough influences program development through iterations of testing, revising, and testing again.

How does one know what “evidence” to use?

Evidence is more than ONE study showing support. Involves a body of knowledge, specifying:

the quality of evidence the quantity and type of replication of effects

Level or rigor of evidence falls along a continuum. Sufficient Threshold:

Evidence must be of a specific quantity and quality before a program should be considered evidence-based.

Hierarchy of evidence approach: Rely on the best available evidence to date.

Source: Detrich, Keyworth, & States (2007). Evidence-based Education: It Isn’t as Simple as You Might Think.

Continuums of Evidence

Quality of the Evidence

Personal Observation

Expert Opinion

Current “Gold Standard”High Quality

Randomized Controlled Trial

Uncontrolled Studies

General Consensus

Single Case Designs

Semi-Randomized Trials

Well-conducted Clinical Studies Threshold

of Evidence

Program Development

Research-Filtered

Research-Guided

Research-Guided and Informed

Research-Informed

Consensus of Experts

Common Wisdom or Lore

Quantity of the Evidence

Meta-analysis (systematic review)

Single Case Replication (Direct and Parametric)

Single Study

Various Investigations

Repeated Systematic Measures

Convergent Evidence

An Example:How states determined their Reading First programs:

States Report Difficulty in Determining Evidence-based Reading Programs

Are we preparing our teachers?

Source: National Council on Teacher Quality

85% earned a “failing” grade, even though a passing grade was possible if a professor devoted less than 20 percent of the lectures to the science of reading.

All 5 4/5 3/5 2/5 1/5 None ofcomponents the components

How can we expect our

educators to understand

evidence-based instruction

in reading, if our teacher

preparation institutions aren’t

teaching reading science?

When faced with “evidence”

…ask at least three questions: Is the evidence valid?

Where does the evidence fit within the continuum? Could it be biased?

Is it important? Are the findings to relevant your situation? What is the significance of the outcome compared to other options?

Is it applicable to the learner in front of me?Are the participants and procedures in the original research comparable to your learner and setting?

Sources for “Evidence”

http://www.whatworks.ed.gov

Assistance in determining evidence:

Florida Center for Reading Research

http://www.fcrr.org

www.fcrr.org/FCRRreports

Provides one of the most

comprehensive lists of reviews

of reading programs

Oregon Reading First

http://oregonreadingfirst.uoregon.edu

“Curriculum Review” leads to the Consumer's

Guide to Evaluating Reading Programs an evaluation of various reading programs.

Big Ideas in Beginning Reading

http://reading.uoregon.edu/curricula/

Promising Practices Network

http://www.promisingpractices.net/programs.asp

National Institute for Literacy

http://www.nifl.gov/cgi-bin/pfr/search.cgi

California Learning ResourceNetwork

http://www.clrn.org

Logging in provides access to

evaluations of reading

programs

Interventions in School Psych.

http://www.sp-ebi.org

“Projects” leads to:

Procedural and Coding Manual for

Identification of Evidence-Based

Interventions

Other Interventions

http://nrepp.samhsa.gov/index.htm

IRIS Center – Star Legacy Modules

Interactive Tutorials on:

ACCOMMODATIONS BEHAVIOR COLLABORATION DIFFERENTIATED INSTRUCTION Response to Intervention Reading StrategiesDISABILITY DIVERSITY

Scientifically Based Reading Research (SBRR)http://www.headsprout.com/aol/teachers.cfm

Reading FirstTechnical Assistance Centers

Western Regional Reading First Technical Assistance Center (WRRFTAC)Center on Teaching & LearningUniversity of Oregon (Eugene)(541) 346-1608

Central Regional Reading First Technical Assistance Center (CRRFTAC)Center for Reading & Language ArtsThe University of Texas at Austin(512) 232-1901

Eastern Regional Reading First Technical Assistance Center (ERRFTAC)Florida Center for Reading ResearchFlorida State University (Tallahassee)(850) 644-9352

Wing Institute Home Page

What does this mean for real educators, real students, real classrooms?

Find and use programs with rigorous levels of research Programs furthest along the continuum of empirical

evidence of effectiveness (quantity, quality, and type of program development)

Access available public resources reviewing “evidence” Learn from the community of available tools and

tutorials Use and be responsive to data from your own students,

classroom, school CCCROD CBM Progress Monitoring

DIBELS* AIMSweb*

Need for Research into Practice

A key outcome of evidence-based education is for scientifically proven interventions to be adopted and successfully implemented at the “practice” level.

In order for research to translate into practice, interventions must be efficacious (have a foundation in rigorous scientific research) as well as effective (address the necessary social influence factors to ensure the desired outcomes).

Source:

RtI facilitates the use of evidence and research-based programs

RtI framework encourages a system for identifying: need, treatments, measuring progress and outcomes, evaluating teaching practices, and revising whenever the data indicate to do so.

RtI is integral to knowing what is working and what is not, for each individual child.

Relevance for Special Education students For some students require the modification

of the general education reading curriculum, in terms of content or instructional approach.

Other students may require general education classroom reading instruction to be supplemented with an alternative, more intensive program.

Yet other students may require a complete replacement reading program, including other publicly available materials, or “teacher-devised” curricula.

Suggested Principles Regarding Reading Instruction for Special Education Students

Address all areas of the reading process as identified by reading research.

Identify, early, students demonstrating difficulty learning to read. Design a tier (continuum) of intervention services prior to referral to

special education. Include in each special education student’s individual plan of

instruction carefully selected curriculum and instructional techniques matched to student needs.

Base instructional plans on initial and ongoing assessment that reflects research-based best practice, including a historical analysis of the student’s reading instruction, consideration of the student’s specific characteristics and knowledge of the student’s deficits and strengths.

Use assessment data to guide and inform instruction. Vary the intensity of intervention (amount of time for reading

instruction, the size of the instructional group, means of service delivery, and the nature of the selected reading curriculum) provided to individual special education students based on the level of need and progress made (or lack thereof).

Use technology, as appropriate, to enhance the development of reading skills and to ameliorate the effects of deficiencies in reading.

Handout available in resource packet.

IMPORTANT CAVEAT

Don’t become formulaic;even with SBRR or RTI

Never lose sight of your own ongoing data collection and analysis