Short forms of:Assessing and Improving Curriculum Materials andAssessing and Improving Instruction and the Classroom Environment

CURRICULUM MATERIALS

1. Materials (especially programs) (1) are consistent with scientific research on instruction (this is called “research based”); and (2) have been field tested and shown to be effective with scientific research (this is called “evaluation research.” Level 3 is preferred).

Are claims of effectiveness based on empirical research or on a sales pitch?

Is there any research on the materials?What level(s)?

Is the research (“research base”) generally adequately designed so that credible conclusions can be drawn?

a. Some resources on scientific research.

Kozloff. Research vocabulary.

Three Levels of Research

Telling the difference between baloney and serious claims doc

Telling the difference between baloney and serious claims ppt

Assessing the Quality of Research Plans and Publications ppt

Checklist of guidelines for evaluating research and research claims doc

Checklist of guidelines for evaluating research and research claims html

Comprehensive Guidelines for Evaluating Research and Publications

Some resources on “research based instruction.”

Are features of the materials (e.g., what is taught, scaffolding) consistent with scientific research?

1. Cotton, K. (1995). Effective school practices: A research synthesis 1995 Update.

http://www.nwrel.org/scpd/esp/esp95.html

2.    Kozloff, M.A. (2002). Sufficient Scaffolding, Organizing and Activating Knowledge, and Sustaining High Engaged Time.

http://www.uncwil.edu/people/kozloffm/scaffolding.pdf

3.    Rosenshine, B. (1997). Advances in Research on Instruction.

http://epaa.asu.edu/barak/barak.html

4.    Rosenshine, B. (1997). The Case for Explicit, Teacher-led, Cognitive Strategy Instruction.

http://epaa.asu.edu/barak/barak1.html

5.    Ellis, E.S., & Worthington, L.A. (1994). Research Synthesis on Effective Teaching Principles and the Design of Quality Tools for Educators.

http://idea.uoregon.edu/~ncite/documents/techrep/tech05.pdf 

6.    Anderson, J.R., Reder, L.M., & Simon, H.A. Applications and Misapplications of Cognitive Psychology to Mathematics Education. Department of Psychology. Carnegie Mellon University. Pittsburgh, PA 15213.

Online at http://act.psy.cmu.edu/personal/ja/misapplied.html

7.   Dixon, R. "Review of High Quality Experimental Mathematics Research." University of Oregon.  National Center to Improve the Tools of Educators.

       On-line at http://idea.uoregon.edu/~ncite/documents/math/math.html

8. Follow Through. The Largest Education Evaluation

Effective School Practices, on Project Follow Through.

2

Follow Through figure 1.

Follow Through figure 2.

Follow Through figure 3.

9. Reading

Reading First. [Main features of effective reading instruction] ppt

Reading First.html

Big Ideas in Beginning Reading.  Identification of the big five reading skills; research reviews; methods of instruction.  MUST SEE!  

"Put Reading First: The Research Building Blocks for Teaching Children to Read, Kindergarten Through Grade 3

Materials from National Institutes of Child Health and Human Development

Materials from National Institutes of Child Health and Human Development

National Institute for Literacy Partnership for Reading.   Large literature reviews and position papers.

"Teaching Reading IS Rocket Science" (Must read!!! What teachers need to know.)

c. Some resources on evaluation research.Go to publishers’ websites and look for research on products. Also, use Google and JSTOR.

https://www.sraonline.com/

Sopriswest materials.

Curriculum Associates materials.

Hempenstall. Research on 100 Easy Lessons .

Haddox. Research on 100 Easy Lessons .

3

Strengths, Weaknesses, Improve How?

Teach Your Child to Read in 100 Easy Lessons: (1) is research based (its features have been tested and validated by scientific research (e.g., Cotton, Ellis, Rosenshine); and (2) has been field tested.

Research based. Features of 100 Easy Lessons that are research-based include the following: (1) distributed practice (cumulative review) to build retention.

(2) systematic (step by step), explicit (teacher explains and shows), direct (teacher focuses on objectives) instruction (e.g., model, lead, test).

(3) work on fluency and generalization.

(4) clear objectives.

(5) lessons involve review and firming of recent skills and pre-skills; teaching new material; review of new material.

(6) all errors are corrected.

Field Test(1) 100 Easy Lessons is a shortened version of Reading Mastery. Reading Mastery has been extensively tested and shown to be effective.https://www.sraonline.com/products.html?tid=9&sid=76

(2) Reading Mastery (originally called DISTAR Reading) was tested with thousands of children in project Follow-Through. Effective School Practices, on Project Follow Through.

(3) 100 Easy Lessons was also field tested in a project conducted by Kerry Hempenstall. Research on 100 Easy Lessons .

2. Curriculum materials (lesson-based programs and textbooks) should teach knowledge systems, such as math, beginning reading, biology, history.

You should NOT use materials that teach faddish, unvalidated, or

4

fashionable “methods,” such a multiple intelligence, learning styles, and brain-based instruction.

Strengths, Weaknesses, Improve How?

100 Easy Lessons teaches the knowledge system of beginning reading.

3. Well-designed materials provide a comprehensive and varied sample of knowledge (e.g., equations to solve, poems to analyze, words to decode).

Note. You are supposed to “align” instruction with your state’s standard course of study. But who says that IT is adequate? You have to rely on research and expert opinion.

a. See state standard course of study.

http://www.dpi.state.nc.us/curriculum/ncscos

Curriculum Standards

b. See expert opinions on different subjects or knowledge systems.

Reading. http://people.uncw.edu/kozloffm/reading.html

Reading First ppt and html

Big Ideas in Beginning Reading

Strengths, Weaknesses, Improve How?

100 Easy Lessons focuses on four of the elementary reading skills: phonemic awareness, alphabetic principle (letter-sound correspondence and sounding out/decoding); fluency, and comprehension.

The main phonemic awareness skills taught are blending (compound words and regular words, such as run) and segmenting.

The alphabetic principle is taught throughout the program.

5

Comprehension begins in lesson 13 and continues throughout the program. It begins with picture comprehension and then moves to story comprehension.Comprehension questions are literal (the answer is stated in the text), inferential (the answer requires integrating information spread in the text), and evaluative (e.g., “Why?” questions).

A variety of fluency skills are taught: saying sounds in isolation fast; reading letter-sounds fast; reading words fast; reading word lists fast; reading sentences fast; reading stories fast.

100 Easy lessons has little work on the fifth basic skill—vocabulary.

Procedures for Teaching Vocabulary to Add to 100 Easy Lessons

Teaching Vocabulary by Modeling (Examples):When it is impossible to use language to explain the meaning of a word (e.g., between, in). Model positive and negative examples of the new concept.

(e.g., "This is a mitten." or "This is not a mitten."). Test student on their mastery of the examples (e.g., "Is this a

mitten or not a mitten?"). Present different examples of the new word along with

examples of other previously taught words. Ask for names (e.g., "What is this?", "What color is this?" or "Tell me how I'm writing.").

6

Teaching Vocabulary by Synonyms:When a student knows a word(s) that can explain the meaning of a new, unknown word (e.g., damp means a little wet). Teacher equates a new word (huge) with a known word(s)

(very big). (e.g., "Here is a new word. Sturdy. Sturdy means strong.").

Teacher tests a set of positive and negative examples for the new word. (e.g., "Tell me sturdy or not sturdy.").

Teacher provides practice in applying several recently taught synonyms. (e.g., "Is that sturdy? Is it tidy? Is it mild?").

When Teaching Synonyms Use words students know Test on a range of positive and negative examples

“Huge means very big.”

“What does huge mean?”

“Tom put his pet in his pocket. Was his pet huge?”

“The animal wouldn't fit through the door. Was the animal huge?”

7

Teaching Vocabulary by Definitions:When students have adequate language to understand a longer explanation and when the concept is too complicated to be explained through a synonym (e.g., service station is a place where gasoline is sold and cars are repaired). Teacher tells the students the definition and has them repeat

it. (e.g., "An exit is a door that leads out of a building. What is an exit?").

Teacher tests the students on positive and negative examples to ensure that the students understand the definition and that they are not just memorizing a series of words. ("Is this an exit or not an exit? How do you know?").

Teacher provides a review of previous words. ("What is this? How do you know?").

4. Well-designed curriculum materials have scope and sequence charts (or at least subject matter outlines) showing how knowledge is organized—what is covered, and when.

Scope and Sequence for 100 Easy Lessons

Language arts.http://www.ncecorp.com/scopeandsequenceread.htm

Strengths, Weaknesses, Improve How?

100 Easy Lessons does not contain a scope and sequence chart. However, it does list all of the letter-sounds in sequence. (page 17)

5. In well-designed materials, lessons, units (sequences of lessons), or textbook chapters are built consistently from knowledge items selected from important strands (groups of knowledge). For example, each lesson or unit includes new vocabulary, big ideas, important facts.

Strengths, Weaknesses, Improve How?

Lessons draw on all of the strands, as these are introduced. For

8

example, later lessons work on new sounds (letter-sound correspondence) and decoding, fluency, and comprehension.

6. Well-designed materials, lessons, units (sequences of lessons), or textbook chapters state and focus instruction on specific objectives—what students will do.

Instructional Objectives Strengths, Weaknesses, Improve How?

Specific objectives are not started, but the objectives are obvious given the name of the task (e.g., “sounds introduction”) and how the task is framed (“Here’s the first sound you’re going to write”---p. 147).

7. Well-designed materials teach knowledge items in a logical sequence. They

a. Teach elements or parts (necessary pre-skills and background knowledge) before teaching new material that requires skill with the parts.

Pre-skills are always taught before the “composite skills” that include the pre-skills. For example, students learn to say sounds, to segment and blend sounds, and to read letter-sounds (m says mmm) before they are taught to blend letter-sounds into words. (Lessons 1-3)

b. Teach pre-skills and background knowledge early enough and continually, so that students are firm.

Pre-skills are reviewed almost every lesson. Pre-skills for the most complex skills (e.g., text comprehension) are introduced early and reviewed until the complex skill is introduced.

For example, text comprehension requires skill at pronunciation, blending sounds into words, letter-sound correspondence and sounding out, and fluency. These are worked on throughout the program.

9

c. Teach what is more general and more frequent before what is irregular or uncommon.

Regular words are introduced before irregular words. See lesson 39, Task 3 for the “funny word” format.

d. Separate instruction on similar and confusing knowledge items.

For example, b and d are separated by 42 lessons (p. 17).

e. Teach what is more useful before what is less useful.

Begin with terminal objectives and work backwards. Are pre-skills taught and reviewed before the current task that requires the pre-skills?

Do a skills trace. Pick a skill or strand (e.g., letter-sound correspondence). Are examples taught in a logical sequence?

Task Analysis

Strengths, Weaknesses, Improve How?

Program meets all of the above criteria for a logical progression.

8. Well-designed materials, lessons (math, writing, spelling, reading, or foreign language programs) or chapters (history or science textbooks) are a series of smaller, knowledge-rich units (chunks), such as tasks, exercises, or paragraphs. [No filler and pc baloney.]

Each chunk serves a clear instructional function. Ask, “What is THIS section supposed to do? It better:

a. Teach something new (facts, concepts, rules, theories, cognitive routines).

b. Summarize.

c. Build fluency.

10

d. Review and probes/tests (retention).

e. Expand---add more to existing facts, examples, concepts.

f. Generalize knowledge to new examples.

g. Strategically integrate---combine information into a larger whole, such as an explanatory essay, or a research project.

Strengths, Weaknesses, Improve How?

The program meets all of the above criteria. Each lesson reviews, builds fluency, adds more examples, and integrates pre-skills or part-skills into composite skills.

9. Well-designed materials (either lesson-based programs or textbooks) teach new knowledge (phase of acquisition) in a systematic and explicit (focused) way: [See also “Assessing and Improving Instruction” ]

a. Review and firm prior knowledge.

b. Regarding new knowledge, gain attention, frame new task, model, lead, test/check, verification; correct errors; more examples; delayed acquisition test.

[Procedures appropriate for each form of knowledge (fact, list, sensory concept, higher-order concept, rule, routine) are used.]Forms of Knowledge

Procedures for teaching the four forms of knowledge

11

c. Review and firm.

Strengths, Weaknesses, Improve How?

(1) The program meets all of the above criteria for review and firming, teaching new information, and review and firm what was covered. See lessons 1, 18, 52.

(2) Concepts (e.g., letter-sound correspondence: r says rrrr) are taught with an example (model) “This sounds is rrrr.” The new sound is then taught with additional examples within words. run, car, charm.

(3) Routines (e.g., sounding out words) are taught by teaching the steps (e.g., first say the sound on the left) and all of the knowledge needed to perform the steps (e.g., sound pronunciation, letter-sound correspondence); and then students are taught to perform the steps in sequence as teacher uses model—lead—test.

(4) Model-lead-test are used. But these are NOT followed by verification; e.g., “Yes, mmm.” Or, “Yes, you read those words the FAST way.

The teacher should add verification after the Test/check.

10. Well-designed curriculum materials adequately cover (teach, assess) all phases of mastery: acquisition (see #9), fluency, generalization, retention.

For each phase, there are stated objectives, instructional procedures, assessment of progress, and suggested remediation (if too little progress) based on assessment data

Phases of Mastery

Phases of Mastery Table

12

Strengths, Weaknesses, Improve How?

All phases of mastery are taught. For example, in lesson 54, new sounds re introduced (Task 1. acquisition), earlier sounds are reviewed (Task 2. retention); skill is generalized to new words (Tasks 3 and 4); fluency is increased (Tasks 4, 6, 9).

11. Well-designed curriculum materials provide scaffolding; i.e., various kinds of assistance to help teachers communicate information, and to help students acquire, organize, retrieve, and apply information/knowledge.

Examples are stated objectives, big ideas, advance organizers (lesson and unit outlines, guided notes, concept/proposition maps), summaries, glossaries, wait (or think) time.

Big ideas

Advance organizers

Strengths, Weaknesses, Improve How?

(1) Special orthography is used (e.g., blended letters [ch] connected, macron over long vowels, different shape of the oval in b and d, small e, o, and a to indicate silent) to enable students to read all words despite unusual spellings. E.g., students can read the word “read” WITHOUT being taught a verbal rule about silent a.

(2) Prompts are added to provide directions; e.g., arrows and ball under words and letters.

(3) These are faded out as skill is developed.

INSTRUCTION[Some of the following are also found above, in assessment

of curriculum materials.]

1. Students are prepared for new material being taught. They are firm on the pre-skill elements and/or background knowledge. Strengths, Weaknesses, Improve How?

Addressed above.

13

2. Instruction is designed on the basis of objectives, and focuses precisely on objectives.

Strengths, Weaknesses, Improve How?

Addressed above.

3. Instruction begins with review, especially elements and background knowledge relevant to the current instruction. The teacher corrects errors and firms knowledge or reteaches before introducing new material that requires this background knowledge.

Strengths, Weaknesses, Improve How?

Addressed above. See Lessons 1, 5, 10, 56, 90.

4. The teacher gains student readiness: attention, sitting properly, materials handy.

Strengths, Weaknesses, Improve How? These criteria are satisfied.

14

5. The teacher frames the instruction by stating the kind of new knowledge to be taught, the objectives, and big ideas that will help students organize, remember or access, and comprehend the new knowledge, and connect new with prior knowledge.

Strengths, Weaknesses, Improve How?

Framing is used throughout.

Big ideas (“We always sound out words”) are presented.

6. The teacher models or presents new information clearly and focuses on the objectives. The teacher: (a) Shares his or her thought processes. (b) Uses clear wording. (c) Repeats the information as needed. (d) Presents one step or item at a time in a verbal chain or a cognitive routine, depending on how many steps or items students can handle.

Strengths, Weaknesses, Improve How?

Modeling is the main method for transmitting information. See Task 1, Lesson 7; Task 1, Lesson 43.

7. The teacher leads students through the application of the new information. Strengths, Weaknesses, Improve How?

The lead portion of direct instruction is usually used. However, the teacher may have to use it more often with some students for whom a model alone is not sufficient.

8. The teacher gives an immediate acquisition test/check to determine whether students learned the new information. The teacher tests/checks every time new information is presented to be

15

sure that students learned it. This is especially important when teaching diverse learners, essential material, and difficult material. Strengths, Weaknesses, Improve How?

The immediate acquisition test usually follows model—lead. See Task 1, Lesson 7; Task 1, Lesson 21.

9. The teacher corrects all errors and/or firms weak knowledge. Matter of fact way and directed to the group. Model. Teacher immediately gives the answer or

demonstrates the step. Lead. Students say the answer or do the step with the

teacher. Test/check. Teacher asks the question or gives the problem

step again. Verification. Specific praise. Retest/starting over. Delayed test. Teacher comes back and checks again.

Strengths, Weaknesses, Improve How?

The program tells the teacher what kids of errors may occur and how to correct them---usually with the above procedure. See Task 1, Lesson 1; Task 1, Lesson 19; Task 1, Lesson 21.

10. If new material is a concept, rule-relationship, or cognitive routine, the teacher: Uses a wide and varied range of examples. Juxtaposes examples to reveal sameness. Juxtaposes examples and nonexamples to reveal difference. Strengths, Weaknesses, Improve How?

The program satisfies this criterion. See juxtapositions to reveal sameness and difference (e.g., at, that) in Task 4, Lesson 19; Task 5, Lesson 5 (seat, hear); Task 7, Lesson 50 (near, ears); Task 4, Lesson 57 (big, bug, pig, bit).

16

11. The teacher gives a delayed acquisition test (calling on both the group as a whole and then individual students) to determine whether students learned the concept, rule relationship, or cognitive routine from the examples and nonexamples, or whether students remember the set of facts presented.

Strengths, Weaknesses, Improve How?

Delayed acquisition tests are usually at the end of lessons; e.g., Task 5, Lesson 67).

12. The teacher reviews the instruction (e.g., main things taught) and states how what was taught is relevant to next lessons.The review: States what was learned, how it built on what came before, and

how it will be built on by next lessons. Has students once more reveal essential knowledge.

Strengths, Weaknesses, Improve How?

This is covered earlier.

13. The teacher uses information from the delayed acquisition test to determine whether students have sufficiently mastered the new material and can advance to the next step of instruction, or whether reteaching or more intensive instruction for some students is needed.

Strengths, Weaknesses, Improve How?

Rules are NOT specified for doing this. Rules should be added; for example, every five lessons.

17

14. The teacher teaches at a brisk pace by speaking more quickly; staying on task; using words whose meanings are clear; using the same instructional vocabulary from one task to another; cutting out unnecessary words.Strengths, Weaknesses, Improve How?

These criteria are met.

15. The teacher gives frequent opportunities for group (choral) and individual responses to test/check learning. The teacher asks the question first, and then calls on the

group or an individual. The teacher think time before calling on the group or an

individual. After presenting new information, the teacher calls on the

group as a whole. After calling on the group, the teacher calls on individual

students, and makes sure to call on students who have made errors or who in general have a harder time learning.

Strengths, Weaknesses, Improve How?

The first two criteria are satisfied.

16. The teacher uses pre-corrections, or reminders, to prevent errors. For example, “When we see an x between two numbers or parentheses, we multiply. What do we do when we see an x between two numbers or parentheses? Multiply. Yes, multiply.” Strengths, Weaknesses, Improve How?

Pre-corrections are sometimes used. For example, Task 2, Lesson 67 (Say it slowly if you can. Don’t get fooled.”); Task 2, step 2, Lesson 32.

18

17. The teacher uses a questioning technique such as Socratic dialogue as an instructional/communication procedure. Asking questions that probe students’ knowledge. Asking questions that require students to use rules of

reasoning. Helping students revise their knowledge. Strengths, Weaknesses, Improve How?

Not applicable.

18. When students are firm on new knowledge (acquisition phase), the teacher works on generalization of knowledge to new examples, fluency, and retention of knowledge.Strengths, Weaknesses, Improve How?

Covered earlier.

Features of a Productive Classroom Environment19. Increase time available for teaching and time engaged in teaching. Decrease noninstruction activities. Use activities for which students are prepared. Make certain subjects sacred. Use lesson-based materials. Use routines for distributing materials. Teach and practice getting ready for learning.

20. If Possible, teach in small, homogeneous Groups. Give pre-tests or placement tests to place students in groups

with other students at the same level or spot in a program. During beginning instruction, keep the group small—say six to

eight students.

19

Groups can consist of students from different classes and grade levels (at most two grade levels, as a rule).

Note students’ progress. Move students who are making quicker progress to groups with similar students.

21. The teacher uses different kinds of instructional groupings properly, including whole class instruction; small, homogeneous groups; small, heterogeneous groups; and paired peer groups.22. The teacher establishes a learning community with: A shared group mission. Shared group rules. Shared high expectations. Reinforcement for individual and group achievement. Students sitting near and facing the teacher. Providing frequent opportunities to respond (choral group, and

individual). Ensuring mastery of every task. Celebrating progress.

20