Post on 07-Jul-2020
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
1
Title: The Development of Multiple Measures of Curriculum Implementation in Secondary Mathematics Classrooms: Insights from a Three-Year Curriculum Evaluation Study Author Team: James E. Tarr, Melissa D. McNaught, and Douglas A. Grouws Complete Mailing Addresses: James E. Tarr Department of Learning, Teaching and Curriculum 303 Townsend Hall University of Missouri Columbia, MO 65211-2400 Phone: 573.882.4034 Fax: 573.882.4481 E-mail: tarrj@missouri.edu Melissa D. McNaught Department of Teaching and Learning Mathematics Education N287 Lindquist Center University of Iowa Iowa City, Iowa 52242 Phone: 319.335.5433 Fax: 319.335.5608 E-mail: melissa-mcnaught@uiowa.edu Douglas A. Grouws Department of Learning, Teaching and Curriculum 303 Townsend Hall University of Missouri Columbia, MO 65211-2400 Phone: 573.884.5982 Fax: 573.882.4481 E-mail: grouwsd@missouri.edu Abbreviated Title: Curriculum Implementation in Secondary Mathematics Classrooms
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
2
Introduction
In comparative curriculum research, achievement scores provide only a partial picture of
student knowledge acquisition. Factors such as students’ opportunity to learn and how teachers
present mathematical content also impact what students learn and how well they learn it.
Therefore, attention to curriculum implementation is a high priority in the Comparing Options in
Secondary Mathematics: Investigating Curriculum (COSMIC) project. In the remainder of this
chapter, the importance of examining curriculum implementation and the methods the project
uses to garner this information are described and discussed. We begin with an overview of the
context in which the instruments were developed, the COSMIC project, and then describe our
conceptual approach to instrument development, which includes the use of two data collection
perspectives and the gathering of data in several grain sizes. In the third section of the chapter,
we outline our instrument development process, describe the instruments developed, and provide
validity and reliability data. In the final two sections, we report on how the instruments have
been used, provide some illustrative findings, and suggest future directions.
The COSMIC Project
The goal of the COSMIC project is to examine student mathematical learning associated
with secondary mathematics curriculum programs of two types – a subject-specific approach,
where students follow a course sequence of Algebra I, Geometry, and Algebra II, and an
integrated content approach, where students follow a course sequence of Integrated I, Integrated
II, and Integrated III. The study is situated in 11 high schools in five states where each school
offers students a choice of either curricular approach, with more than 100 teachers and 5,800
students participating in the first two years of the project. Student learning over a three-year
period is tracked using standardized measures of achievement along with project-designed tests
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
3
to assess depth of knowledge, skills acquisition, ability to reason mathematically, and conceptual
development. Moreover, we pay careful attention to teachers’ implementation of curricular
materials in order to draw causal inferences between curriculum type and student learning.
The research questions related to the implementation component of the study are:
(1) How do teachers use textbooks with different approaches to content organization in
the ongoing process of mathematics teaching?
(2) What is the relationship between curriculum implementation and student learning?
To answer these questions, the project has developed and used multiple measures of curriculum
implementation, including teacher surveys, classroom observation tools, Textbook-Use Diaries,
and Table of Contents Records.
The Importance of Measuring Curriculum Implementation
One impetus for undertaking the COSMIC project was the call for more comparative
curriculum research put forth in the National Research Council report On Evaluating Curricular
Effectiveness: Judging the Quality of K-12 Mathematics Evaluations (NRC, 2004). The
recommendations in the report provided the guidelines for the development of our
implementation measures. The COSMIC project used the NRC report authors’ definition of
curriculum as the student textbooks and auxiliary materials that accompany them, including the
teacher guides. The representative textbook series of the integrated mathematics curriculum
approach in this study is Contemporary Mathematics in Context (Coxford et al., 2003), also
known as the Core-Plus Mathematics Project (hereafter Core-Plus). Several subject-specific
textbook series, all with similar content organizations, represented the subject-specific
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
4
curriculum approach (see COSMIC Technical Report 1.8 for additional details about the
textbooks included in the study1).
To develop an understanding of the learning that occurs through the use of a particular
curriculum, researchers must consider the way in which the curriculum materials are used in the
classroom, including how closely aligned classroom instruction is to the intent of the textbook
authors. The NRC (2004) refers to this alignment as “implementation fidelity” and describes it as
“a measure of the basic extent of use of the curricular materials” (p. 114). George, Hall, and
Uchiyama (2000) emphasize that assessing implementation is key to productively evaluating
curriculum programs. The NRC expanded on this admonition, stating that evaluations of
curricula require a measure of implementation in order to draw conclusions regarding their
influence on student achievement. More specifically, they stated,
Evaluations should present evidence that provides reliable and valid indicators of the
extent, quality, and type of the implementation of the materials. At a minimum, there
should be documentation of the extent of coverage of curricular materials (what some
investigators referred to as ‘opportunity to learn’) and the extent and type of professional
development provided. (p. 194)
Teachers are active decision makers with regard to how and when mathematical content
is taught and, as such, they are influenced by the instructional materials available to them as well
as events that occur within the mathematics classroom (Ben-Peretz, 1990; Clandinin & Connelly,
1992; Clarke, Clarke, & Sullivan, 1996; Remillard, 1999; Remillard and Bryans, 2004). For
these reasons, the ways in which teachers use the same curriculum vary. The need to document
teacher use of curriculum materials is supported by research suggesting that curriculum
1 Available by request from the first author.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
5
implementation is an uneven process within and across schools (Grouws, 1992; Grouws &
Smith, 2000; Jackson, 1992; Kilpatrick, 2003; Senk & Thompson, 2003). Kilpatrick argued,
Two classrooms in which the same curriculum is supposedly being ‘implemented’ may
look very different; the activities of teacher and students in each room may be quite
dissimilar, with different learning opportunities available, different mathematical ideas
under consideration, and different outcomes achieved. (p. 473)
The next section outlines our development of implementation tools and the steps we took
to ensure that we collected data to assess implementation from multiple perspectives using
different lenses and various grain sizes. The focus was on developing multiple measures with
known validity and high reliability.
Conceptualizing Implementation Fidelity: Two Lenses
There are varying perspectives on the issue of curriculum implementation. While some
argue that a truly faithful implementation is not possible (Remillard, 2005), there is substantial
agreement among researchers that it is an important variable to consider in analyzing data from
curriculum studies. In the COSMIC project, we conceptualized implementation fidelity along
two dimensions, content and presentation.
Through the content fidelity lens, we examine what mathematics content in the intended
curriculum (the textbooks in this case) was taught as part of classroom instruction. We
conceptualized this dimension as lying on a continuum from using the curriculum content
exactly as it is written in the textbook to the other extreme of regularly skipping content or
substituting content for what is in the textbook. In COSMIC, content fidelity is assessed at both
the course level and the lesson level. At the course level, we broadly examine what content from
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
6
the textbook was used on a daily basis across the school year. At the lesson level, we look in-
depth at the extent of textbook use during a specific lesson or in consecutive lessons.
Through the presentation fidelity lens, we examine how textbook lessons are presented to
students and assess the way students are expected to engage with the textbook material during
the mathematics class period. For example, textbooks that are purportedly based on research
regarding how students learn mathematics and embody recommendations of the National
Council of Teachers of Mathematics’ Standards documents (NCTM, 1989; 1991; 1995; 2000)
often call for students to work in small group settings, to engage in discussion of ideas, to
discover skills and procedures, and so on. In our project, we assess each of these facets of
implementation through classroom observations, and therefore presentation fidelity was
measured at the lesson level, but not at the course level.
Textbook Authors’ Conceptualizations of Implementation Fidelity
Prior to the development of tools to measure curriculum implementation, we engaged in
discussions about how validity of such instruments could be achieved. As a project team, we
certainly could have outlined what we believe constitutes a faithful implementation of a textbook
curriculum, but would those who designed and developed the textbooks share our conceptions of
implementation fidelity? We had some misgivings about our capacity to design valid measures
of teachers’ enactment of the written curriculum without consideration of the textbook authors’
perspective. Consequently, we decided to “go to the source” and directly interview textbook
authors about their conceptualizations of implementation fidelity with respect to both content
and presentation. We selected the Project Director of the integrated textbook series as well as an
author of one of the subject-specific textbook series, namely one who consistently served as a
member of the authorship team through numerous editions of its publication. Both authors were
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
7
granted consent from their respective publishing companies; as such, we considered them to be
“spokespersons” for their products. However, we acknowledge that it is plausible that different
members of each authorship team could have provided alternative conceptions of
implementation fidelity.
In preparation for our author interviews, we examined the various components of
curriculum programs, including student and teacher materials. Additionally, we identified the
structural components within each textbook, including the organizational structure such as units,
lessons, and elements that comprise individual lessons. Our analysis was specific to the two
types of textbooks, subject-specific and integrated, and yielded distinctive structures for each.
In the integrated curriculum studied, the textbooks are composed of Units broken down
into Lessons that contain multiple Investigations. Each Lesson is structured similarly utilizing
the following components: Launch, Explore, Share and Summarize, and Apply, with the
accompanying activities spanning multiple days. Exercises, known as MORE (Modeling,
Organizing, Reflecting, Extending) problems accompanying each Lesson, are typically assigned
over a period of days. These assignments offer students additional opportunities to apply, reflect
on, and extend their knowledge of the concepts developed during the Investigations within the
specific Lesson. Each of these Lesson components serves a distinctive purpose, as depicted in
Table 1.
! INSERT TABLE 1 – LESSON COMPONENTS "
The subject-specific curriculum typically includes an Algebra I, Geometry, and Algebra
II textbook. The books are organized by chapters consisting of 6 to 15 single day lessons, with
each lesson organized into three main components. Although the labels the textbook authors give
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
8
to each component vary, they can be logically categorized as (a) Lesson Preview; (b) Teach; and
(c) Practice and Apply. The purpose of each lesson component is presented in Table 1.
Based on the structural components identified, we devised a separate interview protocol for
an author of the Core-Plus materials and for an author of a popular Algebra I textbook. During
our interviews with the textbook authors, we asked questions that focused on the authors’ views
about a faithful implementation of the curriculum materials and gathered information about how
the authors envisioned the lesson components being enacted. For example, for both textbook
authors we asked: “If you were to visit a classroom in which a teacher was implementing your
curriculum materials ‘faithfully,’ what would you expect to observe?” and “What might you
observe that would lead you to conclude, ‘That is not what we intended when we wrote these
textbooks’?” We also inquired about each of the lesson components in Table 1. For example, we
asked the author of Core-Plus, “Are teachers expected to explicitly state the lesson objectives?
Why or why not?”, “How are teachers expected to deal with vocabulary? Should they explicitly
define words that may be new to the students?”, and “What is meant by ‘Teacher is director and
moderator’ (p. 12 of the Implementation Guide)?” Our overarching goal was to develop a
classroom visit protocol that contained specific observable teacher behaviors that textbook
authors would expect to see in a faithful implementation of their curriculum materials. We found
the textbook authors were forthright in offering their visions of curriculum enactment, and both
emphatically agreed that no curriculum is “teacher proof.” From both interviews, we recognized
the importance of classroom observations to assess alignment with textbook authors’
conceptualizations of implementation fidelity.
Instruments to Measure Implementation Fidelity
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
9
Evidence concerning implementation can be, and should be, measured from several
perspectives. In the COSMIC project, we developed instruments that separately reflected the
perspectives of the researcher and the teacher. In doing so, we were able to gather data that
allowed us to examine patterns in implementation within and across different perspectives.
Researcher Perspective
We used the Classroom Visit Protocol (CVP) to document the use of materials and
classroom activities from the researcher perspective. The CVPs we developed took account of
previous work in describing classroom instruction (e.g., Romberg & Shafer, 2003; Tarr, Reys,
Reys, Chávez, Shih, & Osterlind, 2008), but were more specific in nature than the tools used in
previous research. For example, we had separate CVPs for the integrated curriculum and the
subject-specific curricula, each designed to embody the components of their respective lesson
structures (discussed subsequently in this section).
Although there were separate CVPs for each curriculum type, there were common
elements in the CVPs, too. For example, in the first of three parts of the form, during each
classroom visit, the observer recorded anecdotal evidence of particular presentation features
being implemented and noted grouping of students, examples used from the textbook, questions
posed by the teacher, student responses, homework assignments, and use of assessments. At the
conclusion of the class period, the researcher used these data to complete the second and third
parts of the CVP, namely, a Lesson Summary form and a Classroom Learning Environment
scale.
! INSERT FIGURE 1 – EXCERPT OF OBSERVATION PROTOCOL "
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
10
Lesson Summary. Based on the information garnered from the interviews with the
authors and examination of the curriculum materials specific to each sample site2, we developed
the Lesson Summary Form to document specific classroom events and activities such as
interactions between the teacher and students, interactions among students, instructional
materials used by the teacher or students, and exercises or examples worked on or discussed. An
excerpt of the CVP for the integrated curriculum appears in Figure 1. Note that in the Launch
component of Core-Plus the textbook author expected to see four behaviors in a “faithful”
implementation of a Launch. We generated similar behavioral checklists from author interviews
for each of the lesson components denoted in Table 1. These measures require a dichotomous
judgment by the observer: “Did I see this action?” or “Did I not see this action?” The checklist
thus provides a low inference measure of fidelity to the structural components of the lesson
elements. Analyses of the checklists provide a means of judging whether appropriate attention is
given by the teachers to various lesson parts when examined within and across lessons.
As part of the researcher perspective we also developed high inference measures of
content fidelity and presentation fidelity. These measures focused, respectively, on the content
taught aspect of the curriculum and on the pedagogical aspect of the curriculum. We wanted a
numeric assessment and found that we could reliably measure these aspects of the enacted
curriculum if we coupled observer training with a carefully developed rubric for each scale.
Thus, we developed descriptions of “high,” “moderate,” and “low” Content Fidelity ratings and
Presentation Fidelity ratings (Table 2) that represented holistic judgments of the observed
behaviors. Rather than offer three distinctive descriptions (“high,” “moderate,” and “low), we
2 The curriculum for the subject-specific course was taught from one of several textbooks, among these the most widely used were the Glencoe Publishing Company textbook, Algebra I (Holliday, et al., 2005) and McDougal Littell, Algebra I (Larson, et al., 2004).
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
11
settled on a 5-point scale, such that ratings of 2 and 4 could serve as compromise positions
between adjacent anchors. We provided more detailed descriptions and examples for observers
in the coding training manual.
! INSERT TABLE 2 – RUBRIC FOR CONTENT & PRESENTATION FIDELITY "
Classroom Learning Environment. Our experience in previous curriculum evaluation
projects led to the decision to document selected elements of the classroom learning
environment. Our rationale for doing so was based on two factors. First, in an earlier study (Tarr,
Reys, Reys, Chávez, Shih, & Osterlind, 2008) classroom learning environment factors offered
predictive power in analyses of student achievement data. Second, we sought to document
particular features of the classroom learning environment that are considered important across
different types of curriculum (e.g., development of conceptual understanding). In the COSMIC
Classroom Learning Environment measure, there are 10 elements that collectively represent the
classroom environment. These 10 elements are classified into three themes: Reasoning about
Mathematics, Students’ Thinking in Instruction, and Focus on Sense-making. Using a well-
defined rubric for each of the 10 elements, observers rendered ratings from 1 to 5, with a 1
indicating the absence of the feature and a 5 indicating a strong presence of the feature during the
observed lesson. These ratings offered a common measure across curriculum types. However,
because the Classroom Learning Environment is curriculum independent, for the purpose of this
paper we have limited our discussion of it.
Teacher Perspective
To gain the teacher perspective, we had teachers report information by completing two
surveys, Textbook-Use Diaries, and a Table of Content Record. These instruments, described in
the sections that follow, served different, yet related, purposes.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
12
Teacher Surveys. Each teacher completed two written surveys adapted from the 2000
National Survey of Science and Mathematics Education (Weiss, Banilower, McMahon, & Smith,
2001). The first survey, titled Initial Teacher Survey, was completed at the beginning of each
school year and was used to gather teacher demographic and background data as well as teacher
beliefs about teaching and learning and teacher involvement in professional development. In
analyses of student achievement data, we statistically controlled for a variety of teacher-level
variables such as the number of years of teaching experience, years teaching the textbook
curriculum, and quantity and nature of professional development. Data from the Initial Teacher
Survey were collected because such teacher-level variables have the potential to explain
variation in student outcomes. The second survey, titled Mid-course Teacher Survey, was
completed at in the middle of the year gathering data about teacher use of curriculum materials.
More specifically, teachers were asked about the use of their particular textbook during
instruction, perception of the quality of the textbook, use of key instructional practices, use of
graphing calculators, assignment of homework, and assessment practices.
Table of Contents Record. The Table of Contents Record is a textbook-specific indicator
of content implementation that mirrors the table of contents of the particular textbook. For each
section of each Chapter, teachers are asked to indicate whether the content of each section
(Investigation) was (a) taught primarily from the textbook; (b) taught from the textbook with
some supplementation; (c) taught primarily from alternative(s) to the textbook; or (d) not taught.
A sample of part of a Table of Contents Record for the integrated textbook appears in Figure 2.
These records provide an indication of content coverage and the extent to which the teachers
utilized their textbook during instruction. More specifically, this instrument enables us to
determine what mathematics content students were afforded opportunities to learn, and whether
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
13
some mathematics content is disproportionately emphasized or skipped. Moreover, such
opportunity-to-learn data is particularly useful in analyses of student achievement data.
! INSERT FIGURE 2 – SAMPLE TABLE OF CONTENTS RECORD "
Textbook-Use Diary. The Textbook-Use Diary serves as a daily record of how teachers
engaged with curriculum materials during instruction. Questions pertaining to the subject-
specific curricula (Figure 3) ask for information about examples the teacher used during
instruction, textbook problems assigned to the students, and printed materials other than the
textbook (e.g., auxiliary curriculum resources, teacher-developed worksheets) used during
instruction. Questions pertaining to the Core-Plus curriculum ask for information regarding
which portions of the Think About This Situation, Investigations, and Checkpoint were used
during instruction and the assignment of On Your Own and Modeling-Organizing-Reflecting-
Extending (MORE) problems. Going beyond the Table of Contents Record, which reveals
information about what mathematics students are afforded opportunities to learn, the Textbook-
Use Diary provides information about the emphasis teachers placed on the content. Specifically,
comparisons can be made between the number of days a teacher spends on a particular section
and the number of instructional days recommended by authors in the scope and sequence section
of the textbook.
! INSERT FIGURE 3 – SAMPLE TEXTBOOK-USE DIARY "
Pilot Testing the Classroom Visit Protocol
Initially, a subject-specific curriculum protocol and an integrated curriculum protocol were
piloted in Algebra I and Integrated Course 1 classrooms, respectively. In early phases of piloting,
our primary objective was to determine whether our extensive observation protocol was
manageable. More specifically, we sought to determine whether it was possible for an observer
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
14
to attend to all of the features of curriculum implementation without the use of video or
audiotape recordings of instruction. To gauge classroom observers’ facility with completing the
CVP, over the course of one month three COSMIC researchers simultaneously viewed the same
four videotaped lessons, with each observer taking field notes independently. Immediately
following each lesson, we completed the observation protocol, subsequently comparing codes
and providing commentary about the strengths and limitations of the CVP.
After extensive deliberation, we concluded that the observation tool, in its present state,
required too many sophisticated judgments for observers to make. Accordingly, we made
substantial adjustments and refinements to the CVPs and User’s Guides. We reduced the number
of coding options, thereby limiting the number of features of curriculum implementation to
which the observers would have to attend. We also wrote supplemental narrative for the User’s
Guides to draw clearer distinctions between many of the available coding options.
After completion of the pilot testing, we consulted with several mathematics educators and
educational researchers who were not otherwise involved with the COSMIC project. Our
reviewers included two former presidents of the National Council of Teachers of Mathematics
and two researchers with expertise in teaching and teacher education. They were asked to
provide feedback on the revised protocols and User’s Guides. Specifically, we asked:
• Is the protocol a manageable tool for a trained classroom observer?
• Is there an adequate level of specificity in the User’s Guide so that the observer will
understand what notations he/she should be marking on the form?
• Will the data we collect from the forms be useful in making judgments about extent
of implementation?
• Are we missing some important concepts that should be measured?
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
15
• Are there some areas measured that in your opinion are not particularly relevant or
helpful for our purposes?
Although the reviewers’ comments were exceedingly positive, there were key suggestions that
necessitated revisions to the protocol. For example, one reviewer suggested that we document
the seating configuration because Core-Plus is designed for students to work in cooperative
learning groups. Therefore, if students were seated in rows, this feature would compromise the
notion of a “faithful” implementation of Core-Plus. Another reviewer suggested that we record
whether interactive software was used by the teacher and by students during mathematics
instruction. Finally, a reviewer suggested that we document the extent to which students were
engaged (i.e., “on task”) because it was conceivable that a teacher could demonstrate all aspects
of faithful implementation and yet have many students off-task during the lesson. These
modifications were made to the CVP and the User’s Guide was updated accordingly.
Training on Use of the Classroom Visit Protocol
Preliminary training on use of the CVP
Training for the project team on the preliminary version of each of the two CVPs (one for
Core-Plus and one for subject-specific textbooks) was conducted in a two-day session at the start
of our project. Prior to the training, we distributed copies of each CVP and accompanying User’s
Guide to all members of the COSMIC research team. Also prior to training, developers of the
CVP coded the lesson, shared codes, and negotiated consensus codes that essentially represented
the “key,” that is, a target set of “correct” codes for those in training. At the training, one of the
developers of the CVPs used a PowerPoint presentation to provide an overview of the Core-Plus
curriculum, identifying the overall structure of the curriculum and components common to all
Investigations that comprise each Lesson. A second developer then provided an overview of the
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
16
essential elements of the integrated CVP. Following this discussion, the project team viewed a
videotape of one Core-Plus lesson. Subsequently, this procedure was used to train observers to
use the subject-specific CVP.
After the observers viewed a lesson in a training session, one of the facilitators led
discussions of how codes should be assigned. This step-by-step process involved toggling back
and forth between the CVP and the User’s Guide, identifying the most appropriate code and
justifying its assignment by explicit reference to narrative in the User’s Guide. Moreover,
facilitators provided explanations to help those in training understand why other available codes
were not appropriate. This laborious induction to the use of the CVP spanned most of the first
day of training, but was a necessary step in learning the coding scheme.
After bringing closure to the first lesson, the project team viewed a second videotape and
coded the lesson individually before the coding “key” was shared. Relatively low consistency
was observed across researchers and in relation to the “key,” and, consequently, the development
team concluded that it was necessary to scale back the specificity of the CVP. In short, for each
aspect of a given lesson, the preliminary version offered too many options to code in a reliable
manner. This notion was confirmed on Day 2 of training, when researchers experienced similar
struggles as they attempted to assign codes for two lessons using subject-specific curriculum
materials. As a result, we streamlined the CVPs to focus on only the most critical elements of
curriculum implementation, and subsequently planned a second phase of training.
Final training on use of the CVP
Training on the final version of the CVPs involved viewing videotapes of four
mathematics lessons in order to gauge the consistency of codes rendered by COSMIC project
personnel. Specifically, two subject-specific lessons (in Algebra 1) were shown to the project
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
17
team in one training session. One week later, two Core-Plus lessons were viewed, coded, and
discussed. We acquired the videos of the Core-Plus lessons from the curriculum developers; one
video represented the case of a high-fidelity teacher and the second video represented the case of
a lower-fidelity teacher.
In an effort to simulate an actual classroom visit, each video ran uninterrupted so that the
project team members could experience the recording of field notes in real time. After each
lesson, individual members of the project team completed the Lesson Summary Form, consulting
the User’s Guide as needed. Next, each researcher assigned one code to each of the 10 elements
that comprise the Classroom Learning Environment. When all coding was completed, discussion
commenced and focused on the codes of each researcher relative to one another. Feedback from
the research team led to modest changes in the User’s Guides in order to further enhance the
reliability of coding and attain a high level of coding consistency, as described in the following
section.
Establishing Coding Reliability
The reliability study of the Classroom Visit Protocol (CVP) coding was a two-stage
process. In addition to collecting reliability data during training to determine whether observed
lessons could be coded consistently, we gauged ongoing reliability through the double coding of
selected lessons during classroom visits during the data collection phase of the study.
Exploratory Reliability Study
As noted earlier, initial reliability was gauged in the final training on use of the CVP.
With respect to the Lesson Summary section, relatively high inter-rater reliability was attained.
In particular, researchers assigned identical codes to 93% of items comprising the Lesson
Summary Form for the subject-specific CVP. Similarly high inter-rater reliability was evident on
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
18
the Lesson Summary Form for the CVP for Core-Plus, with agreement reached on 90% of all
codes including dichotomous codes (e.g. observed/not observed) and the five-point fidelity
ratings scale. Across the two types of CVPs, agreement was reached in the assignment of 92% of
codes, lending credence to the notion that this portion of classroom visit data could be coded
reliably.
There was somewhat less agreement on the codes assigned by researchers in relation to
the Classroom Learning Environment. In particular, researchers assigned the same code at a 72%
rate on the two subject-specific CVPs, 69% on the two Core-Plus CVPs, and 70% overall. When
inconsistencies were observed, rubrics in the User’s Guides were read aloud and discussed in
order to negotiate the optimal code for the given classroom element. Researchers’ initial codes
differed by no more than ± 1 from the negotiated code (on a five-point scale) in 94% of all cases
(98% for subject-specific, 90% for integrated). Given the relatively high consistency in coding
across observers in other parts of the instrument and the high inference nature of this measure,
the decision was made to proceed with data collection in the classrooms of teacher participants in
the COSMIC project.
Confirmatory Reliability Study
During data collection, a confirmatory reliability test was conducted by double coding 15
lessons, chosen based on feasible observation schedules. Individual researchers took their own
field notes; immediately following the lesson each researcher worked in isolation to complete the
protocol including the content and presentation fidelity judgments. When researchers completed
all coding, they compared codes, negotiating disagreements until all were resolved. Consensus
codes were used in subsequent analysis of implementation data, but the set of original assigned
codes were used to gauge the ongoing reliability of the classroom visit coding.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
19
! INSERT TABLE 3 – CODING RELIABILITY "
As depicted in Table 3, remarkably high consistency was evident in the codes researchers
assigned to the Lesson Summary, with agreement observed for more than 94% of codes. The
reliability of coding Content Fidelity and Presentation Fidelity was similarly high. With regard to
Content Fidelity, 14 of 15 (or 93%) rating pairs from the two observers were identical. In the one
instance when the two observers disagreed, their individual ratings were within one unit of each
other. The results for Presentation Fidelity revealed 10 of 15 (or 67%) rating pairs were identical,
with the remaining 5 pairs all within one unit of each other. Higher reliability of codes was
observed for the Lesson Summary than for the Classroom Learning Environment. However,
although there was exact agreement in only about two-thirds of codes in the Classroom Learning
Environment, more than 92% codes differed by no more than 1 (on a 5-point scale), providing
evidence of relatively high inter-rater reliability for this component of the Classroom Visit
Protocol.
Analyses of Implementation Data
Although we continue to analyze implementation data, our completed analyses have
yielded several interesting findings. We organize this section around the data sources and offer
insights into the processes of making sense of such voluminous data.
Classroom Visit Protocols
Each of the teachers in the study was observed at least three times during the school year,
for a total of 325 observations completed during the first two years of data collection. The
relatively few teachers who participated in both years were observed three times each year.
During classroom observations, observers recorded judgments regarding the degree to which the
textbook influenced the content taught and the manner of presentation of the mathematics
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
20
lessons. The individual ratings for the overall Content Fidelity and Presentation Fidelity ratings
were aggregated across visits for each teacher to provide a mean Content Fidelity and
Presentation Fidelity rating for each teacher.
With respect to Content Fidelity, our analysis revealed that the content of lessons was
primarily attributable to the textbook. Across 109 teachers, the mean Content Fidelity rating was
3.67 (on a 5 point scale). Moreover, our data indicate that Content Fidelity ratings were similarly
high regardless of whether the teacher was teaching from an integrated or a subject-specific
textbook and the means were not significantly different across textbook types. The mean overall
Presentation Fidelity rating was 3.11 across the 109 teachers, which was significantly lower and
statistically different than the Content Fidelity ratings. This difference suggests that the manner
in which the lesson was taught was less consistent with the authors’ expectations than was the
content of lessons taught. Furthermore, the average Presentation Fidelity rating for teachers of
the integrated curriculum was 2.91, significantly lower than the mean rating of 3.28 for teachers
of the subject-specific curriculum.
The correlation between the Content Fidelity and Presentation Fidelity ratings was 0.50, a
moderate relationship between the two dimensions. The magnitude of the correlation between
Content and Presentation Fidelity suggests that these two dimensions of implementation should
be examined separately. The substantial variation we found on both of these dimensions across
teachers (1.02 SD for Content Fidelity and 0.96 SD for Presentation Fidelity) underscores why
attention to multi-dimensions of fidelity is warranted.
Background Characteristics
As noted earlier, all teachers who participated in the COSMIC study completed an Initial
Teacher Survey in which they reported numerous background characteristics, including: number
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
21
of years of experience teaching mathematics; beliefs about teaching and learning mathematics;
familiarity and agreement with NCTM Standards; implementation of NCTM Standards; the
amount of time allotted for and focus of professional development; the impact professional
development has on teachers’ instructional practices; and the use of technological tools to
supplement mathematical instruction. We collected such data, in part, in order to examine
associations between implementation fidelity and teacher characteristics. However, given the
focus of this chapter on the development of measures of curriculum implementation, we forego
the reporting of such findings in this chapter.
Table of Contents Records
A total of 182 Table of Contents Records were collected. Three indices were developed
to capture the nature and extent of textbook use: (1) Opportunity to Learn index; (2) Extent of
Textbook Implementation index; and (3) Textbook Content Taught index.
Opportunity to Learn (OTL) index. The OTL index indicates whether the mathematical
content contained within the textbook lessons3 was or was not taught. The OTL index is
computed by summing the frequency of occurrence of content taught (reported across all
textbook lessons on a Table of Contents Record) and then dividing by the total number of lessons
included in the particular textbook. The OTL index essentially represents the percentage of the
content in the textbook that students were provided an opportunity to learn.
As an example, Teacher 26, who taught from the integrated curriculum, reported 29
Investigations taught primarily from the textbook, 11 Investigations taught from the textbook
with some supplementation, 9 taught primarily from an alternative source, and 28 not taught out
of a total of 77 Investigations. Thus, the OTL index is calculated as follows:
3 Although we use “lessons,” in the case of the integrated curriculum the unit of analysis is an Investigation; several Investigations comprise one “Lesson”.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
22
!
OTL =29 +11+ 9
77•100 = 63.64
! INSERT FIGURE 4 – SAMPLE OTL GRAPH "
It should be noted that when interpreting OTL indices, an index of 63 does not imply that the
coverage corresponds to the first 63% of the textbook, as can be seen in Figure 4. For this teacher
implementing the integrated textbook, we see that many of the Investigations that were not
taught occurred midway through the textbook.
Extent of Textbook Implementation Index (ETI). The ETI index is determined by
weighting each of the first three options provided to the teachers on the Table of Contents
Record. The largest weight was given when the first option was identified for a lesson. That is,
when lesson content was taught primarily from the textbook, it was given a weight of 1. Content
not taught was given a weight of 0. The two options in between, content taught with
supplementation and content taught primarily from an alternative source were assigned weights
of
!
23 and
!
13 , respectively. The index was then calculated by summing the weights across textbook
lessons and then dividing by the number of lessons contained in the particular textbook. The
quotient was then multiplied by 100 giving the ETI index a scale ranging from 0 to 100. An
index of 100 would represent that every lesson contained in the textbook was taught directly
from the textbook and done so without supplementation or use of alternate sources. An index of
0 would indicate that no lessons from the textbook were taught. Using the previous example
(Teacher 26), the ETI index is calculated as follows:
ETI =1 29( ) + 2
311( ) + 1
39( ) + 0 28( )
77!100 = 51.08
! INSERT FIGURE 5 – SAMPLE ETI GRAPH "
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
23
This index indicates the degree to which the content contained in the lessons was taught directly
from the textbook. Note the heavy shading in Figure 5 takes on a different connotation than in
Figure 4. In particular, the heavy shading indicates only those Investigations that were directly
taught from the integrated textbook. The dark gray shading indicates Investigations taught with
supplementation, light gray shading shows Investigations taught from another source, and
unshaded cells indicate Investigations not taught. The first bar in Figure 5 depicts the manner in
which Teacher 26 taught each individual textbook Investigation. When these Investigations are
grouped together in the second bar by method taught, the figure reveals the proportion of the
content of the Investigations that was taught directly from the text, taught with some
supplementation, taught primarily with alternatives to the textbook, or not taught.
Textbook Content Taught Index (TCT). The TCT index differs from the ETI index by
considering only those lessons where content was taught in some manner, thereby ignoring
content students were not given the opportunity to learn (see Figure 6). The lessons were
weighted in the same manner as in the ETI, but the index was calculated by dividing by the
number of lessons reported as being taught in any manner and again multiplied by 100. The
index is reported on a scale ranging from 0 to 100. An index of 100 would indicate that all
lessons were taught using only the textbook. Thus, indices less than 100 indicate the extent to
which textbook lessons taught were supplemented or replaced. Ultimately, this index reports the
extent to which teachers, when teaching textbook content, followed their textbook, supplemented
their textbook lessons, or used altogether alternative curricular materials. The computation of the
TCT index for Teacher 26 is shown below.
TCT =1 29( ) + 2
311( ) + 1
39( )
29 +11+ 9!100 = 80.27
! INSERT FIGURE 6 – SAMPLE TCT GRAPH "
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
24
Teacher 26 represents the case of a teacher with markedly different ETI and TCT indices,
51.08 and 80.27, respectively. However, the decisions of other teachers regarding textbook use
resulted in remarkably similar ETI and TCT indices. For example, Teacher 44, who taught from
the subject-specific curriculum, “covered” 76 of 80 lessons in her textbook. As depicted in
Figure 7, she taught 38 lessons directly from her textbook, 32 lessons from the textbook with
some supplementation, 6 lessons primarily from alternative sources, and 4 lessons were not
taught, resulting in an ETI of 76.67. Because the textbook was the primary source for precisely
one-half (38 of 76) of the textbook lessons she taught, black comprises exactly one-half of the
bar in the TCT graph in Figure 8. Because so few lessons were omitted by Teacher 44, the TCT
index of 80.70 is quite similar to the 76.67 score on the ETI. In fact, for any teacher who omits
no textbook lessons, the ETI and TCT indices will be equal.
! INSERT FIGURE 7 – SAMPLE ETI GRAPH "
! INSERT FIGURE 8 – SAMPLE TCT GRAPH "
Textbook-Use Diaries
A total of 219 Textbook-Use Diaries have been collected. These diaries can be analyzed
to describe the extent and nature of the use of the curriculum materials over 15 consecutive days
of instruction. Researchers can perform a number of analyses, including examining how many
instructional days were necessary to “cover” the specified lesson, determining what homework
problems were assigned, and assessing the degree of supplementation used during the lesson.
Instructional days. When coding diaries with respect to how many instructional days
were used to cover a specified chapter or unit, we accounted for variation in class period formats
across schools. In this study, a day is defined as a class period in a typical 7-period day with
these periods ranging in length from 47 to 60 minutes. For those teachers on a block schedule,
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
25
the periods ranged from 85 to 90 minutes. Conventionally, a block period is considered as two
periods in a 7-period school day. Thus, the mean number of days allocated to a lesson will be
reported in terms of a 7-period day and data for those teachers who reported their days on a block
schedule are doubled. The distributions of days allocated to the lesson can be determined along
with the means and standard deviations. Despite the fact that textbook authors recommend
teachers spend a certain number of instructional days on the particular lessons upon which the
Textbook-Use Diaries were based, an initial examination of the diaries revealed that teachers
often do not abide by this recommendation (McNaught, 2009). For example, for a particular
integrated lesson on which the authors recommended 12 days of instruction, the number of
instructional days devoted to the lesson varied from 2 to 30 days. By requesting data on a
common set of lessons across the sites, this analysis sheds light on the variability among teachers
within and across schools in regard to the pace at which the curriculum unfolds for particular
mathematical content.
Homework Assignments. With respect to the assignment of homework, both types of
curricula include problems that are intended to be out-of-class exercises to help students
reinforce and extend the knowledge they acquired during the classroom portion of the lesson.
Although the authors offer recommendations regarding the types of problems and the length of
the assignments, one analysis to date indicates that teachers do not closely abide by these
recommendations. For example, few teachers using the integrated curricula assigned the number
of problems recommended by the textbook authors, typically assigning fewer problems than
recommended (McNaught, 2009).
Degree of supplementation. Textbook-Use Diaries can be used to study the regularity
with which the textbook is used by the teacher to teach particular mathematical content and the
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
26
extent to which the textbook is supplemented. The number of days a teacher used the textbook is
determined by counting the days the teacher reported using any problems from the textbook,
including days when a teacher reported using both the textbook and some supplements. Each
entry that did not indicate any sort of textbook use (e.g., using only a teacher-made worksheet)
would be coded as no textbook use. This information can then be used to calculate a frequency of
use index by dividing the total days the textbook was used by the total days of instruction. The
degree of supplementation can then be reported by providing percentages with regard to three
categories: (1) use of textbook only; (2) use of supplements only; and (3) use of a mix of
textbooks and supplements.
Conclusion
This chapter has focused on describing the development and nature of instruments that
can be reliably used to assess curriculum implementation. The instruments developed are
valuable tools in characterizing the enacted curriculum, but no single tool or collection of tools
can portray all aspects of how a teacher implements a curriculum program in the classroom.
Thus, researchers need to develop other tools that target specific aspects of implementation that
our tools do not. Furthermore, our instruments would benefit from refinements that will help us
move beyond measures of implementation to develop an understanding of how teacher decisions
about curriculum implementation are made and to identify factors that dictate or at least
influence this decision-making. Although many paths can be taken in this follow-up research,
our current efforts are directed at developing surveys and interview protocols that can be used to
ascertain how teachers make decisions about which textbook lessons to skip and what textbook
lessons require the use of supplemental material or replacement units. Moreover, we expect to
explore the complex interaction between implementation fidelity, teacher characteristics, and
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
27
student learning in mathematics. Such findings have the potential to inform teacher education,
curriculum development, and professional development as well as spawn numerous follow-up
studies in mathematics education.
In conclusion, given the importance of measuring curriculum implementation, we hope
other researchers will move forward with the development of additional innovative tools to
measure fidelity of implementation as well as find creative ways to use the instruments we have
developed.
Notes
This paper is based on research conducted as part of the Comparing Options in Secondary
Mathematics: Investigating Curriculum (COSMIC) project, a research study supported by the
National Science Foundation under grant number REC-0532214. Any opinions, findings, and
conclusions or recommendations expressed in this paper are those of the authors and do not
necessarily reflect the views of the National Science Foundation.
Portions of this paper were presented at the annual meeting of the American Educational
Research Association in Denver, Colorado in May 2010.
References
Ben-Peretz, M. (1990). The teacher-curriculum encounter: Freeing teachers from the tyranny of
texts. Albany, NY: State University of New York Press.
Clanindin, D.J., & Connelly, F.M. (1992). Teacher as curriculum maker. In P. Jackson (Ed.), The
handbook of research on curriculum (pp. 363-396). New York: Macmillan.
Clarke, B., Clarke, D., & Sullivan, P. (1996). The mathematics teacher and curriculum
development. In A. J. Bishop, K. Clements, C. Keitel, J. Kilpatrick, & C. Laborde (Eds.),
International handbook of mathematics education (pp. 1207–1233). Dordrecht, The
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
28
Netherlands: Kluwer.
Coxford, A. F., Fey, J., Hirsch, C. R., Schoen, H., Burrill, G., Hart, E., et al. (2003).
Contemporary mathematics in context. New York, NY: Glencoe/McGraw-Hill.
George, A. A., Hall, G. E., & Uchiyama, K. (2000). Extent of implementation of a Standards-
based approach to teaching mathematics and student outcomes. Journal of Classroom
Interaction, 35(1), 8-25.
Grouws, D. A. (Ed.). (1992). Handbook of research in mathematics teaching and learning. New
York: Macmillan.
Grouws, D. A., & Smith, M. S. (2000). NAEP findings on the preparation and practices of
mathematics teachers. In Results from the seventh mathematics assessment of the
National Assessment of Educational Progress (pp. 107–139). Reston, VA.
Holliday, B., Cuevas, G., Moore-Harris, B., Carter, J. A., Marks, D., Casey, R. M., Day, R., &
Hayek, L. M. (2005). Algebra 1. New York, NY: Glencoe/McGraw-Hill.
Jackson, P. W. (Ed.). (1992). Handbook of research on curriculum: A project of the American
Educational Research Association. New York: Macmillan.
Larson, R., Boswell, L., Kanold, T. D., & Stiff, L. (2004). Algebra 1. Evanston, IL: McDougal
Littell.
Kilpatrick, J. (2003). What works? In S. L. Senk & D. R. Thompson (Eds.), Standards-based
school mathematics curricula. What are they? What do students learn? (pp. 471-493).
Mahwah, NJ: Lawrence Erlbaum.
McNaught, M.D. (2009). Implementation of integrated mathematics textbooks in secondary
school classrooms. Unpublished doctoral dissertation, University of Missouri, Columbia.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
29
National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for
school mathematics. Reston, VA: Author.
National Council of Teachers of Mathematics. (1991). Professional standards for teaching
mathematics. Reston, VA: Author.
National Council of Teachers of Mathematics. (1995). Assessment standards for school
mathematics. Reston, VA: Author.
National Council of Teachers of Mathematics. (2000). Principles and standards of mathematics.
Reston, VA: Author.
National Research Council. (2004). On evaluating curricular effectiveness: Judging the quality
of K-12 mathematics evaluations. Washington, DC: The National Academies Press.
Remillard, J. T. (1999). Curriculum materials in mathematics education reform: A framework for
examining teachers; curriculum development. Curriculum Inquiry, 29(3), 315-342.
Remillard, J. T., & Bryans, M. B. (2004). Teachers’ orientations toward mathematics curriculum
materials: Implications for teacher learning. Journal for Research in Mathematics
Education, 35, 352-388.
Remillard, J. (2005). Examining key concepts in research on teachers' use of mathematics
curricula. Review of Educational Research, 75(2), 211-246.
Romberg, T. A., & Shafer, M. (2003). Mathematics in Context (MiC): Preliminary evidence
about student outcomes. In S. Senk, D. Thompson (Eds), Standards-based school
mathematics curricula: What are they? What do students learn? (pp. 224-250). Mahwah,
NJ: Lawrence Erlbaum Associates, Inc.
Senk, S. L. & Thompson, D. R. (2003). Standards-based school mathematics curricula: What
are they? What do students learn? Mahwah, New Jersey: Lawrence Erlbaum Associates.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
30
Tarr, J.E., Reys, R.E., Reys, B.J., Chávez, Ó., Shih, J., & Osterlind, S. (2008). The impact of
middle-grades mathematics curricula and the classroom learning environment on student
achievement. Journal for Research in Mathematics Education , 39(3), 247-280.
Weiss, I. R., Banilower, E. R., McMahon, K. C., & Smith, P. S. (2001). Report of the 2000
National Survey of Science and Mathematics Education. Chapel Hill, NC: Horizon
Research, Inc.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
31
Table 1 Lesson elements of integrated and subject-specific textbooks
Subject-Specific Lesson Element Purpose
Integrated Lesson Element Purpose
Lesson Preview To introduce the instructional objective for the class period, typically through a real-world context to illuminate the relevancy of the mathematics in the section.
Launch To generate student interest and provide a context for the Lesson, and to enable teachers to informally assess students’ prior knowledge.
Explore To investigate the mathematics of the Lesson in small groups, to gather data, look for patterns, construct models and meanings, and make and verify conjectures.
Teach To present new mathematical content, typically through worked examples closely tied to the lesson objectives.
Share and Summarize
To use whole class discussion of student ideas to review the content of the Investigation, amplify the mathematical ideas and reinforce connections made during the Investigation.
Practice and Apply
To work independently on exercises to achieve proficiency, typically by mirroring the worked examples from the Teach component of the lesson.
Apply To reinforce student learning through individual practice: On Your Own, a series of questions to assess student understanding of the content of the Investigation; and MORE problems, exercises to be worked out-of-class.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
32
Table 2 Content and Presentation Fidelity Scales
1 2 3 4 5 Lower Content Fidelity Moderate Content Fidelity Higher Content Fidelity
The content of the enacted curriculum was largely inconsistent with the written curriculum. The textbook was not the primary source of the lesson content because of omissions, significant modifications, and/or supplementation.
The content of the enacted curriculum was moderately consistent with the written curriculum. Although the textbook was a source of some of the lesson content, other portions of the lesson could not be attributed to the textbook.
The content of the enacted curriculum was consistent with the written curriculum. The textbook was the primary source of the lesson content with little or no deviation or supplementation.
Lower Presentation Fidelity Moderate Presentation Fidelity Higher Presentation Fidelity The presentation of the enacted curriculum was not consistent with the expectations of the textbook authors. During the lesson, the teacher implemented actions/activities that were not recommended and/or neglected to implement actions/activities that were advised or recommended. The teacher placed disproportionate emphasis on particular lesson components at the expense of others.
The presentation of the enacted curriculum was moderately consistent with the expectations of the textbook authors. During the lesson, the teacher either implemented some actions/activities that were not recommended or neglected to implement actions/activities that were advised or recommended. The teacher generally placed appropriate emphasis on each lesson component.
The presentation of the enacted curriculum was consistent with the expectations of textbook authors. During the lesson, the teacher implemented recommended actions/activities and refrained from actions/activities that were not advised or recommended. The teacher placed appropriate emphasis on each lesson components.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
33
Table 3 Percentage of coding agreements for each protocol element, by curriculum type and overall, in data collection phase
Protocol Element Subject-Specific Integrated Overall
Coding Matches
Total Codes Reliability Coding
Matches Total Codes Reliability Coding
Matches Total Codes Reliability
Lesson Summary 277 300 .923 566 594 .953 843 894 .943
Content Fidelity 14 15 .933
Content Fidelity (±1) 15 15 1.000
Presentation Fidelity 10 15 .667
Presentation Fidelity (±1) 15 15 1.000
Classroom Learning Environment 40 60 .667 66 110 .600 106 170 .624
Classroom Learning Environment (±1) 56 60 .933 101 110 .918 157 170 .924
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
34
Figure 1. Excerpt from integrated Classroom Visit Protocol.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
35
Unit 1 Matrix Models
Taught primarily from
Core-Plus textbook
Taught from Core-Plus textbook with
some supplementation
Taught primarily from alternative(s) to Core-Plus
Did not teach
content
Lesson 1 Building and Using Matrix Models
Inv1
There's No Business Like Shoe Business ! ! ! !
Inv 2 Analyzing Matrices ! ! ! !
Inv 3 Combining Matrices ! ! ! !
Figure 2. Excerpt from Table of Contents Record for integrated textbook.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
36
Glencoe Geometry Textbook-Use Diary
Instructions: For the first regular Geometry class you teach, record the instructional activities that correspond to Chapter 3. Log activities for the first 15 days only. A sample entry is provided below.
Date Section
Number of examples used from textbook
Number of examples used from
other sources
Homework Assignment
(include page numbers)
Oct 7 3.1 4 1 Pages 134-137
#3-36 even; 44-47
Figure 3. Excerpt from Textbook-Use Diary for subject-specific textbook.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
37
Figure 4. Opportunity to Learn (OTL) index for a teacher implementing an integrated textbook. Note: Black shading indicates one of the first three options on the TOC reported: (1) content taught primarily from textbook; (2) content taught from the textbook with some supplementation; (3) content taught primarily from an alternative source. No shading indicates option (4) content not taught.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
38
Figure 5. Extent of Textbook Implementation (ETI) index for a teacher implementing an integrated textbook.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
39
Figure 6. Textbook Content Taught (TCT) index for a teacher implementing an integrated textbook.
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
40
Figure 7. Extent of Textbook Implementation (ETI) index for a teacher implementing a subject-specific textbook
DRAFT
Curriculum Implementation in Secondary Mathematics Classrooms
41
Figure 8. Textbook Content Taught (TCT) index for a teacher implementing a subject-specific textbook