Setting the Pace:the Pace:the Pace:the Pace:the Pace - UChicago
Transcript of Setting the Pace:the Pace:the Pace:the Pace:the Pace - UChicago
1Setting the Pace
Julia B. Smith
BetsAnn Smith
Anthony S. Bryk
CImproving hicago's Schools
November 1998
SettingSettingSettingSettingSettingthe Pace:the Pace:the Pace:the Pace:the Pace:Opportunities toOpportunities toOpportunities toOpportunities toOpportunities to
Learn in Chicago'sLearn in Chicago'sLearn in Chicago'sLearn in Chicago'sLearn in Chicago's
ElementaryElementaryElementaryElementaryElementary
SchoolsSchoolsSchoolsSchoolsSchools
Consortium onChicago SchoolResearch
1313 East 60th StreetChicago, Illinois 60637773-702-3364
2 Improving Chicago’s Schools
Six Reasons Why Instruction Slows Down
examined and reconsidered. We find no connectionbetween slow pacing and improved student learning,a finding echoed in the recent Third InternationalMath and Science Study (TIMSS).1 All we can saywith the evidence at hand is that steady exposure toslow pacing leaves Chicago’s students farther andfarther behind.
Some repetitions more understandably reflectthe fact that certain topics and skills have no obvi-ous or fixed place in the curriculum. An examplefrom our observations was teaching the eight partsof speech. It is common to introduce middle gradestudents to the eight parts of speech, but many highschool language arts textbooks cover them also. Allof us forget what the eight parts of speech are (youcan test yourself ). But there is an important differ-ence between being reminded of them and spendingwhole lessons writing up their definitions. Forget-ting that pronouns are one of the eight parts of speechmay not mean that one doesn’t know what a pro-noun is or how to use it. But many teachers we ob-served and interviewed either feared this to be thecase or believed it wiser to hedge their bets throughthorough reteaching.2
2. Grades and test preparation. Ironically, andpainfully for thousands of teachers, the demands forimproved grade and test outcomes seem to slowinstruction. As the end of each quarter approaches,teachers are increasingly tempted to assign familiar,easy to complete tasks to help their students earnpoints and avoid a failing grade. Time spent in thesefamiliar tasks limits time spent exploring new materialand developing new skills.
Our study of schools and classrooms taught us that manyfactors contribute to slow pacing. Some factors areorganizational, such as very weak grade to gradeinformation about what is actually being taught andlearned. Indeed, we were regularly queried about suchthings by teachers during our school visits. “What arethey teaching down there in those other grades?”
Other factors are human. Practices and habits thatcontribute to slow pacing are rational and well inten-tioned, but they accumulate into patterns that steadilyerode learning opportunities. Below we outline six ofthe most pervasive factors suggested by observationsand interviews with teachers. We offer these factorsto foster understanding and to encourage reflectivediscussion and action.
1. Review and repetition. Starter segments thatreview prerequisite knowledge and skills are a time-tested method for introducing new material. Forexample, when we observed lessons aboutparallelograms, teachers began by reviewingdefinitions of polygons and parallel lines. But whenreview segments last longer than the segments focusedon new material and skills, as we often observed,instructional programs acquire a “two steps back, onestep forward” rate of progress. Teachers’ decisions toreview material frequently stem from the bestintentions, such as a desire to present material in aclear, step-by-step process. As more than one teachersternly reminded us, “People say that repetition isbad, but sometimes it isn’t. I think the kids do betterwhen they repeat things.” Indeed, many teachers weobserved and interviewed believed that slower pacingand repetition helped their students to learn. But howoften this repetition and review occurs needs to be
1William H. Schmidt, Curtis C. McKnight, Pamela M. Jakwerth, Leland S. Cogan, Senta A. Raizen, Richard T. Houang, Gilbert A.Valverde, David E. Wiley, Richard G. Wolfe, Leonard G. Bianchi, Wen-Ling Yang, Seung-Ho Kang, and Edward D. Britton, Facingthe Consequences: Using TIMSS for a Closer Look at United States Mathematics and Science Education (Boston: Kluwer AcademicPublishing, 1998).
2To ease your minds, the eight parts of speech are noun, pronoun, verb, adverb, adjective, article, conjunction, and interjection.
3Setting the Pace
The most predictable and extensive instructionalslowdown occurs during the spring, as teachers stopintroducing new material to review and practice forthe state-mandated achievement tests and the IowaTests of Basic Skills. During the first year of our fieldstudies (1994), this review typically began in March.In each successive year that we visited schools, this“test prep season” began earlier and lasted longer.3
3. Weak homework norms. A great deal of practiceand review occurs in classrooms because it does notoccur at home. Homework is a source of instructionaltime and contributes to instructional pacing for thesimple reason that it extends opportunity to learnbeyond the school day. Furthermore, the commonpractice of beginning homework in class also slowsinstruction by absorbing time when teachers andstudents could work together on new material.
4. Student resistance. Students can slow instructionwhen they resist difficult or unfamiliar material. Weobserved students telling teachers that they had neverbeen taught a particular topic when we had, in fact,observed teachers in earlier grades doing so. Howoften do teachers hear their students protest, “Thisis too hard!” or “We don’t know how to do this” beforestopping to reteach former topics or to providestudents with opportunities to do well by giving themfamiliar tasks? On any given day it may feel wiser orsafer to review and practice old content, postponing
new work for another day when it seems easier topush forward.
5. Battered faith. The constantly shifting prioritiesand mandates of school and district leaders haveeroded many teachers’ faith that it is worthwhile toassume the difficult task of raising and standardizingtheir expectations through formal and informalcollaboration. As one teacher confessed in utterfrustration: “I have been asked to write a differentcurriculum every year, with a different set of teachers,and a different theory of reading. This year, all theycare about is the test; all the other work we did lastyear doesn’t matter any more. Why should I exhaustmyself working for new goals when they will bechanged before we even get there?”
6. Low expectations. In its plainest form, pacingreflects how much teachers aim to accomplish on adaily basis. Teachers who allow blocks of time to gounused cannot make the same progress as those whoare more vigilant about getting started and gettingengaged. Similarly, teachers who set out to accomplishmore during their lessons typically do so. A second-grade class that takes 45 minutes to review sevenwords from the story of the Ugly Duckling does notaccomplish as much as a class that reviews these wordsin 15 minutes and then builds them into somesentences they write together.
3BetsAnn Smith, It’s about Time: School Development and Achievement in Chicago’s Elementary Schools (Chicago: Consortiumon Chicago School Research, 1998).
Setting the Pace: Opportunities to Learn inChicago's Elementary Schools
November 1998
Consortium on Chicago School Research
4 Improving Chicago’s Schools
5Setting the Pace
Setting the Pace:Opportunities to Learn inChicago's Elementary Schools
Julia B. SmithUniversity of Michigan
BetsAnn SmithMichigan State University
Anthony S. BrykUniversity of Chicago
Introduction 1A Visit to Some Chicago Classrooms 3Instructional Opportunities to Learn 9Interpretive Summary 23Endnotes 25References 27
Consortium on Chicago School Research1313 East 60th StreetChicago, Illinois 60637773-702-3364773-702-2010 - fax
November 1998
6 Improving Chicago’s Schools
7Setting the Pace
Introduction
Nineteen ninety-eight completes an important decade of the Chicago PublicSchools’ (CPS) evolving reform movement. The Consortium on ChicagoSchool Research has contributed to this process through a series of reportson school improvement and student learning. In this study, we turn ourattention to one of the most essential concerns of all: the quality of classroominstruction.
At issue here is a critical concern—the opportunities for Chicago’s chil-dren to learn all that is expected of them. We find that while some ChicagoPublic Schools (CPS) students experience instruction that keeps pace withgrade level expectations and test demands, many do not. Moreover, whilesome Chicago teachers are working to align their instructional programsacross classrooms and grades, others operate under widely different as-sumptions about what students should know and be able to do. Especiallytroublesome is the finding that students attending schools in Chicago’smost disadvantaged neighborhoods are much more likely to encounterinstruction that is poorly coordinated and that conveys weak expectationsfor student learning.
Fortunately, there is also good news in this report. We find that someChicago schools—including some that serve disadvantaged neighborhoods—present their students with a progression of challenging instructionalopportunities. In these schools, administrators and teachers are work-ing together to coordinate instructional programs that keep pace withnationally normed expectations for student achievement. Some of thedistinctive characteristics of these schools are also identified in this study.
8 Improving Chicago’s Schools
AcknowledgmentsOur study of classroom instruction involved many researchers, but is particularly indebtedto Eric Camburn, JoAn Chun, Karen DeMoss, Russ Donofry, Gudelia Lopez, and LisaScruggs. It also involved the professional cooperation of hundreds of Chicago Public Schoolsteachers. We appreciate and applaud their willingness to forgo the privacy of their classroomsand to share their insights and experiences.
We would also like to acknowledge the contributions of Benjamin D. Wright, John M.Linacre, and Stuart Luppescu in guiding the development of the content pacing scale. Thiswork is linked to earlier research by Stuart Luppescu on equating multiple forms of the IowaTests of Basic Skills used by the Chicago Public Schools. We continue to be indebted to boththe leadership of the school system and the staff of the Office of Accountability for access todata and advice and assistance in its proper use. The CPS also facilitated the collection of thesurvey data used in this study.
We are grateful for the editorial and production expertise of Consortium staff: Kay KerschKirkpatrick, Sandra Jennings, and Rebecca Williams. The Consortium’s directors and theSteering Committee read and reread this report, and we thank them for their comments.
This report represents a culmination and synthesis of multiple lines of research that havebeen ongoing for several years. This would not have been possible without the sustainedfinancial support for this work from The MacArthur Foundation, The Joyce Foundation,and The Spencer Foundation. Some of the early classroom observation research also wassupported by the Illinois State Board of Education. We would like to thank Al Ramirez,former deputy superintendent, and Connie Wise, Division Administrator of Research andPolicy Development, for their assistance.
We hope this research will inform public understanding and discussion of the challengesteachers face and the supports they need to reach the student learning goals set by the IllinoisState Board of Education. Most important, we hope this research helps to improve the learn-ing opportunities and achievements of Chicago’s students.
9Setting the Pace
A Visit to Some Chicago Classrooms
Imagine yourself in the hallway of a Chicago public school. Pausing outsidea classroom door, you hear a teacher begin a math lesson by saying, “Todaywe are going to study parallelograms. A parallelogram is one example of apolygon with parallel sides.” Without looking into the room to see the ageof the students, what grade level would you guess is being taught? Secondgrade? Fifth grade? Eighth grade? Possibly even tenth grade? If you answeredany or all of the above, you were correct. During our observations of mathinstruction in Chicago, we observed students in all the above gradespresented with introductory lessons on the parallelogram. Each class beganwith either the teacher or a student reading the definition of a polygon anda parallelogram from a math textbook. All of the classes then discussed therequirement that parallelograms have parallel lines. The second-gradestudents went on to draw and measure the perimeter of parallelogramsusing a ruler. In all the other grades, teachers and students spent the rest ofclass learning to measure the angles inside a parallelogram.
When we began observing classes across primary, middle, and high schoolgrades, we did not set out to record and chart students’ exposure to topicand skill sequences. As we observed more and more classes, however, webecame increasingly concerned. We were most troubled when we saw verysimilar lessons taught to students several grades apart, as in the parallelo-gram example above, or the day when two researchers at the same schoolemerged from primary and middle school classrooms to find that they hadobserved comparable lessons on how to write a paragraph. In response tonumerous experiences like this, we began investigating how curriculumcontent progresses from grade to grade in the Chicago Public Schools.
We recognize the need to be careful in our judgments of these observa-tions. Many topics and skills can and should be revisited across students’school careers. Parallelograms can be studied in simple or complex ways,and writing paragraphs can always be improved. But we did not observesuch continuous development and refinement of learned concepts and con-tent. These were not cases of teachers choosing to explore a few topics indepth rather than many topics superficially. Rather, the same topics andskills were being taught again and again, with little to no development intheir content, depth, or complexity.
10 Improving Chicago’s Schools
About Our Survey and Observation DataThe Consortium on Chicago School Research conducted a broad-based survey of Chicago PublicSchools elementary and high school teachers and students in 1994. These surveys covered a widerange of topics, including local school governance, school climate, opportunities for professionaldevelopment and work conditions of teachers, and the dynamics of instruction and classroom life.1
Teachers at each grade also responded to survey items concerning either reading or mathematicsinstruction in their classrooms. In the full survey, there were 6,264 teachers from 384 schools. Theanalyses in this study use information from 2,036 teachers in these schools who were full-timeclassroom teachers and who responded to the mathematics portion of the questionnaire. While thefull sample of teachers was used to generate school information on professional community, onlythe mathematics teacher responses were used in content pacing analyses.
On average, 12 teachers per school responded to questions about mathematics instruction, witha fairly even distribution of grades taught throughout. Most of the teachers who responded to thisportion of the survey were female (84 percent), a portion entirely consistent with the gender distri-bution of the elementary teachers in the city schools as a whole (85 percent female). In addition, 45percent of the teachers identified themselves as white, 40 percent as African-American, and 10percent as Latino/Hispanic, with the remaining 5 percent as Asian-American or “other.” Finally,about 45 percent of the teachers surveyed reported holding a master’s degree or more.
These survey analyses are complemented by classroom case studies that provide more in-depthunderstanding of instruction in the schools. From spring 1994 through spring 1996, a team ofresearchers observed over 800 language arts and mathematics classes in eight Chicago public el-ementary schools and seven high schools. Our sample of schools reflects the most common studentand community characteristics of the Chicago Public Schools.
The observational data in this report reflect the general instructional programs in the CPS. Wedid not observe lessons in magnet programs, honors programs, or remedial and special educationclasses. We observed language arts and mathematics instruction in grades two, five, eight, nine, andten, keeping detailed logs of teaching and learning activities.
Roughly half of the teachers we observed were new or mid-career teachers who had taught be-tween one and nine years. The other half were veteran teachers who had taught for ten or moreyears. The racial distribution of the observed teachers was similar to that of the district’s teacherforce, but included slightly more white teachers and slightly fewer African-American teachers. Nearlyall of the 250 teachers observed participated in one or more in-depth interviews about their teach-ing practices and programs. Observing instruction directly and talking with teachers personallyprovided data and insights that strengthened our findings and enriched our understanding.
All schools and teachers participated in the studies voluntarily. We aimed to record and under-stand rather than critique their work. This positive approach did not obscure the serious problemsand challenges of the general instructional programs of the schools.
1Consistent with the general philosophy of the Consortium as a public informing organization, all datacollected by the Consortium are made available to the public, subject only to concerns about preserving theconfidentiality of respondents. To access the data used in this study, see Bilcer, Luppescu, Sebring, andThum (1996). Contact the Consortium via the World Wide Web at http://www.consortium-chicago.org.
11Setting the Pace
Consider, for example, the observa-tion logs of a set of CPS language artsclasses that we visited (see Figure 1).These classes focused on literary con-cepts such as a story’s setting, maincharacter, and main events. We re-corded dozens and dozens of classesand assignments devoted to these top-ics. In many respects, these lessonswere entirely appropriate. Knowledgeof literary concepts is clearly stated asa learning goal for CPS students, and areview of basic story elements is a natu-ral opening to all kinds of literary dis-cussions. But we frequently observedhigh school students spending as muchtime defining and identifying these ba-sic concepts as their elementary counter-parts (see, for example, the left column inFigure 1), rather than simply noting themfor context and moving on to exploremore advanced literary concepts such asdialogue, figurative speech, hyperbole,satire, symbolism, or personification (seethe right column in Figure 1).
These contrasting examples point tokey problems in the organization of in-struction in the Chicago Public Schools.The students who experienced classeslike those described in the right-handcolumn were exposed to more literaryconcepts and skills than those who tookthe classes described in the left column.In the most basic sense, then, these stu-dents had more opportunities to learnbecause they were required to exploremore disciplinary content. Beyond this,the extra topics and skills studied werevaluable because they built on students’prior knowledge and made progressivelygreater intellectual demands of them. Asa result, these students’ opportunities tolearn were enhanced both by the factthat the instruction was more strongly
Observations of Chicago ClassroomsTeaching Literary Terms
Grade 2
Grade 5
Grade 8
Grade 9
Grade 10
Students begin with appro-priate lessons, but the con-cepts used to study and under-stand new literature do not build on prior knowledge as students progress through the middle grades and high school.
New and increasingly sophisti-cated literary concepts aresteadily added to students'exploration of literature as they advance through the middle grades and high school.
Read The Mitten. Identify andrecord the title, author, setting,and main characters. Review thestory and record events from the beginning, middle, and end ofthe story.
Identify the sequence of events in a story by looking for time-order words such as "first," "next," "then," or "the next day." Summarize the stories according to their main events.
Read Philip Hall Likes Me,I Reckon, Maybe. Do bulletinboard reports describing thetheme, main character, andsetting of the story.
Read The King Who Rained. Payattention to its use of idioms.List these idioms and write downwhat you think they mean.
Read And Now Miguel. Who isthe narrator? What is thesetting?
Select a novel to read and write about. Make note of its use of hyperbole. List some examples. How does hyperbole help us to write creatively?
Name some of the main char-acters of To Kill a Mockingbird.Where does the story takeplace? Who is your favoritecharacter? Write one paragraphexplaining why.
Select one Greek myth and ana-lyze it according to dimensions such as its hero, the call, the quest, the journey, helpers and guides, and transformation.
Read Miss Cynthie. Who is thenarrator? What is the setting?Describe Miss Cynthie in threesentences. Describe David inthree sentences.
Select one African fable and chart it according to the compo-nents of a traditional plot line. Discuss its use of the following: protagonists, antagonists, conflict, foreshadowing, flashback, and irony.
Slow Pacing/Constant Reteaching
Pacing Aimed atStudent Developmentvs.
Figure 1
12 Improving Chicago’s Schools
The Instructional Depth vs. Coverage DebateThe topic of instructional pacing can easily raise confusion and criticism. Cross-national analyses ofcurriculum have criticized United States schools for excessive concern about covering an extensive arrayof content at the expense of in-depth student work on specific content.1 To be clear, the phenomenon weare observing in Chicago is not that of teachers taking time to engage their students in in-depth studiesof a smaller number of key disciplinary concepts. In fact, over 80 percent of the lessons we observedduring our field studies were rated as having little to no depth or complexity in the information trans-mitted or in the understanding students demonstrated.2 In truth, our observations and analysis suggestthat the “depth versus coverage” debate does not often apply to instruction in Chicago classrooms. Ourdata indicate that neither occurs with much frequency. The pacing lag we identify seems an outcome ofheavy topic review and repetition unaccompanied by any increase in instructional depth.
Our concern is not that instruction be mindlessly speeded up or that more is necessarily better.Instead, we believe that students should experience a sequence of instruction that exposes them ina systematic and developmentally challenging fashion to the content on which they are tested.
The Chicago case we illustrate here may well be an acute example of problems associated with math-ematics programs in other urban school systems and, indeed, the country as a whole. In the most recentreport of the Third International Math and Science Study, U.S. math classrooms were notable for thelimited amount of time they gave to introducing new material to their students (about 10 minutes permath lesson) and a persistent focus on basic skills. To quote from the report, “U.S. mathematics instruc-tion was centered on computation and homework. It focused comparatively less on new instruction andmore on review. These practices are consistent with a view of school mathematics as made up of discretebits, to be mastered cumulatively and incrementally, for which constant review is necessary...”3
The importance of more in-depth learning opportunities in the Chicago Public Schools is a seriousconcern. We examine this aspect of instruction in a companion study, titled The Quality of IntellectualWork in Chicago Schools: A Baseline Report.4
1See, for example, studies by Stevenson and Stigler (1992) and Schmidt et al. (1998) for TIMSS curriculumanalysis reporting.
2Ratings of in-depth learning were made using rubrics developed at the Center on Organization and Restructur-ing of Schools by Newmann and Wehlage (1995).
3See Schmidt et al. (1998), p. 19.
4See Newmann, Lopez, and Bryk (1998).
13Setting the Pace
Reading and Exploring a NovelLiterature Progresses, but Intellectual Activities Do Not
Fifth Grade: Charlotte's Web Tenth Grade: Bless Me, Ultima
Students read much of thebook aloud in class.
Students read much of thebook aloud in class.
Students take short testsand quizzes with who,what, and where questionsto review the facts of thestory.
Students take short testsand quizzes with who,what, and where questionsto review the facts of thestory.
The teacher selects thetheme of friendship as afocus of discussion andwriting.
The teacher selects thetheme of superstition asa focus of discussion andwriting.
Students write a shortessay on friendship. They write one draft only.
Students write a shortessay about superstition.They write one draft only.
paced and because lessons were morethoughtfully developed over time.
Figure 2, which documents how afifth-grade and a tenth-grade class ex-plored a novel, offers another view ofthese problems. We observed bothclasses reading high quality, grade ap-propriate works of literature: Charlotte’sWeb in the fifth grade, Bless Me, Ultimain the tenth. In this sense, we could saythat everyone was working at gradelevel. But the assignments given to thetenth-grade students were so much likethose asked of the fifth graders that theolder students’ opportunity to learn wasgreatly minimized. Both classes spentmost of their lesson time reading thenovels aloud in class and answering ba-sic comprehension questions posed bytheir teacher: “What was the rat’s re-sponse to Charlotte’s question?” “Whatdid the grandmother hang over thedoorway?” Both classes had a theme,selected by the teacher, that was tracedthrough the chapters. And both classesof students were given several weeks towrite one draft of a short report relatedto this theme. While these assignmentsseemed reasonably challenging for fifth-grade students, they do not come closeto meeting standards for high schoollearning and achievement. For example,tenth-grade students should read moreat home. They should move beyond thebasic facts of their novel to interpret theauthor’s purpose and message and todiscuss their own thoughts on the story.We also expected significant differencesin the writing assignments given to stu-dents and the standards used to assessthem.
The examples in Figures 1 and 2 arefrom language arts lessons, but we couldalso illustrate these points with data
Figure 2
from the math classes we visited (see Repeat Offenders sidebar). Many ofthe lessons we observed reflected confused or dampened expectationsfor student learning. Indeed, some lessons completely defied expecta-tions when, for example, elementary lessons were more demanding thansome of the middle grade and high school lessons we saw. As our obser-vations of these phenomena accumulated, we began to question whatwas actually guiding teachers’ instructional practice. When, for example,are students expected to know and use the literary concept of a story’ssetting without further instruction? It became clear that, in at least someof the schools we studied, faculty members lacked a shared conceptionof the instructional program overall, and of their own particular set ofresponsibilities for advancing it.
14 Improving Chicago’s Schools
Repeat OffendersSome topics and learning goals seem to have a clear time and place in the schools’ instructional program. Forexample, we consistently observed capitalization taught in the second grade, the metric system in the eighthgrade, and Romeo and Juliet in the ninth. But many other topics do not appear to have a clear time and place.Below are some of the subjects and skills we saw repeated or stretched out across as many as five differentgrades.
Language ArtsUnderstanding fiction versus nonfictionIdentifying and defining setting, main character, and main eventsHow to create a story boardHow to write a formal letterThe eight parts of speechHow to use the commaHow to write a summary paragraphThe structure of a five-paragraph essayHow to write a footnote or bibliography
Additionally, many language arts activities that necessarily repeat do not necessarily develop. The mostcommon examples observed were class presentations, three-minute speeches, summarizing a book by mak-ing a book jacket, and writing a book report. The expectations and standards applied to these products donot systematically rise as students move through middle and high school grades.
MathMoney problemsParallel and perpendicular linesPlace value, 1-1000DecimalsArea and perimeter of polygons and trianglesLong divisionFractions, proportion, and percentHow to use a number lineRight anglesRadius, diameter, and circumference
Both Math and Language ArtsHow to read, create, and color bar graphs and pie chartsHow to read street maps and transportation schedules
15Setting the Pace
Instructional Opportunities to Learn
Our in-depth field studies raised critical concerns about how some Chicagoschools organize and pace instruction and how this structure affects students’opportunities to learn. Now, we turn to a system-level analysis of theseissues. While we could not create a systemwide database of direct classroomobservations, we were able to gather survey reports from Chicago PublicSchools teachers of the content they taught their students. These dataallowed us to investigate the prevalence of the problems identified in thefield work and how they were distributed among the diverse schoolcommunities that comprise the CPS.
Put simply, we wanted our analysis to inform us whether Chicagostudents are given the opportunity to learn all of the content for whichthey are held accountable. We focused our inquiry on mathematicsbecause math instruction is more commonly approached as a system-atic presentation of topics that build upon prior knowledge and thatgrow in complexity.1 Accordingly, it was easier to develop reliable sur-vey items that ask teachers to report their math content emphases foreach elementary grade level.
We organized this research around the de facto math standards of theCPS—the subject matter content of the Iowa Tests of Basic Skills (ITBS).The ITBS has been used by the CPS as a measure of student achievementfor more than two decades but has taken on heightened prominence in thelast few years. Student performance on these tests is now the primary in-formation base for an increasingly high-stakes student accountability sys-tem. Among other uses, students’ test results are now used to mandatesummer school attendance, to deny grade promotions, and to delay entryinto the city’s regular high schools.
Before we analyzed teacher surveys, however, it was necessary to con-duct an extensive content analysis of the actual math items in the ITBS.2
Each test item or question was grouped into one of 48 categories. Thesecategories sort test items according to both mathematic concepts (e.g.,addition, geometric congruence, ratio) and overall difficulty (difficulty levelsreflect the percent of students at a given grade level able to answer an itemcorrectly).3 Though individual items associated with different mathematic
16 Improving Chicago’s Schools
Fig
ure
3G
rad
e L
evel
Mas
tery
Ru
ler
in t
he
Iow
a Te
sts
of
Bas
ic S
kills
Bey
on
d
8th
Gra
de
8th
Gra
de
7th
G
rad
e
6th
Gra
de
How
muc
h w
ater
mus
t be
adde
d to
50
ml o
f 20
% a
cid
to p
rodu
ce a
mix
ture
whi
ch is
12%
ac
id?
Iden
tify
the
root
s of
:
2 +
4 x
- 5
Sol
ve:
5A +
3B
= 1
22A
–
5B =
9
If th
ere
are
40 r
ed b
alls
and
30
blac
k ba
lls, w
hat i
s th
e pr
obab
ility
of
get
ting
2 re
d ba
lls in
a s
ingl
e dr
aw?
Wha
t is
the
equa
tion
of a
line
cont
aini
ng th
e po
ints
(2,
1) a
nd(-
1, 2
)?
Sim
plify
:
2(3
x +
1)
x (
5 x
+ 2
)
Ske
tch
a gr
aph
of:
y =
4 –
4x
If 12
can
s co
st$3
.00,
how
muc
hw
ill 4
0 ca
ns c
ost?
3.4
- .6
01 =
?W
hat i
s 12
%of
80?
-34
– 21
= ?
4 =
?2
3
9
4
+
=?
- 5
+
1 =
?
645
73
+ ?
< 9
Whi
ch s
how
s th
eco
mm
utat
ive
pro-
pert
y of
add
ition
?
Wha
t wou
ld 6
49be
rou
nded
toth
e ne
ares
t ten
?
Wha
t pla
ce is
the
5 in
3.0
052?
Wha
t are
the
prim
e fa
ctor
s of
85?
How
man
ym
eter
s ar
e in
10 y
ards
?
Qu
adra
tic
Eq
uat
ion
sS
yste
ms
of
Eq
uat
ion
s
Sta
tist
ics
and
Pro
bab
ility
Eq
uat
ion
s o
f L
ines
Sim
plif
y A
lgeb
raic
E
xpre
ssio
ns
Gra
ph
Eq
uat
ion
s
Rat
io, P
rop
ort
ion
Per
cen
t P
rob
lem
sO
per
atio
ns
wit
hN
egat
ives
Exp
on
ents
,R
oo
tsO
per
atio
ns
wit
hF
ract
ion
sA
bso
lute
Val
ue
Lo
ng
Div
isio
nS
olv
eIn
equ
alit
yN
um
ber
Pro
per
ties
Ro
un
din
gD
ecim
alP
lace
Val
ue
Pri
mes
, Fac
tors
,M
ult
iple
sC
onv
erti
ng
Mea
sure
men
t
Iow
a Te
sts
of B
asic
Ski
lls it
ems
repr
inte
d w
ith p
erm
issi
on fr
om R
iver
side
Pub
lishi
ng
Not
e: A
ll of
thes
e ar
e m
ultip
le c
hoic
e qu
estio
ns in
whi
ch th
e st
uden
t is
aske
d to
iden
tify
the
corr
ect a
nsw
er fr
om fo
ur p
ossi
bilit
ies
give
n. B
ecau
se o
fsp
ace
limita
tions
, we
pres
ent o
nly
the
ques
tions
.
Ad
van
ced
Wo
rd P
rob
lem
s
3
x
3+(2
+4)
= (
3+2)
+4
3+2+
4 =
4+
3+2
Op
erat
ion
s w
ith
D
ecim
als
17Setting the Pace
5t
hG
rad
e
4th
Gra
de
3rd
G
rad
e
2nd
Gra
de
Ap
pro
xim
atio
nA
rea
Volu
me
Mu
lti-
dig
itM
ult
iplic
atio
n
1st
Gra
de
Sim
ple
Div
isio
nP
lace
Val
ue
and
Reg
rou
pin
gF
ind
ing
Len
gth
,P
erim
eter
Sh
ifti
ng
Bet
wee
nW
ord
s an
d N
um
ber
sW
ord
Pro
ble
ms
wit
hM
ult
iplic
atio
n, D
ivis
ion
Mu
lti-
dig
itA
dd
itio
nM
ult
iplic
atio
nFa
cts
Mu
lti-
dig
itS
ub
trac
tio
nM
easu
rin
gC
om
mo
n O
bje
cts
Co
un
tin
g b
yn
>1
Co
nver
tin
g U
nit
so
f Tim
e
Co
ins,
Val
ue
of
Mo
ney
Telli
ng
Tim
eU
sin
g M
od
els
for
Op
erat
ion
sU
sin
g N
um
ber
Lin
esan
d R
ule
rs
Co
un
tin
gO
bje
cts
Iden
tify
Sh
apes
Nam
e, O
rder
Nu
mb
ers
Sim
ple
A
dd
itio
nS
imp
leS
ub
trac
tio
nId
enti
fyO
per
atio
ns
Wo
rd P
rob
lem
sw
/Ad
d, S
ub
trac
t
Wha
t por
tion
is s
hade
d?2
( ?
+ 4
) =
10
Wha
t num
ber
is th
e cl
oses
tto
the
sum
of 2
98 a
nd 3
004?
Wha
t is
the
area
of t
he c
ircle
2"
32 x
45
= ?
25
5 =
?B
y ho
w m
uch
will
the
valu
e of
the
num
ber
chan
ge if
the
4 is
chan
ged
to a
5 in
742
?
How
long
is th
e pe
rimet
er o
fth
e fe
nce
show
n?
4'
3'1'
2'
How
do
you
writ
e 2,
340
in w
ords
?If
one
penc
il co
sts
15 c
ents
,ho
w m
any
can
you
buy
for
$3.0
0?
43
+ 1
9 ?
Wha
t is
3 x
5? 6
7-
8 ?
How
tall
is th
e do
or?
Fill
in th
e m
issi
ng n
umbe
r3,
6,
? ,
12
How
man
y se
cond
s ar
ein
2 a
nd a
hal
f hou
rs?
Jan
has
3 qu
arte
rs a
nd 2
dim
es. H
owm
uch
mon
ey is
that
?
Wha
t tim
e is
sho
wn?
12
39
6
Wha
t pic
ture
sho
ws
that
hal
f of 6
is 3
?U
se th
e ru
ler
to fi
nd th
e le
ngth
of
the
line.
1+
2+
(spo
ken)
How
man
y?W
hich
is a
circ
le?
(spo
ken)
Whi
ch n
umbe
r is
ei
ghte
en?
8
81
18
(spo
ken)
Wha
t is
4 +
4?
8 -
6 =
?
Wha
t sig
n te
lls y
ou to
subt
ract
?
Ric
k ha
d 9
cook
ies
and
ate
3. H
owm
any
did
he h
ave
left?
. .
,
3200
33
00
1'
Siz
es o
f F
ract
ion
sS
olv
e fo
r O
ne
Un
kno
wn
18 Improving Chicago’s Schools
concepts can overlap in their level ofdifficulty, the elementary math con-tent of the ITBS generally arrangesitself in a progression of concepts thatbuild on one another and grow in dif-ficulty. Figure 3 lays out this sequenc-ing of item content by difficulty. Theruler is anchored in national mathnorms at each grade level and identi-fies the concepts and skills that stu-dents are expected to master year byyear. These math norms are based onactual test results from a nationalsample of students. Notice that thebottom end of the scale assesses mas-tery over relatively simple mathemat-ics concepts of “counting objects,”“simple addition,” and “telling time.”In the mid-range, a student perform-ing on level for fifth grade, for ex-ample, should show competence withsuch skills as “sizes of fractions,” “area,volume,” and “multi-digit multiplica-tion.” (This student should also dem-onstrate competence in the simpleskills that appear below fifth gradeon the ruler, such as “simple divi-sion.”) At the very top of the scale,beyond eighth grade, are algebraicitems that assess mastery over suchtopics as “systems of equations” and“quadratic equations.”
Overall, the math ruler providesa general picture of the grade-to-grade organization of instructionassumed in the ITBS. These are thede facto standards against whichCPS students are being judged bythe accountability system.
Next we turned to elementaryschool teacher survey reports to ex-amine whether students actually havethe opportunity to learn this content.Specifically, elementary teachers were
Pacing of Math Curriculum Slows Downin Upper Grades
Grade Taught
Gra
de L
evel
Con
tent
Em
phas
is
K 1st 2nd 3rd 4th 5th 6th 7th 8th
9th
8th
7th
6th
5th
4th
3rd
2nd
1st
K
Grade-level pacing line
How to Read a Box PlotEach box encloses the middle 50 percent of the teachers' respon-ses. The broken line inside the box is the median; half of the schools are above this line, half are below. The lines extending up and down from the box, called "whiskers," show the top and bot-tom 25 percent of the teacher content coverage reports.
Note: The grade-level pacing line indicates that content coverage is keeping pace with grade level learning, based on equating teachers' reports of content coverage with the Iowa Tests of Basic Skills.
Figure 4
Note: The grade-level pacing line indicates that content coverage iskeeping pace with grade level learning, based on equating teachers’reports of content coverage with the Iowa Tests of Basic Skills.
How to Read a Box PlotEach box encloses the middle 50 percent of the teachers’responses. The broken line inside the box is the median; halfof the schools are above this line, half are below. The linesextending up and down from the box, called “whiskers,” showthe top and bottom 25 percent of the teacher content cover-age reports.
19Setting the Pace
asked to indicate the grade level oftheir class, the particular math con-tent covered that year, and the rela-tive amount of time spent on eachcontent area. Using these data, wewere able to create a scale of con-tent taught, comparable to the ITBSmeasurement ruler in Figure 3. Thisallowed us to directly compare thecontent coverage by Chicago teach-ers with the content students are ac-tually tested on by the ITBS.4 Thiscomparison, in turn, enabled us toassess whether Chicago students areafforded an adequate opportunity tolearn the material for which they arenow held accountable.
The dark black line in Figure 4identifies the results expected if teach-ers’ content emphases align with ITBSgrade level mastery.5 In other words,the line represents math content thatwe should expect teachers to teach ifstudents are being given the opportu-nity to learn what is being tested onthe ITBS. The box plots indicate therange of content emphases actuallyreported by Chicago teachers at eachgrade level. At second grade, for ex-ample, more than half of the teachersindicate content coverage consistentwith (or above) grade level mastery(that is, their content emphases scoreat or above the heavy black line).About a quarter of those teachers re-port instruction well above grade level,moving into the third to fourthgrade mastery range. In contrast,three-quarters of the teachers ateighth grade report content empha-ses below the grade level line, withthe bottom quarter of eighth-gradeteachers indicating a content focusequivalent to about grade three.
The most immediate impression here is that teachers’ own accountsvalidate our classroom observations. Teachers are reporting in our surveyswide variability in students’ exposure to subject matter and frequent rep-etition of topics across one or more years. While some classrooms are work-ing on content consistent with student achievement at or above nationalnorms, the vast majority are not.
We have used the teacher survey reports on content emphases to for-mulate two key indicators of each school’s organization of instruction.
Figure 5
About Half of CPS Schools Maintain Grade-Level Pacing;the Other Half Fall Far Behind
Top 10 percent of schools
Top 50-90 percent of schools
Bottom 10 percent of schools
Gra
de L
evel
Con
tent
Em
phas
is
K 1st 2nd 3rd 4th 5th 6th 7th 8th
9th
8th
7th
6th
5th
4th
3rd
2nd
1st
K
Bottom 10-50 percent of schools
Grade Taught
▲
▲
▲
▲
▲
▲
▲
▲
▲
**
* **
* ** *
Grade-level pacing line
▲
*
20 Improving Chicago’s Schools
Instructional PacingBy combining the survey responses ofmathematics teachers within a school,we can compute an instructionalpacing trajectory for that school.6
Figure 5 illustrates the range of pacingwithin the Chicago school system. Wesee that in most Chicago elementaryschools, kindergarten teachers reportinstructional emphasis consistent withor above ITBS norms. Moreover,between kindergarten and second grade,teachers in most schools continue toreport a steady introduction of newtopics (as evidenced by the steady climbof their pacing lines), keeping close toor exceeding the ITBS expectations.
Content coverage starts to differen-tiate at third grade. The mathematicscontent tested on the ITBS continuesto advance, yet the pacing of roughlyhalf of Chicago’s schools begins to flat-ten out and fall seriously behind. In-deed, many schools at or above ITBSexpectations in primary school are al-most a year behind by the fourthgrade. By eighth grade, many schoolscontinue to teach material from theprimary and middle grades curricu-lum. Indeed, less than 50 percent ofChicago’s schools keep pace with theITBS during the last three years of el-ementary schooling.
Unfortunately, we do not have thesame kind of data for the high schoolTAP test as we do the ITBS, so wecannot continue this graph throughthe upper grades. But our field obser-vations of hundreds of high schoolmath lessons provide considerable evi-dence that this pacing lag continuesthrough high school, seriously dimin-ishing students’ math achievement andtheir preparation for higher education.
One indicator examines the rate at which teachers in a school introducenew and more difficult subject matter to their students. We call this indi-cator instructional pacing. Instructional pacing is key to understandingwhether topics are being introduced at a rate consistent with the demandsand expectations of the ITBS. Second, we look at the coordination of in-structional content within each school grade, which we call grade-level co-herence. Is there, for example, a distinctive fifth-grade curriculum, or doeseach fifth-grade teacher “do her own thing”? Both good instructional pac-ing and grade-level coherence are necessary to provide students with theopportunities to learn what the ITBS tests.
Pacing Slows Down Dramatically in High-Poverty Schools
Mixed-income schools
High-poverty schools
Gra
de L
evel
Con
tent
Em
phas
is
K 1st 2nd 3rd 4th 5th 6th 7th 8th
9th
8th
7th
6th
5th
4th
3rd
2nd
1st
K
Note: These two groups represent the highest and lowest quartiles in terms ofpercent low-income enrollment in the CPS. The mixed-income schools have less than 50 percent poverty. High-poverty schools enroll greater than 90 per-cent low-income students.
Grade Taught
Grade-level pacing line
Note: These two groups represent the highest and lowest quartiles in termsof percent low-income enrollment in the CPS. The mixed-income schoolshave less than 50 percent poverty. High-poverty schools enroll greaterthan 90 percent low-income students.
Figure 6
21Setting the Pace
Effects of basic school character-istics. Our concern that all studentsbe provided opportunities to learn thecontent on which they are tested ledus to investigate whether certain typesof Chicago schools are more pronethan others to slow pacing. We exam-ined the effects that a variety of schooland community characteristics mighthave on the pace of instruction. Thesefactors include the prior achievementlevels of the school, racial composi-tion, percent low-income enrollment,school size, and rates of student mo-bility. Not surprisingly, we found thatcurriculum pacing was positively re-lated to the school’s achievement his-tory. Many selective academic magnetschools, for example, have pacing tra-jectories well above the ITBS nationalnorm reference line.
More telling is what we find aftertaking prior achievement into ac-count. Even when we control forprior achievement, the percent oflow-income students enrolled in aschool influences instructional pac-ing.7 To illustrate this effect, Figure6 compares instructional pacing tra-jectories from high-poverty schools(where more than 90 percent of thestudent body comes from low-incomefamilies) with more mixed-incomeschools (where less than 50 percentof the student body comes from low-income families). Chicago studentswho attend schools with high con-centrations of poverty experienceslower instructional pacing that ba-sically flattens out around fourthgrade. While this repetition mayhelp reinforce primary skills, it alsolimits students’ exposure to new top-ics, including topics on which they
Figure 7
Pacing Is Fastest in Integrated Schools and Slowest inPredominately Minority and African-American Schools
Integrated schools (over 30% white)
Predominately Hispanic schools(over 85% Hispanic)
Gra
de L
evel
Con
tent
Em
phas
is
K 1st 2nd 3rd 4th 5th 6th 7th 8th
9th
8th
7th
6th
5th
4th
3rd
2nd
1st
K
Racially mixed schools (15% to 30% white)
Predominately minority schools(over 85% mixed minority)
Predominately African-American schools(over 85% African-American)
Grade Taught
*
▲
x
*
▲
x
Grade-level pacing line
▲
▲
▲
▲
▲
▲
▲
▲
x
x
x
x
x
x
x x
***
**
* *
*
22 Improving Chicago’s Schools
Equally troubling are the relation-ships between pacing and the racialcomposition of the school (see Figure7). Though the pattern varies somefrom grade to grade, integrated schoolsare the most likely to keep pace withITBS norms, followed by Chicago’s ra-cially mixed schools and predomi-nately Hispanic elementary schools.Predominately minority and African-American schools are the most likelyto suffer slow pacing that lags behindITBS demands.9 These findings echothe results of other Consortiumstudies, which have suggested thatmany of Chicago’s most racially iso-lated schools, especially its African-American schools, have been left behindby reform. These schools continue to facea very steep climb to national norms.10
The evidence in Figures 6 and 7implies that students’ social back-ground has a great deal to do with howmuch material they are given and ex-pected to learn. This gap between whatis taught to students from differentfamily and community backgroundsreveals major differences in students’opportunities to learn within the Chi-cago Public Schools. This finding iscause for serious concern. A significantbody of research has demonstrated thataccess and exposure to math contenthas clear and predictable effects on theachievement of students. Regardless offamily and community background,students who are exposed to moremath content and an overall richermath curriculum have higher levelsof math achievement than peerswhose learning opportunities aremore limited.11
More troubling still, we find thatsuch students are further disadvan-
will be tested. Other inquiries into school life suggest that slow pacingmay also leave students unmotivated and resentful toward school.8 And,as mentioned, our observations found no evidence that slower pacingwas contributing to more in-depth teaching and learning. The class-room life of these students appears to consist of repetitive cycles ofbasic skills instruction.
In contrast to what we found in predominately high-poverty schools,students in schools with mixed-income populations typically follow amath curriculum that keeps pace with the ITBS. Students in theseschools are introduced to significantly more topics and skills in eachgrade than are the students in high-poverty schools. The pattern be-comes apparent between fourth and fifth grade, and the gap grows largerwith each successive year.
Small Schools Are More Likely To Keep Pace
Small schools (enrollment less than 350 students)
Larger schools (enrollment greater than 750 students)
Gra
de L
evel
Con
tent
Em
phas
is
K 1st 2nd 3rd 4th 5th 6th 7th 8th
9th
8th
7th
6th
5th
4th
3rd
2nd
1st
K
Grade Taught
Grade-level pacing line
Figure 8
23Setting the Pace
taged by other organizational charac-teristics common to their schools. Forexample, Chicago’s high-povertyschools tend to be larger, with morestudents in each grade level. The av-erage school enrollment in a CPS el-ementary school serving a more than90 percent low-income student bodyis 676 students. In contrast, schoolsserving a mixed-income student bodyhave an average enrollment of 435 stu-dents. High-poverty schools also havemuch higher levels of student mobil-ity (34 percent) than mixed incomeschools (10 percent). Our analysesshow that both of these factors—school size and the level of student mo-bility in and out of schools—are alsoassociated with instructional pacing(see Figures 8 and 9 respectively). Thatis, even after we control for the effectsof prior academic achievement andstudent poverty, we still find signifi-cantly stronger pacing in smallerschools and in schools with lower lev-els of student mobility. These findingsillustrate how family and communitypoverty interact with aspects of theschool organization to limit studentlearning opportunities.
An organizational antidote:school-based professional commu-nity. We also considered in ouranalysis how a variety of school or-ganizational characteristics mightinfluence the pace of instruction.12
One organizational characteristicstood out as especially important:the level of professional communityamong teachers. As we have elabo-rated in earlier Consortium reports,three core activities characterizeadult work in a professional commu-nity. First, teachers regularly engage
in reflective conversations that hold teaching practices and student per-formance up to scrutiny. Second, to substantiate these conversations, teach-ers open their classroom doors to share their work with peers and to lend apublic character to their teaching. This openness leads to a third key char-acteristic—peer collaboration. Through shared work, teachers learn fromone another and continue to develop the skills, knowledge, and ideas nec-essary for continuous school improvement. Undergirding these professionalbehaviors is a set of shared beliefs and values, central to which is a collec-tive responsibility for student learning. In sum, schools with strong profes-sional communities provide a normative environment that guides adultbehavior and promotes strong commitments to the welfare and educationof students.13
Note: Schools in the lowest quartile, low-mobility schools, have less than19 percent mobility. The top-quartile (high-mobility) schools experiencemore than 34 percent mobility.
Schools with High Mobility Have Pacing Problems
Low-Mobility Schools
High-Mobility Schools
Gra
de L
evel
Con
tent
Em
phas
is
K 1st 2nd 3rd 4th 5th 6th 7th 8th
9th
8th
7th
6th
5th
4th
3rd
2nd
1st
K
GradeTaught
Grade-level pacing line
Figure 9
24 Improving Chicago’s Schools
munity—students pay a price. Whileit doesn’t significantly affect studentsin kindergarten through fourth grade,students in succeeding grades experi-ence slower curriculum pacing.
It is important to emphasize thatthis difference remains even when vari-ous aspects of students’ background aretaken into account. To demonstratethe independent influence of profes-sional community on pacing, Figure11 presents results from an analysis ofhigh-poverty schools only. Althoughthe pacing trajectories for these high-poverty schools are somewhat weakerthan the system as a whole, we againobserve a large pacing gap betweenschools that have strong professionalcommunity and those that do not.
In short, it isn’t just the back-grounds of the students that slowsdown instruction, it is also the wayteachers do or do not work togetherin the school. Our results indicate thatwhen teachers do not collaborate intheir curriculum development andteaching, do not regularly confer withone another, and do not reflect on theirpractices and outcomes, slower pacingwith more extensive repetition and re-view is likely to ensue.Conversely, schools with a strongprofessional community are places inwhich the faculty seem much moreable to pursue grade-level learninggoals. This positive relationship offerspromise for schools seeking to developbetter pacing and alignment of theirinstructional programs. While there isno precise recipe for how to developteachers’ professional community, itskey building blocks—common goals,collaborative work, frequent dialogue
We found strong relationships in our analysis between the instructionalpacing of a school and its level of professional community. Figure 10 showsthe results. In this figure, we compare the pacing trajectories of the CPSschools with the strongest measures of professional community (the top25 percent) to those with the weakest measures (the bottom 25 percent).Notice that students who attend an elementary school where faculty enjoystrong professional community tend to receive instruction closely alignedwith the ITBS math content standards (i.e., consistently at or above theITBS grade-level mastery line). On the other hand, in schools where teacherswork in isolation and do not collaborate—i.e., a weak professional com-
Schools with Strong Professional CommunityReport Faster Pacing
Strong professional community schools
Weak professional community schools
Gra
de L
evel
Con
tent
Em
phas
is
K 1st 2nd 3rd 4th 5th 6th 7th 8th
9th
8th
7th
6th
5th
4th
3rd
2nd
1st
K
Note: Strong professional community schools comprise the top 25 percent of the CPS elementary schools on the professional community index. Weak professional community schools comprise the bottom 25 percent of the CPS elementary schools on the same index.
Grade Taught
Grade-level pacing line
Figure 10
Note: Strong professional community schools comprise the top 25percent of the CPS elementary schools on the professional commu-nity index. Weak professional community schools comprise the bot-tom 25 percent of the CPS elementary schools on the same index.
25Setting the Pace
about improving practices, and a strongsense of collective responsibility—provide the needed social and structuralsupports for teachers to tackle pacingand alignment problems.
Grade-Level CoherenceA school’s instructional pacingtrajectory describes the overallopportunity to learn afforded by aschool as students pass through theelementary grades. This is a school-wide average. Any individual student’sexperience, however, consists ofexposure to a specific set of classroomsacross grade levels. We note thisdistinction because we often found inour field observations considerablevariability in instruction amongclassrooms at the same grade. In suchsituations, a school’s average pacingtrajectory might appear appropriate,but it would not necessarily indicatethat all students attending that schoolexperience consistent instruction. Astudent may be in a classroom that iswell below average one year and aboveaverage the next. For example,consider again the sequence of literacylessons we presented in Figure 1.Students may not move straight downthe columns, but back and forth fromone side to the other as they progressthrough the grades. Such studentsmight lose time studying topics theyalready know one year and encountertopics they are entirely unprepared totackle the next. The more variable thecontent coverage is within any givengrade, the more likely it is thatstudents will receive incoherentinstruction as they pass from grade to
grade.14 Examining the grade-level coherence of instruction allows us toconsider the likelihood that students experience these problems.
We returned to teachers’ content emphasis reports and computed ameasure of the similarity in math content coverage among teachers bygrade level. If most teachers working in the same grade report that theycover a similar set of math topics, their school’s measure of grade-levelcoherence would be high. In contrast, if these teachers’ reports withingrade levels do not overlap much, their measure of grade-level coher-ence would be low.15
Strong Professional Community Improves CurriculumPacing in High-Poverty Schools
Strong professional community schools
Weak professional community schools
9th
8th
6th
Gra
de L
evel
Con
tent
Em
phas
is
K 1st 2nd 3rd 4th 5th 6th 7th 8th
7th
5th
4th
3rd
2nd
1st
K
Note: These results are for CPS elementary schools that enroll 90 percent or morelow-income students.
Grade Taught
Grade-level pacing line
Note: These results are for CPS elementary schools that enroll 90 per-cent or more low-income students.
Figure 11
26 Improving Chicago’s Schools
Average Pacing and High Coherence
Figure 12a Figure 12b
To illustrate the magnitude of grade level incoher-ence, we grouped together all of the schools with ap-proximately average instructional pacing trajectories andthen examined their grade-level coherence.16 Figure 12asummarizes teachers’ reports of content coverage fromschools in the top quartile in grade-level coherence; Fig-ure 12B summarizes reports from the bottom quartile.Notice that the box plots in Figure 12a indicate rela-tively little spread in teacher reports about content em-phases at each grade level. There is also a steady upwardtrend in the box plots across most grades, demonstrat-ing that teachers in these grades are presenting new andmore difficult topics each year.
As a result, the students in this first group of schoolsare likely to be receiving a coherent program of instruc-tion over time. In contrast, the box plots in Figure 12bdemonstrate extensive variability among classes at thesame grade level. As a result, some students in theseschools might, for example, receive the third-grade cur-riculum in two successive years; others might study the
third-grade curriculum one year and then enter a classthe next fall aimed at fifth-grade material. In short, bothgaps in instruction and repetition of topics may well beexperienced by these students.
Effects of school composition and organizationalfactors. As was the case for instructional pacing, wefound that school composition and organizational char-acteristics had a substantial effect on grade-level coher-ence. Again, even after we controlled for prior schoolachievement, we found that schools with high propor-tions of impoverished families are much more likely tohave incoherent curricula, as are large schools and schoolswith high levels of student mobility. On the brighterside, the positive relationship demonstrated between astrong school-based professional community and instruc-tional pacing also extends to grade-level coherence. Co-herence is more likely in schools where teachers workcollaboratively and share collective responsibility for stu-dent learning.
Average Pacing and Low Coherence
Comparison of Schools with High and Low Within-Grade Instructional Coherence
Grade Taught
Gra
de L
evel
Con
tent
Em
phas
is
K 1st 2nd 3rd 4th 5th 6th 7th 8th
7th
5th
4th
3rd
2nd
1st
K
6th
8th
9th
Grade-level pacing line
Grade Taught
Gra
de L
evel
Con
tent
Em
phas
is
K 1st 2nd 3rd 4th 5th 6th 7th 8th
7th
5th
4th
3rd
2nd
1st
K
6th
8th
9th
Grade-level pacing line
27Setting the Pace
Across all our analyses, we find that the weak pacingand coherence that limits students’ opportunities to learnpersistently characterize many of the system’s high pov-erty schools. The patterns here are quite vivid: in someof the math programs we examined, the pace of learningslows to such a great extent that little to no new materialis introduced across the last several years of elementaryschooling. This result means that tens of thousands of
Chicago students attend schools that fail to provideequitable, grade-appropriate opportunities to learn.Moreover, our extensive classroom observations indicatethat this pattern exists in language arts instruction aswell. This finding leaves us with a major challenge: Ifstudents are not exposed to the knowledge and skills theyare tested on, how then are they to achieve the compe-tencies expected of them?
A Growing Research Base Documents the Effects ofProfessional Communities
The findings on professional community presented in this report add to a growing body of literature on itsimportance to urban school improvement. The Consortium first introduced the concept of school-basedprofessional community in its 1995 report, Charting Reform: Chicago Teachers Take Stock. Early work on thisidea emerged out of a collaboration with colleagues at the federally funded Center on Organization andRestructuring of Schools at the University of Wisconsin-Madison.1 About the same time, closely related ideaswere being developed at the federally funded Center on School Context at Stanford University.2
Over the last five years, evidence has emerged confirming the benefits of schools organized around theprofessional norms we have described. For example, Newmann and Wehlage found teachers’ professionalcommunity to be a critical characteristic of restructured schools that successfully engage students in challeng-ing intellectual work.3
In our previous research, we found that weak school-based professional community characterizes the low-est achieving elementary schools in Chicago, as well as most of the city’s high schools.4 In contrast, Chicagoschools with strong professional communities are more likely to have school climates focused on studentlearning. Evidence is also beginning to appear that strong professional community among teachers may playa key role in sustaining long-term efforts to improve student achievement.5
1See Kruse, Seashore-Louis, and Bryk (1995).
2See Talbert and McLaughlin (1994).
3See Newmann and Wehlage (1995). See also McLaughlin and Talbert (1993).
4See Sebring et al. (1995).
5See Designs for Change (1998). See also Bryk, Sebring, et al. (1998).
28 Improving Chicago’s Schools
29Setting the Pace
Interpretive Summary
Every day Chicago teachers have to juggle dozens of considerations aboutthe young people they teach and the learning goals they strive to reach.Unforeseen events arise; students fail to grasp new material in the allottedtime. But what happens when the juggling, borrowing, and repeatingextends past one or more school years? Our research argues that unalignedand incoherent instructional programs emerge. Students who pass throughthese programs experience delays, repetitions, and/or skips in coreknowledge and skills in ways that seriously diminish their chances for successin school and, in particular, on the tests used to measure their knowledgeand their progress.
A First-Step: Learning StandardsIn the current debates on school improvement, the reform most directlyassociated with instructional pacing and coherence is the standardsmovement. Indeed, both the Illinois State Board of Education and theChicago Public Schools have issued official learning goals and standardsthat articulate what students are expected to know at various grade levels.The Chicago Public Schools are even working to develop daily lesson plansthat illustrate these standards. In our view, well designed and properlypromoted learning standards can assist teachers in reorganizing theirteaching and expectations into more coherent and effective programs ofinstruction. They may also foster educational equity by promoting morecommon learning opportunities for students, regardless of their backgroundand the individual schools and classrooms they attend.
It is critical to remember, however, that curriculum standards arenot new to the Chicago schools. Official “scope and sequences,” not tomention thousands of textbooks outlining grade level curricula, havebeen issued for years. Indeed, the ITBS itself constitutes a long-stand-ing and powerful form of curricular guidance. Our research makes clearthat external guidelines and mandates do not, by themselves, preventtroubling differences in teaching and learning from occurring. As nec-essary and challenging as it is to develop high quality learning stan-dards, standards documents must be understood as only the first stepin a series of changes that actually influence how schools organize andconduct instruction.
30 Improving Chicago’s Schools
A Need to Strengthen Schoolsas Organizations If, in spite of the best laid plans, it is common for teachersto slow down instruction, what might keep thecurriculum moving forward? What will help teacherspush students into new learning? Our research suggeststhat individual teachers need social support to counterthe resistance put forth by students as well as the doubtsthey themselves have about what their students have beentaught and are capable of learning. This study finds thatsome of the most promising supports reside in keyorganizational characteristics and qualities of the school.
In particular, good teacher communication and col-laboration play a vital role in bringing standards frompaper to practice and in shaping how instruction de-velops and progresses across classrooms and grades.Professional interaction and reflection among teach-ers can provide perspective and clarity, whether it isthrough informal sharing of information or formalanalysis of what is being taught and produced in eachclassroom and at each grade level. These are the op-erational dynamics of a school-based professional com-munity at work.
We also find that closer physical and social proximityamong members of a school community can offer pow-erful assistance. Small schools with fewer students in eachgrade facilitate social familiarity and shared knowledgethat reduce some of the doubts and ambiguities that canslow instructional programs. To be sure, small schoolsize does not guarantee more positive relations and in-structional coordination. Such processes, however, aremuch easier for committed adults to sustain in such anenvironment.
Perhaps most troublesome are the negative relation-ships we found between high levels of student mobilityand instructional pacing and coherence. Currently, whennew students enter a school, little information about theirpast instruction accompanies them, even if they are trans-ferring from another CPS school. Typically, they are sim-ply assigned to the classroom at the age-appropriate gradelevel with the most vacant seats. Rather than undertak-ing a careful diagnosis of new students’ knowledge andskills in order to make the best instructional placement
possible, these new students are routinely distributedaround so that “all teachers get their fair share.” Thus,we neither assure instructional continuity throughsystemwide curriculum pacing and coordination northrough individualized assessment and placement. As aresult, many of these new students experience less school
success than their more stable peers. Just as troubling,teachers in high mobility schools resort to even slowerpacing to accommodate the ever changing range of stu-dent knowledge and experience in their classroom—anunderstandable response, but one that unfortunately af-fects all of the students in the classroom.17 To be clear,this problem is not an issue that individual classroomteachers can solve alone; rather, it demands coordinatedschool- and system-level action.
Fortunately, the school characteristics that we haveidentified as significantly related to grade-level pacingand instructional coherence—stronger professional com-munity that sustains rich systems of teacher sharing andreflection, smaller school size, and lower levels of stu-dent mobility—can all be influenced by policy action.Positive developments in each of these areas can advancestudents’ opportunities to learn in a more systematicmanner. Moreover, the future success for institutional-izing new standards for teaching and learning across theChicago schools may well rest on school- and system-level attention to these key organizational concerns.
31Setting the Pace
Endnotes
1See Stodolsky (1988).
2For the purposes of this research, the test form used by the CPSin 1994, identified locally as CPS94, was used. Since the ITBSis not designed to be standards-based in the way that we areusing it in this study, the link between content emphasis andscoring at national norms does vary some from form to form.Since the survey data were collected in 1994, the choice of thistest form seemed most sensible. This test form was reused bythe CPS in 1996 and 1998.
3The results presented here are based on Rasch test item analy-sis. Within this test theory, mastery level is defined as a 75 per-cent probability of getting any particular item correct. Formally,each item has a difficulty measure in logits which also repre-sents a 50 percent probability of a correct response. By adding1.1 logits to this difficulty level, we estimate the mastery level.For a further discussion of our use of these test equating tech-niques, see Bryk, Thum, Easton, and Luppescu (1998).
4We performed a separate Rating Scale Analysis (see Wright andMasters, 1982) on the teacher content reports, where we an-chored the content category difficulties based on the results fromthe ITBS Rasch Analyses. This made the content emphasis scalecomparable to the ITBS difficulty scale, allowing us to producean “initial pacing trajectory” based on the content emphasis datafor each school. Then, in order to establish a precise equatingbetween content emphasis reports and actual student assessedcompetencies, we examined the initial pacing trajectories for asample of schools with students scoring at national norms.This allowed us to introduce a final adjustment to the pacingtrajectories that permits the direct equating of content em-phasis reports and ITBS test results. For a further discussionof these analyses, see Smith and Bryk (Forthcoming).
5This line is empirically derived based on teachers’ reported con-tent emphases in schools where students score near nationalnorms.
6The survey data from each math teacher in the school werescored in terms of content emphases. A two-level HLM analysis(teachers within schools) was then undertaken. At level 1 (withinschools), teacher’s content emphasis was predicted by grade level.This produced a pacing trajectory for each school that was al-lowed to vary at level 2.
7These results are based on a two-level HLM analysis where aninstructional pacing trajectory is calculated for each school atlevel 1 and the shape of those trajectories is predicted by a vari-ety of characteristics at level 2, including prior achievement leveland percent low income enrollment.
8See, for example, Marks (1997).
9These findings are confirmed in HLM analyses like those usedfor examining percent low-income effects. The illustrative re-sults presented in Figure 7 are overall group differences, unad-justed for other factors.
10See Bryk, Easton, Kerbow, Rollow, and Sebring (1993), 19-20. Also see Sebring et al. (1995), and Sebring et al. (1996).
11See, for example, Porter (1994). Also, see Schmidt et al. (1998),and Stevenson and Stigler (1992).
12The school organizational measures considered here derivefrom the Consortium’s work on “Essential Supports for Stu-dent Learning.” They include key aspects of school leader-ship, school-community partnerships, school climate, andprofessional community and development. For more details seeSebring et al. (1995), and Sebring et al. (1996).
13See Sebring et al. (1995). We note that while it is possible thatteachers could apply their collective attentions and efforts toissues unrelated to curriculum pacing and the demands of theITBS, it would be difficult for a faculty to plan and align itscurriculum and its pacing without the supportive dynamics of aprofessional community in the school.
14To study this problem fully, we would like to have longitu-dinal student data over time that tracks the specific subjectmatter and skills to which students are exposed. Unfortunately,the current Consortium database consists only of cross-sectionalinformation on this topic. We note that it is possible for indi-vidual students to receive coherent instruction in schools char-acterized by weak grade level coherence if the school deploysability grouping or tracking. We intend to investigate thisfurther in future research.
32 Improving Chicago’s Schools
15To compute the grade-level coherence measure, we began byestimating for each school, using a two-level HLM analysis, anempirical Bayes pacing trajectory based on teachers’ contentemphasis reports by grade (level 1) and an unconditional modelat level 2. We then computed the residual variance of teacherreports around this pacing trajectory for each school. High re-sidual variance means grade-level incoherence; low residual vari-ance means high grade-level coherence.
16A subset of 22 schools that had pacing trajectories approxi-mately equal to the systemwide average were selected. These 22schools were divided into thirds according to their coherencelevels. The figures display content emphasis reports by gradelevel in the top and bottom third of these schools, respectively.
17For further discussion of the dynamics and consequences ofschool mobility in Chicago, see Kerbow (1996).
33Setting the Pace
References
Bilcer, Diane King, Stuart Luppescu, Penny BenderSebring, and Yeow Meng Thum (1996). The Chart-ing Reform Survey Series: Public Use Data. Chicago:Consortium on Chicago School Research.
Bryk, Anthony S., John Q. Easton, David Kerbow,Sharon G. Rollow, and Penny A. Sebring (1993). AView from the Elementary Schools: The State of Reformin Chicago. Chicago: Consortium on Chicago SchoolResearch.
Bryk, Anthony S., Penny Bender Sebring, John Q.Easton, Stuart Luppescu, Yeow Meng Thum, JennyNagaoka, and Diane King Bilcer (1998). ChicagoSchool Reform: Linkages between Local Control, Edu-cational Support, and Student Achievement. Paperpresented at the annual meeting of the AmericanEducational Research Association, San Diego.
Bryk, Anthony S., Yeow Meng Thum, John Q. Easton,and Stuart Luppescu (1998). Academic Productivityof Chicago Public Elementary Schools. Chicago: Con-sortium on Chicago School Research.
Designs for Change (1998). Chicago Elementary Schoolswith a Seven-year Trend of Improved Reading Achieve-ment: What Makes These Schools Stand Out. Chicago:Author.
Kerbow, David (1996). Patterns of Urban Student Mo-bility and Local School Reform. Journal of Educationof Students Placed at Risk, 1 (2), 147-169.
Kruse, Sharon D., Karen Seashore-Louis, and AnthonyS. Bryk (1995). Professionalism and Community:What Is It and Why Is It Important in Urban Schools?An Emerging Framework for Analyzing School-basedProfessional Community. In Karen Seashore-Louisand Sharon D. Kruse, Professionalism and Commu-nity: Perspectives on Reforming Urban Schools. Thou-sand Oaks, CA: Crown.
Marks, Helen M. (1997). Student Engagement in In-structional Activity: Patterns in the Elementary,Middle, and High School Years. (Revision of StudentEngagement in the Classrooms of Restructured Schools.Madison, WI: Center on Organization and Restruc-turing of Schools, 1995.) Unpublished.
McLaughlin, Milbrey W., and Joan E. Talbert (1993).Introduction: New Visions of Teaching. In DavidCohen, Milbrey W. McLaughlin, and Joan E. Talbert,Teaching for Understanding: Challenges for Policy andPractice. San Francisco: Jossey-Bass.
Newmann, Fred M., Gudelia Lopez, and Anthony S.Bryk (1998). The Quality of Intellectual Work in Chi-cago Schools: A Baseline Report. Chicago: Consortiumon Chicago School Research.
Newmann, Fred M., and Gary G. Wehlage (1995). Suc-cessful School Restructuring: A Report to the Public andEducators. Madison, WI: Center on Organization andRestructuring of Schools.
Porter, Andrew C. (1994). The Uses and Misuses ofOpportunity-to-Learn Standards. Educational Re-searcher 24, 21-27.
Schmidt, William H., Curtis C. McKnight, Pamela M.Jakwerth, Leland S. Cogan, Senta A. Raizen, RichardT. Houang, Gilbert A. Valverde, David E. Wiley,Richard G. Wolfe, Leonard G. Bianchi, Wen-LingYang, Seung-Ho Kang, and Edward D. Britton(1998). Facing the Consequences: Using TIMSS for aCloser Look at United States Mathematics and ScienceEducation. Boston: Kluwer Academic Publishing.
Sebring, Penny Bender, Anthony S. Bryk, John Q.Easton, Stuart Luppescu, Yeow Meng Thum, WinifredA. Lopez, and BetsAnn Smith (1995). Charting Re-form: Chicago Teachers Take Stock. Chicago: Consor-tium on Chicago School Research.
34 Improving Chicago’s Schools
Sebring, Penny Bender, Anthony S. Bryk, MelissaRoderick, Eric Camburn, Stuart Luppescu, YeowMeng Thum, BetsAnn Smith, and Joseph Kahne(1996). Charting Reform in Chicago: The StudentsSpeak. Chicago: Consortium on Chicago School Re-search.
Smith, Julia B., and Anthony S. Bryk (Forthcoming).An Organizational Analysis of Students’ Opportunity toLearn Mathematics.
Stevenson, Harold W., and James W. Stigler (1992). TheLearning Gap: Why Our Schools Are Failing and WhatWe Can Learn from Japanese and Chinese Education.New York: Simon and Schuster.
Stodolsky, Susan S. (1988). The Subject Matters: Class-room Activity in Math and Social Studies. Chicago: TheUniversity of Chicago Press.
Talbert, Joan E., and Milbrey W. McLaughlin (1994).Teacher Professionalism in Local School Contexts.American Journal of Education 102, 123-153.
Wright, Benjamin D., and Geoff N. Masters (1982).Rating Scale Analysis. Chicago: MESA Press.
35Setting the Pace
36 Improving Chicago’s Schools
Notes
37Setting the Pace
This report reflects the interpretations of the authors. Although the Consortium's Steering Committee
provided technical advice and reviewed an earlier version of this report, no formal endorsement by these
individuals, their organizations, or the full Consortium should be assumed.
38 Improving Chicago’s Schools
Consortium on ChicagoSchool Research
MissionThe Consortium on Chicago School Research is an independent federationof Chicago area organizations that conducts research on ways to improveChicago's public schools and assess the progress of school improvement andreform. Formed in 1990, it is a multipartisan organization that includesfaculty from area universities, leadership from the Chicago Public Schools,the Chicago Teachers Union, education advocacy groups, the Illinois StateBoard of Education, and the North Central Regional Educational Labora-tory, as well as other key civic and professional leaders. The Consortiumdoes not argue a particular policy position. Rather, it believes that goodpolicy is most likely to result from a genuine competition of ideas informedby the best evidence that can be obtained.
Melissa RoderickUniversity of Chicago
Penny Bender SebringUniversity of Chicago
Dorothy ShippsUniversity of Chicago
Mark A. SmylieUniversity of Illinoisat Chicago
Directors
Anthony S. BrykUniversity of Chicago
John Q. EastonConsortium onChicago School Research
Albert L. BennettRoosevelt University
Kay Kersch KirkpatrickConsortium onChicago School Research
Steering CommitteeRachel W. Lindsey, Co-Chair
Chicago State University
Angela Pérez Miller, Co-ChairLatino Institute
John AyersLeadership for Quality Education
Tariq ButtChicago School Reform Board of Trustees
Michael E. CarlNortheastern Illinois University
Karen G. CarlsonAcademic Accountability Council
Molly A. CarrollChicago Teachers Union
Victoria ChouUniversity of Illinois at Chicago
Joseph HahnChicago Public Schools
Anne C. HallettCross City Campaign
for Urban School Reform
G. Alfred Hess, Jr.Northwestern University
John K. HoltonHarvard School of Public Health
Richard D. LaineIllinois State Board of Education
James H. LewisChicago Urban League
Donald R. MooreDesigns for Change
Jeri NowakowskiNorth Central RegionalEducational Laboratory
Charles M. PayneNorthwestern University
Barbara A. SizemoreDePaul University
Linda S. TafelNational-Louis University
Beverly TunneyChicago Principals and
Administrators Association
Paul G. VallasChicago Public Schools
Consortium on Chicago School Research
1313 East 60th Street, Chicago, IL 60637
773-702-3364 773-702-2010 - fax
http://www.consortium-chicago.org