Am Educ Res J 2011 Fantuzzo 763 93

7/31/2019 Am Educ Res J 2011 Fantuzzo 763 93

1/32

http://aerj.aera.netJournal

American Educational Research

http://aer.sagepub.com/content/48/3/763The online version of this article can be found at:

DOI: 10.3102/00028312103854462011 48: 763 originally published online 21 October 2010Am Educ Res J

John W. Fantuzzo, Vivian L. Gadsden and Paul A. McDermottfor Head Start Children

An Integrated Curriculum to Improve Mathematics, Language, and Literacy

Published on behalf of

American Educational Research Association

and

http://www.sagepublications.com

can be found at:American Educational Research JournalAdditional services and information for

http://aerj.aera.net/alertsEmail Alerts:

http://aerj.aera.net/subscriptionsSubscriptions:

http://www.aera.net/reprintsReprints:

http://www.aera.net/permissionsPermissions:

What is This?

- Oct 21, 2010OnlineFirst Version of Record

- May 9, 2011Version of Record>>

at UNIV PENDIDIKAN SULTAN IDRIS on June 19, 2012http://aerj.aera.netDownloaded from
http://aerj.aera.net/http://aerj.aera.net/http://aer.sagepub.com/content/48/3/763http://aer.sagepub.com/content/48/3/763http://www.aera.net/http://www.aera.net/http://www.sagepublications.com/http://aerj.aera.net/alertshttp://aerj.aera.net/alertshttp://aerj.aera.net/subscriptionshttp://aerj.aera.net/subscriptionshttp://aerj.aera.net/subscriptionshttp://www.aera.net/reprintshttp://www.aera.net/reprintshttp://www.aera.net/permissionshttp://www.aera.net/permissionshttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://aer.sagepub.com/content/early/2010/10/20/0002831210385446.full.pdfhttp://aer.sagepub.com/content/early/2010/10/20/0002831210385446.full.pdfhttp://aer.sagepub.com/content/48/3/763.full.pdfhttp://aer.sagepub.com/content/48/3/763.full.pdfhttp://aerj.aera.net/http://aerj.aera.net/http://aerj.aera.net/http://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://aer.sagepub.com/content/early/2010/10/20/0002831210385446.full.pdfhttp://aer.sagepub.com/content/48/3/763.full.pdfhttp://www.aera.net/permissionshttp://www.aera.net/reprintshttp://aerj.aera.net/subscriptionshttp://aerj.aera.net/alertshttp://www.sagepublications.com/http://www.aera.net/http://aer.sagepub.com/content/48/3/763http://aerj.aera.net/


2/32

An Integrated Curriculum to ImproveMathematics, Language, and Literacy

for Head Start Children

John W. FantuzzoVivian L. GadsdenPaul A. McDermott

University of Pennsylvania

This article reports on the development and field trial of an integrated Head

Start curriculum (Evidence-Based Program for Integrated Curricula [EPIC])

that focuses on comprehensive mathematics, language, and literacy skills.

Seventy Head Start classrooms (N = 1,415 children) were randomly assigned

to one of two curriculum programs: EPIC or the Developmental LearningMaterials Early Childhood Express, with curricula implemented as stand-

alone programs. EPIC included instruction in mathematics, language, liter-

acy, and approaches to learning skills; formative assessment; and a learning

community for teachers. Multilevel growth modeling through four direct as-

sessments revealed significant main effects and growth rates in mathematics

and listening comprehension favoring EPIC, controlling for demographics

and special needs and language status. Both programs produced significant

growth rates in literacy.

KEYWORDS: at-risk students, early childhood, child development, hierarchi-cal modeling, classroom research

Children from economically disadvantaged households, especially thosefrom minority families in large urban areas, are among the most vulner-able to poor academic outcomes (Aber, Jones, & Raver, 2007). Researchdemonstrates that these children show significantly lower reading profi-ciency relative to peers at all grade levels (Jencks & Phillips, 1998), with

reading achievement gaps exceeding 1.2 standard deviations manifestedas early as preschool (Jencks & Phillips, 1998; West, Denton, & Reaney,2002). Research has also found that children living in povertyfrom allethnic groupsperform worse in mathematics than their nonimpoverishedpeers (Chatterji, 2006; Chernoff, Flanagan, McPhee, & Park, 2007).Moreover, findings from the Early Childhood Longitudinal Study showedthat only 40% of children from low-income homes demonstrated

American Educational Research Journal

June 2011, Vol. 48, No. 3, pp. 763793

DOI: 10.3102/0002831210385446

2011 AERA. http://aerj.aera.net

at UNIV PENDIDIKAN SULTAN IDRIS on June 19, 2012http://aerj.aera.netDownloaded from
http://aerj.aera.net/http://aerj.aera.net/http://aerj.aera.net/http://aerj.aera.net/


3/32

mathematics proficiency versus 65% of children from middle-incomehomes and 87% of children from high-income homes (Chernoff et al.,2007). While such comparative data do not tell the full story of the experi-

ences of children from low-income households, they underscore some ofthe academic challenges facing these young children as they prepare toenter formal schooling and as schools prepare to welcome them. Thesestudies also point to the importance of identifying useful approaches to ad-dressing the challenges they face.

Several studies have found positive associations between comprehen-sive early childhood educational experiences and cognitive achievementfor vulnerable young children (Campbell, Pungello, Miller-Johnson,Burchinal, & Ramey, 2001; Fantuzzo et al., 2005; Schweinhart, 2004). For

example, Head Start, the nations primary early childhood program forlow-income children, has evidenced positive cognitive outcomes for chil-dren. A recent nationally representative, randomized control evaluation ofthe benefits of Head Start found that participant children produced higherperformance on cognitive outcomes (i.e., prereading, prewriting, and vocab-ulary), as compared to children who participated in other early childhoodprograms (U.S. Department of Health and Human Services [USDHHS], 2005).

Taking into consideration the demonstrable benefits of quality earlychildhood education for children from low-income homes and the paucity

of rigorous evaluations of specific curricular programs, the Institute ofEducation Sciences commissioned the Preschool Curriculum EvaluationResearch (PCER) initiative in 2002 (PCER Consortium, 2008). Although

wrap-around early childhood programs, such as the Abecedarian Projectand the Perry Preschool Project, had evidenced positive outcomes for chil-dren from low-income homes, implementation of these programs requiredintensive service increments that restructured the daily routine of programs,making them very costly to operate and operable only in selected settings

JOHN W. FANTUZZO is the Albert M. Greenfield Professor of Human Relations at theUniversity of Pennsylvania Graduate School of Education, 3700 Walnut Street,Philadelphia, PA 19104-6216; e-mail: [email protected]. His research interestsinclude early childhood risk, early childhood education, Head Start, and child mal-treatment and family violence.

VIVIAN L. GADSEN is the William T. Carter Professor of Child Development andEducation, the director of the National Center on Fathers and Families, and the asso-ciate director of the National Center on Adult Literacy at the University ofPennsylvania Graduate School of Education; e-mail: [email protected]. Her

research interests include literacy and at-risk youth, fathers and families, intergener-ational learning, and parental engagement.

PAUL A. MCDERMOTT is a professor at the University of Pennsylvania Graduate School ofEducation; e-mail: [email protected]. His research interests include multivariatestatistics, multilevel modeling, longitudinal analysis, item response theory, and testconstruction.

Fantuzzo et al.

764at UNIV PENDIDIKAN SULTAN IDRIS on June 19, 2012http://aerj.aera.netDownloaded from


4/32

able and willing to restructure their programs. As a result of reviewing theseprogram characteristics, PCER raised the question of whether preschool pro-grams that were integrated into the daily routines of early childhood pro-

grams could provide similar benefits. PCER addressed this question byusing randomized control trials (RCTs) to evaluate the efficacy of compre-hensive preschool curricula for improving the mathematics, language, andliteracy skills of older preschool children (4 to 5 years old).

The PCER evaluation focused on curricula impact in 14 preschool pro-grams. Programs were selected for evaluation if they had sufficient standard-ized training procedures and published materials to support implementationof the curriculum by research resources other than the curriculum developer.The selected programs were evaluated against the local curricula, with class-

rooms being assigned to either the experimental or the comparison localcurriculum. Each program site provided professional development and sup-port during the 1st year and an RCT during the 2nd year to test the efficacy ofa full year of implementation. Overall, 2,911 children, 315 preschool class-rooms, and 208 preschools from 16 geographical locations participated.School readiness was assessed using a battery of standardized measures inthe fall and spring of the trial year and once more a year later. Findingsfrom the evaluation indicated that no single comprehensive program pro-duced significant mathematics, language, or literacy differences when com-

pared to controls; only combinations of programs were effective. Thefindings also indicated that no single combination of programs produced sig-nificant differences in both mathematics and language or literacy outcomes.

In the PCER evaluation, only 2 of the 14 programs yielded significantmathematics, language, or literacy outcomes.1 Notably, both of these pro-grams combined all or part of the Developmental Learning Materials(DLM) Early Childhood Express curriculum (Schiller, Clements, Sarama, &Lara-Alecio, 2003) with other curricula. The first of these combinations

was the DLM with Open Court Reading (Adams et al., 2000). Here, the

impact of the combined programs was compared to the impact of the localcontrol curriculum for 297 children across 30 classrooms. The DLMOpenCourt teachers were provided 6 days of initial professional developmentand support and monthly 2-hour professional development meetings, withhalf of the teachers receiving mentoring visits. The DLMOpen Court curric-ulum produced significant improvements over controls in literacy and lan-guage but not in mathematics.

A second combination added the Pre-K Mathematics and DLM EarlyChildhood Express Math Software to a variety of standard programs, like

High/Scope (Weikart & Schweinhart, 2005) and the Creative Curriculum(Dodge, Colker, & Heroman, 2002), which were in use already. The condi-tion in which Pre-K Mathematics was implemented with DLM EarlyChildhood Express Math Software provided computer-based mathematicsactivities supplemented by 4 days of teacher professional development

Integrated Curriculum for Head Start



5/32

and support during the training year and 2 days of refresher training in thetrial year, plus twice monthly professional development and support ses-sions throughout both years. Children in the experimental condition demon-

strated significant improvements over controls in seven mathematics subskillareas. No improvements over controls were found for literacy or languageareas. The DLM (Schiller et al., 2003) is a comprehensive, research-basedcurriculum created according to state and national early childhood guide-lines. The DLM targets childrens cognitive, social-emotional, aesthetic, andphysical development through 20 thematic units that can be used individu-ally or collectively. Each unit consists of 36 weekly themes that address lan-guage and early literacy, math, science, social studies, fine arts, health andsafety, personal and physical development, and technology. Each thematic

unit includes approximately 200 hands-on learning activities that are de-signed to promote childrens social, emotional, intellectual, aesthetic, andphysical development. Randomized control trial studies have indicatedthat the DLM has been used with varying levels of success to impact themathematics, language, and literacy outcomes for older preschool children(PCER Consortium, 2008).

Findings from PCER have been supported by reviews of randomizedcontrol studies conducted by the What Works Clearinghouse, which foundno discernible effects on oral language, print knowledge, phonological

processing, or math for the Creative Curriculum when used alone. TheCreative Curriculum is a comprehensive curriculum for 3- to 5-year-oldchildren and focuses on childrens social-emotional, physical, cognitive,and language development. According to the What Works Clearinghouse,one study of the Creative Curriculum met its evidence standards, andtwo studies met its evidence standards with reservations. The studiesincluded a total of 844 children from 101 classrooms in more than 88 pre-schools in Tennessee, North Carolina, and Georgia.

No studies of High/Scope and Open Court met the What Works

Clearinghouses standards of evidence, although several experimental andquasiexperimental studies that are not randomized control trials have foundpositive effects. High/Scope is well known for its focus on language andliteracy, mathematics, science, social-emotional development, physicaldevelopment, and the arts. Its reports indicate that the curriculum has pro-duced evidence that the programs improve childrens school success, latersocioeconomic success, and social responsibility. The plan-do-reviewsequence of the program encourages children to achieve their goals throughdecision-making and problem-solving situations throughout the day. Open

Court includes eight thematic units that focus on issues from childrens iden-tity to transitions. Phonological, phonemic, and print awareness as well ascomprehension are incorporated into each session, which typically includes1.5 to 2 hours of daily instruction.

Fantuzzo et al.



6/32

The PCER studies cast considerable light on the relative efficacy ofstand-alone curriculum packages, indicating that none outperformed localbusiness-as-usual preschool curricula for mathematics, language, or literacy

outcomes. Improvement was evident only when different programs werecombined with one another or when circumscribed parts of a published pro-gram were applied on top of an existing local program. Moreover, all discov-eries pertained to 4-year-olds only, leaving unaddressed the more than onethird of preschoolers in Head Start who are between 3 and 4 years old(USDHHS, 2008a). Furthermore, combination and add-on programs arelogistically problematic. They tend to impede the formation of a clear pro-gram model definition, making it difficult to determine the footprint andimpact of the program itself. Combinations and add-ons also raise significant

questions about the feasibility (both in terms of cost and personnel) of im-plementation or replication independent of highly resourced experimentalstudies. This is a critical point inasmuch as a major goal of PCER was todetermine the efficacy of comprehensive interventions that early childhoodprograms (such as Head Start) could actually use. At least three questionsemerge: What conceptual framework should guide the design of integratedinterventions in the context of Head Start? Are programs being developedand tested within Head Start realistic for the context where they are expectedto be applied? And can those programs be implemented with the existing set

of resources?A developmental-ecological conceptual framework (Bronfenbrenner &

Morris, 1998; Zigler, Gilliam, & Jones, 2006) is a fitting model to informthe design of integrated curricula to maximize cognitive and language skillsin a Head Start context. This approach considers multiple and transactionalcompetencies as well as changes in functioning across these competenciesover time. Development is understood in terms of the central tasks that chil-dren are expected to perform as a function of their age and culture (e.g.,transitioning to school). To meet the challenges these tasks present, children

bring to bear all of their competencies across skill areas. Because develop-ment is seen as a progressive process, childrens ability to use their compe-tencies to negotiate tasks at one point of development has an impact on theirability to negotiate tasks at later points (Shonkoff & Phillips, 2000). In thisapproach, proximal context (e.g., the classroom) plays an important rolein determining the course of development. Development does not resultfrom a single influential factor within the child; rather, it is determined bymultiple simultaneous influences occurring both within and around thechild. These multiple influences, in transaction with childrens prior adapta-

tion history and current capacities, determine developmental success (Lutharet al., 2000).

McCall (2009), in a recent issue of the Society for Research in ChildDevelopments Social Policy Report, highlights the importance of developingrealistic evidence-based programming using a developmental-ecological




7/32

perspective. He asserts that we cannot just develop and test a comprehensiveprogram without realizing that context matters. He emphasizes that toenhance effectiveness and replicability of programs, we must develop and

evaluate programs for policy-relevant populations with careful considerationto the natural context. Furthermore, he suggests that being mindful of theresources required to implement an effective program and implementingand evaluating it in partnership with the existing context is essential to pro-ducing robust and sustainable outcomes, especially for children in resource-challenged contexts.

Drawing upon a developmental-ecological approach, this article reportson the development and efficacy trial for a new, stand-alone preschool cur-riculum program designed to improve mathematics, language, and literacy

among Head Start children, while being mindful of the natural context. Itis known as the Evidence-Based Program for an Integrated Curriculum(EPIC; Fantuzzo, Gadsden, & McDermott, 2003). Without combining differ-ent programs or superimposing additional curricula onto an already existingprogram, EPIC is a unified program intended to incorporate systematicallythe components of content, instruction, professional development, andrepeated criterion-based assessments. Its culminating field trial is thenextended to include all of the age levels (3 to 5 years) enrolled in HeadStart classrooms across one of the nations largest urban areas, with random-

ization to contrast it against the DLMthe one published curriculum shownby PCER to be consistently associated with any improvements over preexist-ing local curricula. Unlike PCER, EPICs RCT was able to concentrate exclu-sively on low-income children and to assess differential growth rates whiletesting for relevant covariation and interactions (see Willett, Singer, &Martin, 1998) with childrens age upon program entry, special needs, andlanguage status; prior curricular exposure; variation at the child levelbetween time intervals separating assessments; child age variation betweenclassrooms; number of classroom assistants; and variation in teachers Head

Start and career-long teaching experiences.

Method

Participants

Conducted through academic year 20072008 (AY0708), the experimen-tal trial focused on a sample of 1,415 students, comprising the enrollments of70 classes drawn randomly from the 250 Head Start classes operated by the

school district of Philadelphia, Pennsylvania. The sample included 50.2%females and 49.8% males, ranging 35 to 70 months in age (M = 50.1, SD =6.8). In contrast to PCER, 34.6% of children were younger than 4 yearsold. Approximately 12.8% were considered dual-language learners (DLLs),and 9.3% retained special needs status. Ethnicity was not systematically

Fantuzzo et al.



8/32

reported, with estimates confounded by multiple attributions and 13.9% withno attributions. For this reason, ethnicity was not used as a covariate in sub-sequent analyses. Based on reported attributions, 60.6% of children were

African American, 14.5% Latino, 4.2% Caucasian, and 6.0% other ethnicminorities. Nearly one third of children had been enrolled in those sameclasses during academic year 20062007 (AY0607), the training year that pre-ceded the trial year, whereas the other two thirds were newly enrolled forthe AY0708 trial year with no prior preschool exposure. As pertains nation-

wide to children enrolled in Head Start, children met the Head Start require-ments for admission, which requires 90% of enrolled children to be fromthose families whose incomes were below the federal poverty level or

who were eligible for public assistance (USDHHS, 2008b).

Interventions

EPIC. The development of EPIC centered on three activities: (a) buildingcurriculum modules guided by theoretical and empirical literature pointingto critical cognitive and language abilities that are indicators of early schoolsuccess and practices that promote development of these abilities, (b) inte-grating the modules through pilot research, and (c) conducting a large ran-domized control trial. The EPIC integrated curriculum was primarily focused

on areas that have been found to be critical to childrens development oflanguage and literacy: alphabet knowledge and phonemic awareness, printconcept, vocabulary, listening comprehension, and mathematics (seeMcCardle, Scarborough, & Catts, 2001; National Institute of Child Health andHuman Development, 2000; National Early Literacy Panel, 2009; Snow,1991; Snow, Burns, & Griffin, 1998), bolstered by intentional instruction in ap-proaches to learning. Mathematics development, which is reliably observedthroughout childrens first 5 years, conveys both mediating and causal effectsassociated with later mastery of cultural symbol systems and general strategic

approaches to learning (Kilpatrick, Swafford, & Findell, 2001).During the development phase of the integrated curriculum, two

sources of information were used to provide an evidence base for the scopeand sequence of EPIC: Head Starts National Indicators (USDHHS, 2006) andthe Prekindergarten Pennsylvania Learning Standards for Early Childhood(Pennsylvania Department of Education and Department of Public

Welfare, 2005). Data were collected on the Head Start population at multipletimes across the pilot years using a range of measures: the Peabody Picture

Vocabulary TestIII (PPVT-III; Dunn & Dunn, 1997), the Oral and Written

Language Scales (OWLS; Carrow-Woolfolk, 1995), the Expressive One-Word Picture Vocabulary TestRevised (Gardner, 1990), the Test of EarlyMathematics AbilityThird Edition (TEMA-3; Ginsburg & Baroody, 2003),the Preschool Child Observation Record (High/Scope EducationalResearch Foundation, 2003), and the Learning Express (McDermott et al.,




9/32

2009). Information about skill progression was used by curriculum develop-ers to detect empirically the skills recently mastered and the subskills beingnewly encountered by most children. Thereafter, curriculum contents were

sequenced in a similar fashion, such that the main foci of lessons comportedto the empirical levels at which most children were functioning. Longitudinalstudies of preschool learning behaviors provided the evidence base for theapproaches to learning component of the scope and sequence (McDermott& Fantuzzo, 2000).

With respect to context, EPIC was developed in partnership with exem-plary Head Start teachers in a large, urban public school district and was de-signed to fit within the existing expectations for the delivery of Head Startservices (i.e., in alignment with state standards and Head Start indicators).

The program was delivered by Head Start teachers as part of their routinepractice, supported by indigenous supervisory staff, and conducted withinthe Head Start programs regular allotment of professional development re-sources. For the duration of the study, EPIC was the sole intervention for theparticipating teachers rather than an add-on intervention to an existingprogram.

The EPIC program consists of its integrated curriculum practices,curriculum-based assessment, and professional training and support. TheEPIC curriculum couples tested methods of instruction with the EPIC

Scope and Sequence to provide intentional and systematic classroom expe-riences across eight units of instruction. The EPIC Scope and Sequence is anevidence-based mapping of the primary skill areas and serves as the founda-tion for the curriculum. The EPIC Scope defines subskills in each cognitiveskill area. Each curriculum unit targets a specific set of integrated instruc-tional objectives that reflect skill levels of the EPIC Scope and Sequence.The objectives serve as the basis for the unit activities, resulting in intentionalinstruction of targeted skills along an evidence-based developmentalsequence.

The EPIC curriculum incorporates evidence-based best practices derivedfrom studies that had tested separately curriculum methods for advancingearly language and literacy (Wasik & Bond, 2001) and mathematics (Frye,1991) skills with Head Start children. These instructional methods were builtinto the daily classroom routine. Routine experiences include interactivereading, large-group and small-group activities, transition activities, environ-mental changes, and home connections. Interactive reading involves read-ing storybooks and dialoguing with children about the books using key

vocabulary. Each unit of the curriculum contains key books chosen for their

ability to support targeted skills and expand, connect, and enhance conceptdevelopment. Large-groupEPIC experiences foster a sense of belonging andencourage childrens interest and motivation for learning. Teachers dialogue

with children to introduce key concepts and vocabulary and to engage inactivities that apply concepts in practice. Small-group activities provide

Fantuzzo et al.



10/32

opportunities for introducing, teaching, and practicing skills, and supporta more focused observation of children to adapt instruction to the individualskill levels of each student. Transition activitiesare designed to support con-

tinuous learning during the short periods of time in which students movefrom one structured group activity to another. EPIC environmental changesestablish the environment as the third teacher by using specially designedstations, bulletin board displays, and innovative props and visual cues toreinforce key skills and vocabulary for children, parents, and teachers.Classroom areas are modified, as needed, to reinforce targeted conceptsand vocabulary featured in each unit. The EPIC curriculum also capitalizeson the role that families play as home educators. EPIC home connectionsare weekly home learning activities that parallel classroom learning experi-

ences. They provide family members with concrete opportunities to rein-force childrens learning key vocabulary and concepts. They weredeveloped to foster ongoing, two-way exchanges with family membersabout students skill development and to celebrate families contributionsto student achievement.

Additionally, a distinctive component of the EPIC intervention wasevidence-based approaches to learning modules designed to enhance math-ematics, language, and literacy skills development (McDermott & Fantuzzo,2000; Shure & DiGeronimo, 1996). During the EPIC development and pilot-

ing phase, four learning behavior modules were integrated into the cognitiveskills scope and sequence: Attention Control, Frustration Tolerance, GroupLearning, and Task Approach. Attention Control emphasizes developingskills related to focusing attention and completing tasks (i.e., persistenceand effectively dealing with distractions). Frustration Tolerance is designedto help children recognize and verbalize frustration and use effective strate-gies to deal with frustration (e.g., taking a break, asking for help, and prac-ticing with assistance). Group Learning targets cooperative learning skills(e.g., taking turns, helping others, contributing to group activities and com-

pleting activities with others). Task Approach focuses on skills related togenerating new ideas and trying out these different ideas to solve problems(e.g., brainstorming, creating alternative solutions, and testing them) andcarry out a simple plan.

EPIC also includes curriculum-based assessments that are used by teach-ers to identify the individual competencies and learning needs of each child.EPIC Integrated Check-Ins (ICIs) are brief assessments of skill levels acrossthe integrated scope and sequence of the curriculum. These skills directlymap onto state standards for early childhood education and the national

Head Start indicators. They include alphabet knowledge, phonemic aware-ness, vocabulary, print concepts, listening comprehension, mathematics,motor, social-emotional, and approaches-to-learning skills. Each skill isassessed across a developmental sequence of five levels that have been es-tablished and validated by empirical research (Fantuzzo, Gadsden, &




11/32

McDermott, 2008). ICIs are completed for each child by a teacher as part ofthe routine implementation of the curriculum. These formative assessmentsare embedded in units as standardized curriculum activities that are repeated

three times throughout the year. ICIs help teachers monitor childrens prog-ress and create a classroom profile of individual student differences in abilitylevels to inform instruction.

Another critical element of the EPIC intervention is the means by whichteachers and teaching assistants receive professional development andongoing support as they use the ICIs and implement the integrated curricu-lum. EPIC uses a learning community model of professional developmentbased on distributed leadership principles (Spillane, 2006). This model seeksto build reciprocal teaching and learning relationships among educators

who have different levels of expertise and experience with the EPIC curric-ulum. Productive learning relationships are established within the classroomand across classrooms by having experienced educators share effectiveclassroom strategies as they implement the curriculum. The EPIC learningcommunity meets routinely throughout the year in three different learningcontexts: teaching teams, small groups, and large group. A teaching team in-cludes a classroom teacher and teaching assistant. Teaching teams meet ona weekly basis to review childrens response to the curriculum activities fromthe previous week and plan for their coordinated implementation for the

upcoming week. Small groups meet prior to the onset of each curriculumunit and consist of five to six teaching teams and a mentor teacher. The men-tor teachers are experienced EPIC teachers who are trained to help theirpeers implement EPIC while they are concurrently implementing EPIC intheir own classroom. In small-group learning community meetings, teamsshare their experiences with the previous unit and are introduced to thenew unit. Mentor teachers are in regular weekly communication with theirteams and the Head Start educational coordinator. The educational coordi-nator is given special support to gain expertise in the EPIC curriculum and

is assigned to visit classrooms and engage in onsite support. Large-grouplearning community meetings occur quarterly and provide all involved

with an opportunity to discuss implementation issues and share bestpractices.

In our comparison classrooms for the EPIC study, teachers in the DLMgroup used the Preschool Child Observation Record (High/ScopeEducational Research Foundation, 2003) to conduct individual assessmentsof children and monitor their progress. A comparable number of assess-ments was conducted in both conditions. Allocation of indigenous staff to

support curriculum implementation was comparable for EPIC and DLMclassrooms. Both had access to mentor teachers and educational coordina-tors and the same amount of professional development days for meetings.The difference was the model of professional development and the curricu-lum and assessment content. DLM teachers received professional

Fantuzzo et al.



12/32

development in the form of didactic workshops. Educational coordinatorspresented material on specific topics or introduced new regulations togroups of teachers and teaching assistant in their regions. Teachers were

provided opportunities to ask questions or discuss content.

Intervention implementation. At the end of each unit of the EPIC inte-grated curriculum, teaching teams reported the degree to which they imple-mented each routine component of the curriculum: interactive reading,large-group activities, small-group activities, environmental changes, andtransition activities. On average, they reported completing 97% (SD = .02)of interactive reading activities, 89% (SD = .02) of large-group activities,86% (SD = .05) of small-group activities, and 80% (SD = .04) of transition

activities. The overall implementation across the year was 88% (SD = .04).These ratings matched supervisor reports of implementation. In addition,teachers and teaching assistants were asked to provide anonymous ratingsof their overall satisfaction with the EPIC program on a 4-point Likert-typescale: not satisfied, somewhat satisfied, satisfied, or very satisfied. The aver-age rating across teachers and teaching assistants of being satisfied or verysatisfied with EPIC across the year was 98% (SD= .02). There was no signif-icant difference between teacher and teaching assistant independent ratings.

Data on the fidelity of implementation of EPIC and DLM across the year

were collected by the educational coordinators of the Head Start program aspart of their routine supervision of classrooms. Educational coordinators, atthe beginning of each year, received preparation to assess the fidelity of cur-riculum implementation. This preparation included how to collect teacherslesson plans and observe their classrooms at regular intervals throughout the

year. They were instructed to note where the teacher was in the sequence ofthe curriculum and whether there was a general correspondence betweenthe plan and the observed activity. A 5-point overall rating of implementa-tion was used: 1 = very poor, 2 = poor, 3 = fair, 4 = well, and 5 = very well.

Ratings were collected at three points in time in accordance with programsevaluation of student progress and parenting reporting. The overall ratingfor both EPIC and DLM was rated well. For EPIC classrooms, the median rat-ing was 4, with a range of 3 to 5. The median for the DLM classrooms was 4also, with a range of 3 to 5. There were no significant differences in super-

visor ratings across programs.

Instrumentation

The central focus of this study was the relative effectiveness of curricula

over the period of an RCT. This required assessments of child cognitivegrowth at multiple time points that would allow estimation of stable growthtrajectories rather than simple comparisons of group means at the close ofthe study. The multiple assessments were intended to inform the growthrates throughout a school year and also to fortify against the complete loss




13/32

of outcomes information associated with participants who attend part butnot all of the year. Pilot studies (McDermott, Angelo, Waterman, & Gross,2006) had clarified that the best norm-referenced tests (NRTs; e.g., the Test

of Early Reading Ability, 3rd ed. [TERA-3]; Reid, Hresko, & Hammill, 2001;and the OWLS; Carrow-Woolfolk, 1995) were poorly suited to the taskbecause the average growth ranges for Head Start children were essentiallytrivial (four to five additional items correctly answered over the school year),reducing the sensitivity of those tests for a conventional pretest-posttestdesign and effectively vitiating any growth sensitivity over briefer intervals.These results followed from the fact that for requisite nomothetic and com-mercial purposes, popular NRTs feature item content that is properly cen-tered around the 50th percentile in difficulty, whereas the nations Head

Start population is relatively challenged, with performance centering aroundthe 15th to 20th percentile (U.S. Department of Education, 2007), leavingmarkedly few items appropriate for Head Start children. In this respect, it

was deemed imperative that any demonstration of effectiveness for a curric-ulum should be based on a wide-scoped definition of the component sub-skills that comprised any content area rather than a set of narrowlycircumscribed subskills that would not be representative of a larger cognitivedomain. Nor would experimental results be generalizable were tested sub-skills to be chosen as a function of their tendency to inflate the apparent use-

fulness of a given curriculum. Within this context, we constructed andvalidated a set of measures that were highly sensitive to growth over briefintervals, that broadly represented the target cognitive domains, and thatcould be administered repeatedly over the school year with minimal timeinvestment and maximal precision.

Performance was assessed through the Learning Express (LE;McDermott et al., 2009), an individually administered adaptive battery ref-erenced to Head Starts National Indicators (USDHHS, 2006) andPrekindergarten Pennsylvania Learning Standards for Early Childhood

(Pennsylvania Department of Education and Department of PublicWelfare, 2005). Although all LE content is unique and not identical toany government or commercial tests, content breadth was further demon-strated by the alignment of each item to the content of Head Starts NationalReporting System (USDHHS, 2003), the TERA-3 (Reid et al., 2001), thePPVT-III (Dunn & Dunn, 1997), the OWLS (Carrow-Woolfolk, 1995), theExpressive One-Word Picture Vocabulary TestRevised (Gardner, 1990),the TEMA-3 (Ginsburg & Baroody, 2003), the Preschool ChildObservation Record (High/Scope Educational Research Foundation,

2003), and the Galileo Skills Inventory (Version 2; AssessmentTechnology, 2002). The LE contains 325 items distributed over two equatedforms (to reduce practice effects) and four subscales (AlphabetKnowledge, Vocabulary, Listening Comprehension, and Mathematics).

Fantuzzo et al.



14/32

A total of 56 distinct subskills are featured, with each subscale incorpo-rating multiple subskills representing varied complexity and breadth andrequiring varied response modes for children (i.e., pointing, vocal expres-

sion, and object manipulation). Subscales are calibrated via two-parameterlogistic item response models with scaled scores (M= 200, SD= 50) derivedthrough Bayesian ex a posteriori estimation. Basal adaptive testing is appliedto ensure administration of all subscales within 20 to 30 minutes.Dimensionality was confirmed through full-information bifactor analysis

with all subscales ..90 in reliability and producing information curves de-signed for precision longitudinal measurements across equated forms(McDermott et al., 2009).

Concurrent validity was supported through relationships with NRTs and

teachers assessments of literacy and numeracy (McDermott et al., 2009).Based on May and June 2005 assessments, each LE subscale was significantlyand more highly correlated with its hypothetical counterpart NRT scale.Specifically, LE Alphabet Knowledge correlated .68 with TERA-3, LE

Vocabulary correlated .69 with PPVT-III, LE Listening Comprehension .63with OWLS, and LE Mathematics .69 with TEMA-3. Similarly, spring 2007LE Alphabet Knowledge, Vocabulary, and Listening Comprehension scores,respectively, correlated .62, .56, and .52 with teachers concurrent ChildObservation Record (COR) Language and Literacy scale scores, and LE

Mathematics correlated .63 with COR Mathematics scores. Predictive validitywas assessed through correlation of October 2006 LE Alphabet Knowledge,Vocabulary, and Listening Comprehension scores, respectively, with thecounterpart COR Language and Literacy scores from late spring 2007 (rs =.57, .56, and .52) and LE Mathematics with COR Mathematics (r = .58).Note also that as revealed in the content validity analyses of LE items as com-pared to items in those various external criterion measures (see McDermottet al., 2009), the LE subscales cover a markedly broader array of skills thanthe NRT devices and cover many skills prescribed by the national Head Start

Indicators (USDHHS, 2006) not covered through NRTs.

Procedures

The Learning Express battery was administered to each individual childby a trained assessor during a single session ordinarily taking 20 minutes butno more than 30 minutes. Private locations were identified in each HeadStart center for individual testing. Children were escorted to testing in theorder of the class list, with no more than 5 children removed for testing

simultaneously and always with the teachers knowledge. Standardizedquestions inquired as to each childs status in terms of special needs,English as primary or secondary language, and health at the time of testingand the teachers discretion as to whether testing was advisable. A flipbookbinder of item stimuli was placed on a table and oriented toward the child.




15/32

As each successive item was exposed to the child, the assessor asked a ques-tion (which appears in print on the reverse side of the item page facing theassessor) requiring the child to point to the correct choice, vocally express

the answer, or manipulate objects. A standardized prompt was also availablefor non sequitur child responses or no response. Assessments were con-ducted at four points in time throughout the training and trial years:October, January, March, and June.

The Learning Express has equivalent alternate forms. Items were de-signed in pairs whose members were intended to reflect comparable contentand equal difficulty, with one member of a pair assigned to Form A and theother to Form B. For all item response theory (IRT) equating studies, equiv-alent-groups equating with linking items was applied, where forms were of

equal length and multiple-groups calibration was used with children ran-domly assigned to forms. Equating accuracy was tested through comparisonof uniformity of all four moments of the distributions (means, variances,skewness, kurtosis) across forms via Kolmogorovs D (Conover, 1999) ateach wave. Equating was deemed successful should the distribution ofscaled scores across forms remain identical after equating (per Kolen &Brennan, 2004). Thus, Kolmogorovs D estimated the similarity of the twoform distributions for each subscale at each wave, and the Kolmogorov-Smirnov goodness-of-fit index tested the probability that D was greater

than the observed value under the null hypothesis of no difference betweenforms (Conover, 1999). For every subscale and at every wave, the score dis-tributions for the equated forms were essentially equivalent, yielding verysmall D values (M = .06, SD = .02, range = .03 to .09).

To minimize practice effects over repeated waves of assessment withina school year, the two forms were applied in a counterbalanced fashion.Children appearing as odd numbers on a class list were administeredForm A at Wave 1, whereas those appearing as even numbers were admin-istered Form B. Administration was reversed for each subsequent wave such

that, for example, approximately half of the children during AY0607 receivedform sequence ABAB and half BABA. Each year, the order of classrooms tobe assessed was random for Wave 1, and from wave to wave, there was aneffort to maintain the same approximate order for assessing each child (e.g.,a given child assessed at the start of Wave 1 was likely to be assessed at thestart of other waves). This process served to minimize disparities betweenchildren in the time intervals separating their assessments (although timemeasures were kept to correct for any such disparities in subsequent individ-ual growth modeling).

Child assessments were conducted at each classroom by an independentteam of 45 trained assessors in AY0607 and 38 trained assessors in AY0708.

Assessors consisted of undergraduate- or graduate-level students who wererecruited at the beginning of each academic year through e-mail to psychol-ogy and education departments in the greater Philadelphia region. Ages

Fantuzzo et al.



16/32

ranged from 18 to approximately 60 years (median ages in the mid- to late20s), with more than 40% being ethnic minorities (primarily African

American) and nearly 20% males. Thirty-five hours of professional develop-

ment and support were provided during the training and trial years in earlySeptember, followed by 15 to 20 hours practicing administrations. Theassessment team was employed to evaluate Head Start childrens growth.They did not work with and were not informed about the EPIC implemen-tation activity. Their assessment results were not shared with those directingthe implementation until after the trial was completed.

Data Analysis

Multiple sets of multilevel individual growth-curve models were con-structed for each content area over the trial year. The temporal variabilityof performance within children across the four assessment waves comprisedLevel 1 in these models, whereas performance variation between children

within classrooms was held at Level 2 and variation between classrooms atLevel 3. Because there was variability among testing dates within any given

wave, time in days was applied as the principal time-varying measure (themetameter) and zero-centered around the mean testing day during Wave4, May to June 2008 (final status). The final status metameter was appropri-

ate because the contrasts for the effects of primary interest (those involvingEPIC vs. DLM) were necessarily focused on performance at the close of thetrial year. Random intercepts and slopes were estimated and tested for eachmodel at both the child level (Level 2) and classroom level (Level 3).

The initial model for each content area was an unconditional type enter-ing only the metameter with classroom means as outcomes in order todecompose hierarchically the unexplained content variance within andbetween children and between classrooms. Given the disparate time inter-

vals separating waves and distinguishing the timing between any given

childs assessments, the unconditional models included those assuming com-pound symmetry for the within-child covariance matrices as well as modelspositing a spatial power law (essentially a first-order autoregressive modelsensitive to the differential intervals), a Gaussian spatial model, a sphericalspatial model, and a completely generalized (unstructured) model(Wolfinger, 1993, 1996). Models were compared through Akaikes informa-tion criterion (Burnham & Anderson, 1998, 2004) and chi-square deviancetests (Littell, Milliken, Stroup, Wolfinger, & Schabenberger, 2006) to identifythe best error covariance structure for each content area. Additionally, linear,

quadratic, and cubic fixed effects and higher-order random slopes weretested for each content area.

Subsequent conditional models sequentially added and tested covariatesand interactions under full maximum-likelihood constraints, and the finalmodel for each content area featured only statistically significant random




17/32

and fixed effects that were extracted through restricted maximum-likelihoodestimation (per Littell, Milliken, Stroup, & Wolfinger, 1996, and Milliken &

Johnson, 2002). As an exception to the latter, the main effect for EPIC versus

DLM was retained for each model whether significant or not as required toprovide full information on the relative performance of the curricula. Posthoc comparisons of outcomes were based on Tukey-Kramer (Searle,Speed, & Milliken, 1980) contrasts of least-squares means (means adjustedfor any group imbalance), and where significant fixed effects were discov-ered concerning EPIC versus DLM performance, effect sizes (Cohens D)

were estimated as per the least-squares means and standard deviations forfinal status (Wave 4) score estimates.

Power analyses (per Raudenbush, 1997, and Raudenbush & Liu, 2001)

indicated that assuming statistical significance at p\ .05, power = .80, intra-class r\ .10, and 20% attrition over 2 years, the initial sample was sufficientto detect small to moderate experimental effects. The covariates assessed foreach model included, at the child level, age in months at entry into the study(opening of school, 2007), gender (female vs. male), participation in thetraining year (vs. none), DLL status (vs. none), and special needs status(vs. none) and, at the classroom level, EPIC (vs. DLM curriculum), meanage for the childs classroom, teachers total years teaching experience,teachers total years teaching Head Start, and the number of adult classroom

volunteers. Age at entry was selected as the age covariate rather than incre-menting age over the trial year due to the substantial collinearity of the latter

with the time metameter, and any additional variable reflecting a childs totalyears in Head Start or other preschool experience was excluded because ofits essential collinearity with the participation in training year covariate.

Results

Given 80 classrooms randomly assigned to either the EPIC or DLM pro-

gram condition in fall AY0607, 70 (35 DLM, 35 EPIC) remained at the start ofthe AY0708 trial year and through the duration of the study (attrition rate =12.50%), while child attrition over the trial year averaged 8.24% across thefour content areas, with no appreciable departure for any particular area.These attrition rates did not approach the potential 20% attrition over 2 yearsthat was projected in prior power analyses. The classroom attrition occurringduring the training year was never due to teachers declining participationbut rather to classroom closings given low enrollment, school closings, orteacher transfers and retirements. The more complex error covariance struc-

tures afforded no statistically significant improvement in model fit over mod-els assuming compound symmetry or random covariance structures; thus,the models assuming compound symmetry were adopted. Moreover,higher-order fixed and random effects uniformly were nonsignificant

Fantuzzo et al.



18/32

statistically and frequently produced boundary cases, whereas linear modelsuniformly yielded significant effects and optimal model fit.

Table 1 shows the hierarchical decomposition of variance for each

unconditional model (exact model specifications are reported in the foot-note). Across content areas, the preponderance of potentially explainable

variance was that between children (M = 65.90%, SD = 10.46) with morethan 70% of Vocabulary and Mathematics and just over 50% of ListeningComprehension variability being between individual children. Relativelysmall amounts of variability (for general comparisons, see Raudenbush,Martinez, & Spybrook, 2007; Snijders, 2005) were detected between class-rooms (M = 5.54%, SD = 1.87), the percentage never exceeding 7%. This re-flects more substantial homogeneity among the classrooms as might be

expected for the Head Start population. Based on the number of classrooms,harmonic M classroom enrollment (20.04), and average intraclass r (.056),the experimental design effect (Snijders, 2005) was 2.12.

As noted, an array of child- and classroom-level covariates was tested.The final models incorporate only those statistically significant and eitherstructurally fundamental to the experimental design or for which markeddisparities were evident across treatment conditions at Wave 1 of the trial

year. Inasmuch as the largest portion of variability in young childrens cog-nitive performance is generally accounted for by age (M= 25.19%, SD= 4.76,

in this study; refer to McDermott, 1995, for population estimates), age atentry was included in each conditional model. The binary indicator of child-rens training year inclusion also was included to identify trial participantsenrolled in treatment classrooms through the training year (30.00% of chil-dren). Given randomization at the classroom level in AY0607, equitable dis-tribution of child characteristics across EPIC and DLM curricula could not beassumed. At the opening of the trial year, it was found that 64.09% of DLLchildren were enrolled in EPIC classes and 56.15% of children with specialneeds in DLM classes. Such disparities were apparent as well at the opening

of the training year. Thus, DLL and special needs status were applied as co-variates in all conditional models. Sequencing of covariate entry and treatingof interaction effects proceeded as recommended by Bauer and Curran(2006), Singer and Willett (2003), and Willett et al. (1998).

Table 1 posts parameter estimates and significance levels for the condi-tional models incorporating all statistically significant random and fixed ef-fects and for the effect of interest (EPIC vs. DLM) whether significant ornot (full model specifications appear in the table note). Every model evinceda significant growth rate for LE scores (the time in days), the largest being

0.24 points increment in Mathematics scores per day (or 7.24 per month)and the smallest 0.15 points growth in Listening Comprehension scoresper day (4.54 per month).

Among the various control covariates, a somewhat general pattern wasevident wherewith on average DLL children underperformed non-DLL




19/32

Table 1

Multilevel Individual Growth Curve Analyses for Cognitive Areas Over

Cluster-Randomized Trial Year

Cognitive Area

Effect

Alphabet

Knowledgea VocabularybListening

Comprehensionc Mathematicsd

Unconditional Means Models

Random effects (parameter

estimates)

Between children within

classrooms

Intercepts 1517.27****

1522.69****

890.93****

1629.73****

Covariance 0.51 1.63**** 1.45**** 0.14

Slopes 0.01**** 0.01**** \0.01* 0.01****

Between classrooms

Intercepts 158.49*** 127.72*** 47.75** 142.31***

Covariance 0.38** 0.15

Slopes \0.01** \0.01**

Residuals 669.24**** 483.08**** 780.77**** 378.72****

Random effects (% variance in

hierarchical decomposition)

Temporal (Level 1) 28.54 22.64 45.45 17.61

Between children (Level 2) 64.70 71.37 51.77 75.77

Between classrooms (Level 3) 6.76 5.99 2.78 6.62

Fixed effects (parameter estimates)

Intercepts 229.19**** 225.79**** 219.79**** 228.46****

Time in days 0.22**** 0.19**** 0.14**** 0.23****

Intercepts- and Slopes-as-Outcomes Models

Random effects (parameter

estimates)

Between children within

classrooms

Intercepts 1127.14****

994.50****

628.82****

1064.20****

Covariance 0.97*** 0.43* 0.83****

Slopes 0.01**** 0.01**** 0.01****

Between classrooms

Intercepts 156.22**** 75.64**** 24.80* 133.01****

Covariance 0.40** 0.22*

Slopes 0.00** \0.00**

Residuals 669.28**** 485.57**** 799.12**** 381.43****

Fixed effects (parameter

estimates)

Intercept 233.00**** 231.52**** 225.39**** 235.05****

Time in days 0.22**** 0.19**** 0.15**** 0.24****

Age (in months) at entry 2.34**** 2.15**** 2.12**** 2.90****Time 3 Age \0.01* 0.01**** 0.01**** 0.00****

Training Year Inclusion 10.36**** 9.07*** 15.13**** 17.50****

Age 3 Training Year Inclusion 0.94* 1.12*

(continued)

Fantuzzo et al.



20/32

children, the most noticeable being a full standard deviation gap inVocabulary (M= 31.52 points over all content areas), and special needs chil-

dren underperformed nonspecial needs children by approximately half ofa standard deviation (M = 24.20 points over content areas). On the otherhand, participation in the training year translated to a distinct advantagefor children (Mscore increment = 13.01 points), with the greatest advantagemanifest for Mathematics achievement (17.50 points) and Listening

Table 1 (continued)

Cognitive Area

Effect

Alphabet

Knowledgea VocabularybListening

Comprehensionc Mathematicsd

Intercepts- and Slopes-as-Outcomes Models

Dual-language learner 19.22**** 50.78**** 31.92**** 24.16****

Special needs status 23.17**** 20.89**** 23.14**** 29.62****

EPIC (vs. DLM) curriculum 4.08 0.46 5.29* 9.04*

Time 3 EPIC curriculum 0.03* 0.02*

Note. Models assess cognitive growth over four time points via restricted maximum likeli-

hood estimation assuming compound symmetric error covariance structures. Effects notappearing in models were excluded for failure to achieve statistical significance. EPIC =Evidence-Based Program for Integrated Curricula; DLM = Developmental LearningMaterials Early Childhood Express.aWave 1, N = 1,208; Wave 4, N = 1,106; total number of observations = 4,759.Unconditional model: Alphabet KnowledgeYijk = g000 1 g100Timeijk 1 (m00k 1m10kTimeujk) 1 (m0jk 1 m1jkTimeijk) 1 rijk. Conditional model: Alphabet KnowledgeYijk= g000 1 g100Timeijk1 g010EntryAgej1 (g110Timeijk * EntryAgej) 1 g020TrainingYearj1g030DLLj 1 g040SpecialNeedsj 1 g001EPICk 1 (m00k 1 m10kTimeijk) 1 (m0jk 1m1jkTimeijk) 1 rijk.bWave 1, N = 1,207; Wave 4, N = 1,104; total number of observations = 4,763.

Unconditional model: VocabularyYijk = g000 1 g100Timeijk 1 (m00k 1 m10kTimeujk) 1(m0jk) 1 rijk. Conditional model: VocabularyYijk = g000 1 g100Timeijk 1 g010EntryAgej 1(g110Timeijk * EntryAgej) 1 g020TrainingYearj 1 g030DLLj 1 g040SpecialNeedsj 1g001EPICk1 (m00k1 m10kTimeijk) 1 (m0jk) 1 rijk.cWave 1, N = 1,209; Wave 4, N = 1,107; total number of observations = 4,701.Unconditional model: Listening ComprehensionYijk = g000 1 g100Timeijk 1 (m00k 1m10kTimeujk) 1 (m0jk) 1 rijk. Conditional model: Listening ComprehensionYijk = g000 1g100Timeijk 1 g010EntryAgej 1 (g110Timeijk * EntryAgej) 1 g020TrainingYearj 1(g030EntryAgej * TrainingYearj) 1 g040DLLj 1 g050SpecialNeedsj 1 g001EPICk 1(g101Timeijk * EPICk) 1 (m00k) 1 (m0jk) 1 rijk.dWave 1, N = 1,208; Wave 4, N = 1,106; total number of observations = 4,761.

Unconditional model: MathematicsYijk = g000 1 g100Timeijk 1 (m00k 1 m10kTimeujk) 1(m0jk 1 m1jkTimeijk) 1 rijk. Conditional model: MathematicsYijk = g000 1 g100Timeijk 1g010EntryAgej 1 (g110Timeijk * EntryAgej) 1 g020TrainingYearj 1 (g030EntryAgej *TrainingYearj) 1 g040DLLj 1 g050SpecialNeedsj 1 g001EPICk 1 (g101Timeijk * EPICk) 1(m00k1 m10kTimeijk) 1 (m0jk1 m1jkTimeijk) 1 rijk.*p\ .05. **p\ .01. ***p\ .001. ****p\ .000.




21/32

Comprehension (15.12 points). Also, the significant interaction between timeand age for every model echoed the tendency of slopes to be steeper forchildren who were younger at entry and flatter for those who were older,

as did the interaction between age and training-year participation reflectthe flatter slopes for Listening Comprehension and Mathematics when chil-dren were jointly older to begin with and involved in the training year. Suchslope distinctions in cognitive growth among young children, where the

youngest children manifest markedly steep slopes that slowly flatten asthey approach age 5 and beyond, is a well-known phenomenon in earlychildhood development (Shonkoff & Phillips, 2000). No significant distinc-tions were found among classrooms in terms of variability associated withteaching experience or numbers of adult volunteers.

Statistically significant main effects were found for the superiority of theEPIC curriculum over the DLM curriculum at the conclusion of the experimentand for the diverging slopes for those curricula over time, in favor of the EPICcurriculum. For Listening Comprehension, the main effect was F(1, 67) = 5.20,

p = .0258, and interaction was F(1, 4629) = 6.53, p = .0106. The model incor-porated significant random effects for intercept variation at both child andclassroom levels but no statistically consequential slope effects were detectedat either level. Although the significant interaction effect accounted for 77.3%of the variation in between-groups slope variance, it did so in a context where

the overall available between-groups slope variance was itself trivial. Thus theinteraction should not be interpreted to indicate differential growth rates atconclusion of the trial year, whereas the simple main effect for EPIC (vs.DLM) curriculum clearly points to EPIC superiority at years end. With thegrand mean at final status (the middle of Wave 4) at 222.85 scaled score points(the fixed effects intercept for the Listening Comprehension model in Table 1),the EPIC M = 223.89 and the DLM M = 218.68, a difference of 5.21 points.Given that these least-squares means correct for any curricular-group sizeimbalance, the overall Wave 4 estimated SD(31.03) was applied to show effect

size = .17.2 EPIC and DLM estimated means for the middle of Wave 1 (222 dayspreceding final status) were not statistically discrepant (p = .7827), indicatingthat the curricular groups were essentially equivalent 1 month into the trial

year, with neither curriculum having an apparent starting advantage.The results for Mathematics were more pronounced. Specifically, the

main effect for EPIC versus DLM was F(1, 67) = 6.85, p = .0110, and interac-tion was F(1, 4619) = 4.19, p = .0407. Whereas the grand mean at final status

was 235.05 points, the EPIC M= 234.76 and DLM M= 225.72, a difference of9.04 points (estimated SD = 40.94, effect size = .22). EPIC and DLM perfor-

mance discrepancy was not statistically significant for the first wave of thetrial year (October 2007, p= .2236), nor was it significant 1 month after clus-ter randomization in the preceding training year (October 2006, p = .2596),indicating no advantage for either curriculum 1 month into the experiment.Superiority of the EPIC curriculum was actually detectable by Wave 2,

Fantuzzo et al.



22/32

January 2008 (p = .0400), with the EPIC growth trajectory continuing to pullaway from the DLM trajectory over the trial year.

Table 2 presents for Listening Comprehension and Mathematics the break-

down of least-squares means and their standard errors for the main DLM versusEPIC effect and across DLM and EPIC within covariates over trial waves. Thecovariates are those yielding significant main effects as reported in Table 1.Comparison of DLM and EPIC means for a given covariate (e.g., DLL vs.non-DLL) at a given time point (e.g., Wave 4) indicate relative curricular perfor-mance. Because age (in months) at entry was a continuous variable in the mod-els, Table 2 partitions that variable for convenient interpretation into groups ofchildren who were younger than versus older than 4 years of age at initialenrollment into a given curriculum. The listed means are corrected for imbal-

ance in cell sizes (although actual cell sizes are posted), and they reflect the bet-ter performance of older children, those participating in the training year, andEPIC versus DLM, even among DLL children and those having special needs.

Discussion

The primary purpose of this study was to examine the efficacy of EPIC asa stand-alone, comprehensive program for improving cognitive school readi-ness outcomes for children from low-income households in the context of

urban Head Start centers. Designed to meet state and federal requirementsfor preschool programs, EPIC was developed in partnership with exemplaryHead Start educators and was designed to meet state and federal requirementsfor preschool programs. EPIC was evaluated against the DLM, with effectsbeing tested across important subgroups of children. DLM was a relevant com-parison program since it was associated with the best cognitive effects for pre-school children in the PCER studies (PCER Consortium, 2008). EPIC and DLM

were implemented with comparable fidelity and resources.Analyses revealed main effects for a comprehensive set of mathematics

and listening comprehension skills, with children in the EPIC program per-forming better than children in the DLM program, controlling for age, priorpreschool experience, and special needs and language status. Additionally,interactions between type of program subgroups (i.e., 3-year-old children,DLL, and children with special needs) were tested. There were no significantdifferences between the programs in Vocabulary and Alphabet Knowledge.Irrespective of program, a pattern of underperformance was found for DLLchildren and children with special needs compared to their non-DLL andnonspecial needs counterparts. Moreover, a significant interaction between

time and age for every model indicated that younger children evidencedsteeper slopes than older children. Overall, the EPIC findings are notable

when compared with the 14 PCER studies: No single program showed pos-itive outcomes for mathematics, language, or literacy, and only two foundany positive cognitive outcomes. Neither of the two employed a single




23/32

Table 2

Estimated Least-Squares Means (and Standard Errors) by

Curriculum Within Samples Over the Trial Year

Wave 1 Wave 2 Wave 3 Wave 4

Sample DLM EPIC DLM EPIC DLM EPIC DLM EPIC

Listening

Comprehension

Full sample 190.67 190.16 203.73 205.95 211.16 214.92 218.52 223.81

(1.75) (1.75) (1.53) (1.53) (1.56) (1.56) (1.65) (1.71)

n 608 601 598 605 583 599 544 563

3-year-olds at entry 165.87 165.22 184.03 186.12 194.34 198.00 204.56 209.77

(2.35) (2.40) (2.06) (2.12) (2.10) (2.16) (2.27) (2.34)

n 204 183 212 194 211 196 202 1884- and 5-year-olds at entry 203.66 203.01 214.11 216.20 220.04 223.70 225.93 231.14

(1.82) (1.78) (1.56) (1.53) (1.59) (1.57) (1.76) (1.73)

n 404 418 386 411 372 403 342 375

Training year inclusion 201.28 200.63 214.34 216.44 221.77 225.43 229.13 234.34

(2.67) (2.69) (2.53) (2.56) (2.55) (2.59) (2.64) (2.68)

n 202 205 178 186 173 184 162 167

No training year inclusion 186.21 185.56 199.27 201.37 206.69 210.35 214.05 219.26

(1.78) (1.76) (1.55) (1.54) (1.58) (1.57) (1.72) (1.71)

n 514 494 420 419 410 415 382 396

Dual-language learner 163.13 162.48 176.19 178.29 183.62 187.28 190.98 196.19

(2.92) (2.75) (2.79) (2.61) (2.81) (2.63) (2.89) (2.71)n 56 103 56 109 45 538 47 497

Nondual language learner 195.12 194.47 208.19 210.29 215.61 219.27 222.97 228.18

(1.76) (1.80) (1.56) (1.59) (1.56) (1.63) (1.71) (1.77)

n 552 498 542 496 109 490 106 457

Special needs status 170.80 170.15 183.87 185.96 191.29 194.95 198.65 203.86

(3.08) (3.14) (2.96) (3.03) (2.98) (3.04) (3.05) (3.12)

n 61 47 64 47 60 48 55 47

Nonspecial needs status 192.84 192.19 205.91 208.00 213.33 216.99 220.69 225.90

(1.78) (1.76) (1.56) (1.55) (1.58) (1.58) (1.72) (1.73)

n 547 554 534 558 523 551 489 516

Mathematics

Full sample 177.37 181.10 200.05 206.27 212.94 220.57 225.72 234.76(2.24) (2.23) (2.19) (2.17) (2.31) (2.31) (2.52) (2.51)

n 607 601 597 604 583 598 543 563

3-year-olds at entry 149.81 153.54 176.05 182.27 190.96 198.59 205.74 214.77

(2.80) (2.86) (2.70) (2.75) (2.80) (2.85) (3.02) (3.07)

n 203 183 211 193 212 194 201 189

4- and 5-year-olds at entry 191.48 195.21 212.34 218.56 224.20 231.83 235.95 244.99

(2.29) (2.24) (2.22) (2.18) (2.34) (2.30) (2.57) (2.52)

n 404 418 386 411 371 404 342 374

Training year inclusion 189.41 193.14 212.10 218.32 224.98 232.62 237.76 246.80

(3.20) (3.23) (3.17) (3.20) (3.26) (3.28) (3.41) (3.43)

n 196 196 179 186 172 185 162 166No training year inclusion 172.03 175.76 194.71 200.93 207.60 215.24 220.38 229.42

(2.26) (2.24) (2.21) (2.19) (2.33) (2.31) (2.54) (2.52)

n 411 405 418 418 411 413 381 397

(continued)

Fantuzzo et al.



24/32

comprehensive program but implemented various combinations of pro-grams or add-ons of some or all of the DLM program.

Positive mathematics and language findings in the present study reflectsome distinctive features of the RCT over PCER. First, in this study, EPICand DLM were implemented as stand-alone programs with comparable re-sources and fidelity. Second, the efficacy of EPIC was evaluated across allof the children in the Head Start classrooms. The PCER studies demonstrating

cognitive effects included only older preschool children of mixed socioeco-nomic backgrounds. The present study included younger preschool children(3-year-olds) as well as older children. Furthermore, it was intentionallyfocused on producing effects for a policy-relevant population of urban,low-income minority children. While PCER studies included low-income chil-dren, they did not explicitly assess their effects for these children in their eval-uations. Third, the present study included a comprehensive set of control

variables (e.g., years in preschool, number of adults in the classroom) andtested interactions between program and relevant subgroups of children.

Children participating in EPIC also demonstrated better mathematicsoutcomes than children in DLM. Findings from PCER showed that imple-mentation of the full DLM program plus the Open Court Reading curriculumdid not produce significant mathematics outcomes. In fact, the only study toshow positive mathematics outcomes was the combination of PreK

Table 2 (continued)

Wave 1 Wave 2 Wave 3 Wave 4

Sample DLM EPIC DLM EPIC DLM EPIC DLM EPIC

Dual-language learner 156.44 160.17 179.13 185.35 192.01 199.65 204.79 213.83

(3.57) (3.37) (3.54) (3.33) (3.62) (3.41) (3.76) (3.55)

n 56 103 56 109 45 108 47 105

Nondual language learner 180.60 184.33 203.29 209.51 216.18 223.81 228.96 237.99

(2.25) (2.29) (2.20) (2.24) (2.32) (2.36) (2.53) (2.57)

n 551 498 541 495 538 490 496 458

Special needs status 150.45 154.18 173.13 179.35 186.02 193.66 198.80 207.84

(3.63) (3.70) (3.60) (3.67) (3.68) (3.74) (3.81) (3.87)

n 61 47 63 47 60 49 55 46

Nonspecial needs status 180.07 183.80 202.75 208.97 215.64 223.27 228.42 237.46(2.26) (2.25) (2.22) (2.20) (2.34) (2.32) (2.55) (2.53)

n 546 554 534 557 523 549 488 517

Note. Entries are estimated population marginal means corrected for cell imbalance asbased on all variables in a given model. Parenthetical entries are associated standarderrors. Estimates are centered on the midpoint for each respective wave. Values for thecovariate age (in months) at entry as applied in the Table 1 models are for the conve-nience of interpretation presented for those\4 and 4 years old at program entry. Cellsizes indicate the actual number of available cases for a given wave. DLM =Developmental Learning Materials Early Childhood Express; EPIC = Evidence-Based

Program for Integrated Curricula.




25/32

Mathematics and the computer DLM mathematics supplement, which wasimplemented on top of a variety of existing comprehensive programs(e.g., High/Scope and Creative Curriculum). The clear mathematics effect

for EPIC is encouraging as evidence grows documenting that early mathe-matics achievement may be a better predictor of later academic successthan early reading (Duncan et al., 2007; Ginsburg, Lee, & Boyd, 2008).

Children receiving EPIC evidenced superior listening comprehensionoutcomes compared to children in DLM. Although DLM was associated

with positive language outcomes in the PCER study, it was not implementedin that study as a stand-alone program. Instead, it was combined with theOpen Court Reading curriculum, which is a language curriculum targetinglistening comprehension. This may explain why the DLM program was asso-

ciated with a significant impact on language in the PCER study but not asa stand-alone program in the present study. The findings on listening com-prehension are especially salient, given its importance to preschool child-rens oral language, vocabulary development, and engagement in reading,more broadly (Skarakis-Doyle & Dempsey, 2008). Listening comprehensionenhances childrens understanding of stories and other texts that are readaloud to them and that they read to themselves. It enables children toremember what they read and communicate with others about what theyread (Armbruster, Lehr, & Osborn, 2001, p. 48). Because so much of the

classroom instruction to which young children are exposed centers onbooks being read to them or receiving oral instruction, their listening com-prehension becomes paramount to their ability to engage the text, for exam-ple, make predictions, respond to questions about the text, share their owninterests, generate and respond to questions, and reenact the story. Asa result, children are facilitated in using a range of metacognitive abilitiesthat deepen their engagement with literacy and other learning.

While EPIC did not surpass DLM in Vocabulary and AlphabetKnowledge, as it had in Mathematics and Listening Comprehension, it com-

pared well against DLM. This is significant because DLM was the only pro-gram in PCER to evidence positive preschool literacy outcomes incombination with the Open Court curriculum, including vocabulary andalphabet skills. In the present study, both EPIC and DLM were comparablyeffective in evidencing significant growth rates in Vocabulary and

Alphabet Knowledge. The significant growth rates for Vocabulary andAlphabet Knowledge were 5.7 and 6.7 scaled score points per month,respectively (with approximately one standard deviation). Overall, these ef-fects underscore the importance of intentional mathematic instruction inte-

grated with language and literacy instruction. The findings suggest thatcomprehensive mathematics instruction in early childhood may have a gen-eralization effect to other important school readiness areas, such as listeningcomprehension. Mathematics has been found to be a good predictor of earlyschool performance (Duncan et al., 2007) and may be conceived of as a kind

Fantuzzo et al.



26/32

of prototypic learning of mental discipline, one like listening comprehension,that requires conceptual understanding, adaptive reasoning, and a disciplineabout rules, steps, sequences, and comparisons (Kilpatrick et al., 2001).

Moreover, mathematics and listening comprehension are related to importantfoundational skills supported by intentional instruction and emphasis onlearning behaviors, an additional focus of EPIC. Kilpatrick and his colleagues(Kilpatrick et al., 2001) refer to the interdependence of mathematics profi-ciency as including conceptual understanding, adaptive reasoning, strategiccompetence, productive disposition, and procedural fluency, all approachesto learning skills reflecting attention control, frustration tolerance, group learn-ing and task approach. The findings from the present study point to a need forfuture research to explore the possibilities of using mathematics skill instruc-

tion as a mediator or precursor to foster good listening comprehension andthe mediating or moderating effect of instruction in learning behaviors ona comprehensive set of cognitive school readiness competencies.

With respect to special subpopulations, there were significant main effectsacross time. Regardless of program, there were distinctive differences betweenDLL children, children with special needs, and 3-year-old preschool childrenand their counterparts across all skills areas. The DLL children and children

with special needs evidenced similar growth rates compared to their non-DLL and nonspecial needs peers; however, they consistently lagged behind

in their performance, with substantial gaps across all skill areas. These findingsadd support to national mandates for more effective ways to bolster preschoolprogramming to enhance the learning experiences of these special subpopu-lations (Espinosa, 2005; Odom et al., 2004). The purpose of this study was notto examine the relative effectiveness of EPIC with special populations, andtherefore, there were not large sample sizes of these subgroups to provideadequate statistical power to explore interactions. This is a limitation of thisstudy that should be considered for future research.

Another interesting developmental finding is the differential cognitive and

language skills growth rates for 3-year-old children compared to the rates ofolder preschool children. Across skills, the slopes for the younger children

were steeper, irrespective of program. The slowing in growth rates for theolder children could not be attributed to a ceiling effect for assessmentsbecause the Learning Express manifested no such capping phenomena, andit was intentionally designed to measure fine gradients of change, even amongmore advanced learning materials. Thus, these findings may speak to the cog-nitive and brain development literature supporting evidence of steeper growthin earlier years with changes in patterns of growth for older preschool and ele-

mentary school age children (Shonkoff & Phillips, 2000).Last, the implications of the development and testing of EPIC in context

for early childhood education are found in our need for realistic evidence-based programming that can be used by early childhood educators (McCall,2009, p. 3). All components of EPIC were developed intentionally to be




27/32

responsive to the context of urban Head Start centers. EPIC was developed incollaboration with Head Start teachers, administrators, and parents to bea comprehensive, integrated intervention for early childhood programs serv-

ing primarily minority children in urban Head Start centers. As such, it in-cludes integrated curriculum-based assessments and an integratedcurriculum that meets Head Start performance standards and state early child-hood standards. It also includes a model of ongoing professional develop-ment that uses indigenous personnel and fits within program resource andtime allocations for professional development. High anonymous teacherand teaching assistant ratings of satisfaction with the EPIC program (98% sat-isfaction) reflect successful efforts to fit EPIC to educators context. The currentevaluation compared EPIC as a stand-alone program to another well-docu-

mented comprehensive stand-alone program. Both were implemented withcomparable fidelity and resources. The evaluation was conducted includingall the children in the participating classrooms and effects were tested acrossrelevant subgroups of keen interest to Head Start. The documentation of therelative efficacy of EPIC to bring about changes in important sets of earlymathematics and reading skills in context and in partnership addresses thenational need for realistic evidence-based programming research.

Three significant contemporary realities of early childhood education inthe United States create the necessity for realistic evidence-based programming

research. First, we have decades of persistent achievement gaps in mathemat-ics and reading that document our urgent need for more effective educationalintervention for young low-income minority students, who are disproportion-ately segregated in large urban and rural areas in the United States (Lee &Burkam, 2002). Second, as a result of our urgent need, we have a proliferationof mandates and standards requiring Head Start and state-funded prekinder-garten programs to implement comprehensive educational interventions toadvance a host of physical, cognitive, language, and social-emotional skillsthrough the use of scientifically based assessments and curricula (Hyson,

2008; Scott-Little, Kagan, & Frelow, 2006). Third, this proliferation of standardshas surpassed the actual capacities of many of our early childhood programsserving low-income preschool children to meet these requirements. In toomany cases, the demand far exceeds actual program capacities. Therefore,many programs with insufficient resources and professional developmenttime and expertise are struggling to meet these requirements. They are forcedto adopt commercially available products with little or no empirical evidencethat promise to meet allthe requirements. Available evidence-based, compre-hensive interventions often neglect to ensure that the program can be effective

in the actual early childhood context where they will need to be implemented(i.e., with programming responsive to meet the needs of the targeted popula-tion within the context of the existing personnel and resources).

These three realities call for applied, evidence-based programmingresearch that a priori provides correction for intervention-context misfits.

Fantuzzo et al.



28/32

McCall (2009), in his timely article, asserts that realistic evidence-based pro-gramming requires (a) development of comprehensive programs in contextand in partnership with early childhood educators in that context, (b) eval-

uation of programs in context with careful attention to indigenous capacitiesand resources, and (c) consideration of these important context and process

variables when attempting to replicate the intervention. The EPIC evaluationin the present study represents a step toward realistic programming researchthat increases the likelihood that evidence-based programs will be used andreplicated to enhance their effectiveness for our groups of young childrenmost in need of a high-quality early childhood education.

NotesThis research was supported by the U.S. Department of Health and Human Services

National Institute of Child Health and Human Development, the Admini

Am Educ Res J 2011 Fantuzzo 763 93

Documents

Transcript of Am Educ Res J 2011 Fantuzzo 763 93