DOCUMENT RESUME

ED 463 743 IR 021 165

AUTHOR Dickard, Norris, Ed.TITLE Great Expectations: Leveraging America's Investment in

Educational Technology. The E-Rate at Five, EnhancingPolicymaking and New Evaluation Models.

INSTITUTION Benton Foundation, Washington, DC.; Education Developmen'tCenter, New York, NY. Center for Children and Technology.

SPONS AGENCY Joyce Foundation, Chicago, IL.ISBN ISBN-1-930615-02-7PUB DATE 2002-00-00NOTE 62p.; For a related toolkit, see IR 021 166.PUB TYPE Collected Works General (020)EDRS PRICE MF01/PC03 Plus Postage.DESCRIPTORS Access to Computers; *Computer Uses in Education;

*Educational Finance; *Educational Policy; EducationalResources; *Educational Technology; Elementary SecondaryEducation; Federal Aid; Program Development; *ProgramEvaluation; Technological Advancement; TechnologyIntegration; Telecommunications

IDENTIFIERS *Connectivity; *E Rate (Internet); Technology Implementation

ABSTRACTIn February 2000, the Benton Foundation, with the Center for

Children and Technology, released "The E-Rate in America: A Tale of FourCities." This report is a continuation of that work. Two overarching concernshave emerged in the current policy climate: it is imperative that the E-Rateprogram is structured in such a way as to maximize impact and it is criticalto be able to measure a return on the nation's massive educational technologyinvestment. Findings and observations are compiled in this report, withchapters written by experts in the field of educational technology. The firstchapter, "E-Rate 102," by Norris Dickard, recounts some of the E-Rateprogram's growing pains and new policy challenges facing it. In the secondchapter, "The E-Rate Takes Hold, but Slowly," Donna Harrington-Lueker reportson lessons that emerged from interviews with administrators, teachers, andtechnology researchers in the same four cities studied in "The E-Rate inAmerican" (Chicago, Cleveland, Detroit and Milwaukee). In the third chapter,"New Approaches to Assessing Students' Technology-Based Work," Margaret Honeyaddresses the timely issue of measuring return on investment in a policyclimate focused on accountability. The fourth chapter, "Assessment ofStudents' Technology-Based Work: An Overview of the Toolkit and Case Studies"by Andy Gersick, Constance Kim, Julie Thompson Keane, Wendy Friedman, andKatie Culp, presents case studies in which researchers worked in individualclassrooms with teachers in the cities of Chicago and Milwaukee. Together,they developed assessment tools and guidelines to help teachers measurelearning gains as students learned to use various information technologytools for research and presentation-compiled into an "evaluation toolkit"(available online and as a supplementary publication to this report). In thelast chapter, "Enhancing State and Local Policymaking about EducationalTechnologies," Chris Dede introduces his "state policy framework" andoutlines the need for a more coordinated and systematic approach topolicymaking around educational technology. Appendixes present the "StatePolicy Framework" and outline educational technology evaluation resources.(AEF)

Reproductions supplied by EDRS are the best that can be madefrom the original document.

CCI,MMUNICA'TAON

tB E NTQL.Aftmi.,FOUNDTIPN

'11

AD

a

44,

;4;441

L,

:MI u

s`t

,

SPOLICY P,R0 GRAM

I-K,Csa

kYi Ci 14.1111i Y 1 M 140 71-1,,i

htIJSkq,

,

tr ,,

gz:'=

:I

he E-Rate at Five;T:'Enhippcigg

,

6,nd New Evaluation Vil'idels

CENTER F2RCHILDREN &TECHNOLOV

u .

_ :

u I11

-

r !

S.

-

I II

P -

. 4es "

0 ea

BENTONFOUNDATION

www.benton.org

www.edc.org

'ION! -Fo u N`))A-17 01(0

The mission of the Benton Foundation is to articulate a public interest visionfor the digital age and to demonstrate the value of communications for solvingsocial problems.

Benton's Communications Policy Program (CPP) seeks to accelerate digitalopportunity for all people by promoting equity, access and a diversity ofvoices in the emerging media environment.

President: Andrea L. TaylorExecutive Vice President: Karen MenichelliCommunications Policy Program Director: Anthony WilhelmSenior Associates: Rachel Anderson, Andy Carvin, Norris Dickard

Board of Directors: Charles Benton (Chairman), Shelley Benthn, WorthBruntjen (Treasurer), Elizabeth Daley, Terry Goddard, Henry M. Rivera(General Counsel), Harold A. Richman, Terry Tinson Saario, Jorge ReinaSchement, Andrea L. Taylor.Trustees: Charles Benton, Marjorie C. Benton, Leonard Schrager

For more than four decades the Education Development Center, Inc. hasbeen bridging the worlds of research, policy and practice. EDC's work isfocused in multiple areas including early child development, K-12 education,health promotion, workforce preparation, community development, learningtechnologies, literacy, institutional reform and social justice.

Since 1980, EDC's Center for Children and Technology (CCT) has been at theforefront of educational technology research and development. CCT seeks tofoster learning and improve teaching through the development and thoughtfulimplementation of new technologies in a wide range of educational settings.

President, EDC: Janet WhitlaVice President EDC and Director, CCT: Margaret HoneyAssociate Director, CCT: Cornelia BrunnerAssistant Director, CCT: Katie McMillan Culp

Board of Directors: Alonzo L. Plough (Chair), Charles Benton, EdwinCampbell, Beatriz Chu Clewell, Hans Decker, Pat Mora, Bradley Palmer,Deborah Wadsworth, William N. Whelan, Janet Whitla, Gail T.P. Wickes

© 2002 Benton Foundation

Design: Freedom By Design

ISBN 1-930615-02-7 r;d1MaiMalZ

LEVER 0 G ERIC °S 11 ESTMENT[N

GREAT EXPECTATIONS]EIPUCATION L TECHNOLOGY

Benton FoundationCommunications Policy Program

Education Development Center, Inc.Center for Children and Technology

Edited by Norris Dickard

ACMHOWLEDGENIEHUS

This project was made possible by a gener-ous grant from the Joyce Foundation(www.joycefdn.org), which supports effortsto strengthen policies that improve the qual-ity of life in the Great Lakes Region.

The Benton Foundation's CommunicationsPolicy Program and the EducationDevelopment Center, Inc.'s (EDC) Centerfor Children and Technology (CCT) wouldlike to express their gratitude to the follow-ing people for their contributions:

Peter Mich, of the Joyce Foundation, for hissupport and helpful suggestions. CharlesBenton, chairman of the board of the BentonFoundation and member of the EDC board,for his encouragement and assistance. TonyWilhelm, of the Benton Foundation, for hisoversight and consulting role.

The University of Chicago (Tony Bryk andEben Rogers in particular) and AlvernoCollege, in Milwaukee, for hosting ourMay 2001 policy roundtables. Jill Bond,Leslie Harris, Ken Kay, Steve Kohn,Keith Krueger, Helen Morris, MarkSchneiderman and Cheryl Williams, for pro-viding recommendations of individuals toinvite to the roundtables and advice on thestructure of the agenda. All the individualsfrom foundations, corporations and federal,state and local governments who took timeto attend the roundtables; their insightswere invaluable for refining our productswhile under development and providinginformation on critical policy issues thatneeded to be addressed. The NationalEducation Association for hosting ourrelease event and the American YouthPolicy Forum for co-sponsoring it.

John Bailey, Mel Blackwell, Colin Crowell,Barbara Pryor and Richard White for con-versations on the E-Rate program and toRobert Nelson, Ricardo Tostado and KeithKrueger for reviewing the chapter "E-Rate102." Rachel Anderson, Andy Carvin,Julee Newberger, Carol Schookhoff and thestaff of Bulletproof for editorial assistance.Lisa Walker, of the Education WritersAssociation, for permission and assistancein the reprinting of the chapter "The E-RateTakes Hold, But Slowly."

Principal Sally Brown and Char Harteau,Wisconsin Center for Lifelong Learning;Mary Delgado, Marquete University;Principal Elizabeth Gearon and La TinaTaylor, Nathanael Greene ElementarySchool; Director Michael T. Johnson, NorthKenwood Oakland Charter School; PrincipalAda Rivera, John C. Audubon MiddleSchool for their assistance in facilitatingthe collaboration between CCT and theirschools. The teachers at John C. AudubonMiddle School, Nathanael GreeneElementary School, North KenwoodOakland Charter School and WisconsinConservatory of Lifelong Learning, andDiana Joseph at the Center for SchoolImprovement, University of Chicago, fortheir invaluable work and time in the devel-opment of the evaluation toolkit.

CDIF ©GRIMM'S

GREAT EXPECTATIONS, LEVERAGING AMERICA'S INVESTMENTIN EDUCATIONAL TECHNOLOGY

Preface: Digital Opportunity for All Students 5

Introduction: The E-Rate at Five 6

I. E-Rate 102: Politics, Policy and ProgressNorris E. Dickard 9

II: The E-Rate Takes Hold, But Slowly: A View From Four CitiesDonna Harrington-Lueker 18

III: New Approaches to Assessing Students' Technology-Based WorkMargaret Honey 24

IV. Assessments of Students' Technology-Based Work:An Overview of the Toolkit and Case StudiesAndy Gersick, Constance Kim, Julie Thompson Keane,Wendy Friedman, Katie Culp 29

V. Enhancing State and Local Policymaking about Educational TechnologiesChris Dede 41

Appendices

A. State Policy Framework for Assessing Educational Technology Implementation . . . 48

B. Educational Technology Evaluation Resources 55

PREFACE: DIGITAL OPPORTUNITYFOR ALL STU ENTS

When Congress created the E-Rate byenacting the historic TelecommunicationsAct of 1996, a bipartisan coalition of mem-bers understood that universal service need-ed to be expanded to meet new challenges.Universal service was traditionally associat-ed with equal access to telephone service,but the advent of personal computers andthe Internet resulted in powerful new toolsfor communication. and new disparities toaddress.

In the mid-1990s, our growing recognition ofthe promise that new telecommunicationstools held for education, economic develop-ment, civic discourse and career advance-ment came at a time of groWing concernover a "digital divide" separating individualsand communities that had access to thesetools from those that did not.

The Telecommunications Act of 1996 wasfirst and foremost an effort by Congress tospur the development of affordable andwidespread telecommunications services byderegulation and increased industry compe-tition. It was also acknowledged that moreneeded to be done to ensure that under-served communities, especially in ruralareas and poor communities, were not at acompetitive disadvantage when new infor-mation networks were created or expanded.Thus, the E-Rate, and its discounts ontelecommunications services to schools andlibraries serving disadvantaged individuals,resulted from a bipartisan commitment toprovide equal educational opportunity andaccess to information technology tools forall Americans.

Information released by the U.S. CensusBureau in August 2000 demonstrates thatfocusing needed resources on schools andlibraries in disadvantaged urban and ruralcommunities has made a significant differ-ence. Ninety-four percent of school-age chil-dren in homes with an income above$75,000 have computer access at home,although only 35 percent of school-age chil-dren in households with income under$25,000 have home computer access.Schools have filled the gap for these low-income children, where 72 percent hadaccess.

As the following report shows, greatprogress has been made in bridging the dig-ital divide, and the E-Rate has been a criticalsource of building materials for this bridge.But, this is no time to rest on our laurels. Tosupport the unique opportunities technologyoffers to improve teaching and learning,work remains to be done in a number ofareas, including professional development,curriculum design and assessment. Thistimely report highlights a number of areaswhere work is needed and provides usefultools and suggestions for maximizing thisimportant investment.

Representative Edward J. Markey CD-Mass.)

Senator John D. Rockefeller CD-W.Va.)

Senator Olympia J. Snowe CR-Maine)

Representative Fred Upton CR-Mich.)

7 z

OHTIRODUCT1CDH ThE E-RATE

Over the past 10 years the nation hasinvested $37.9 billion to bring educationaltechnology (ed tech) and Internet connec-tivity to America's schools. Like their coun-terparts in the private sector, educators,administrators and policymakers are hopingthat by harnessing technology, they canvastly improve the productivity of the edu-cational enterprise.

The E-Rate's telecommunications discountsrecently made available to schools com-bined with other federal, state and localfunding has been the catalyst for tremen-dous progress (see Chart 1). Recent dataindicate that:

By the fall of 2000, 98 percent of publicschools in the U.S. had access to theInternet, compared to 35 percent in1994.

0 In 1994 only 3 percent of instructionalclassrooms had computers with Internet

T RVE

access; 77 percent were connected byfall 2000. As in previous years, therewere still signs of a digital divide.Schools with the highest concentrationof students in poverty were much lesslikely to have connected computers inthe classroom (60 percent connected)than those with lower concentrations ofpoverty (80 percent connected). Whilemore needs to be done in this area, thedata suggest that the rapid closing ofthe gap was due in part to the E-Rate.

0 By fall 2000, the ratio of students toinstructional computers had decreasedto 5 to 1, a ratio considered by someexperts to be "a reasonable level for theeffective use of computers within theschools." However, the ratio wasgreater (9 to 1) in schools with thehighest concentration of students inpoverty.

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

... K-12 Instructional Roomswith Internet Access

1998 1999 2000

Source: National Center for Education Statistics, May 2001

Chart 1: K-12 Instructional Rooms with Internet Access

Over the years, changes have also occurredin the type of network connections used byschools and the speed at which they areconnected to the Internet. Since the E-Ratecannot be used for computer, software orsupport services, its investment is expectedto have the greatest impact in the area ofconnectivity. Progress appears to havebeen made in this area as well:

o In 1996, dial-up connections were usedby almost 74 percent of public schoolswith Internet access. By 2000, schoolstended to also use faster, dedicatedlines such as T1/DS1 or fractionalizedT3 lines.

o In 2000, 77 percent of schools useddedicated lines to connect to theInternet while only 11 percent used dial-up connections; 24 percent used othercontinuous-type connections such asISDN or cable modems.

In February 2000, the Benton Foundation,collaborating with the Center for Childrenand Technology (CCT), released The E-Ratein America: A Tale of Four Cities. This report,made possible also through the generoussupport of the Joyce Foundation, is a con-tinuation of that joint work.

The E-Rate in America was one of the firststudies of the impact of the then-new feder-al program, tracing the ideas and politicalbattle-s that led to its establishment andrecounting the practical issues confrontingschool districts as they sought to benefitfrom E-Rate resources. Also included in thatwork were questionnaires designed to helpschool officials begin the process ofassessing technology use in their schools.

As part of The E-Rate in America, four largeurban districts (Chicago, Cleveland, Detroitand Milwaukee) were studied in fall 1999.While each of the districts had a uniqueexperience when it came to planning, apply-ing for and using E-Rate discounts, the

process afforded a number of common ben-efits to all of them:

o E-Rate discounts made it possible forthe districts to accelerate their networkinfrastructure development and dramat-ically expand Internet access.

o The E-Rate freed up resources to payfor other elements of district education-al technology programs such ascomputer and software purchases andteacher professional development.

o The E-Rate application process led toimproved district planning practices.

At the same time, E-Rate implementationraised new challenges and taxed districts'resources in the following manner:

o The E-Rate process strained relation-ships with vendors who provided thetelecommunications services toschools.

o It was necessary for administrators andcommunity stakeholders to be mademore aware of the impact of theprogram.

o Much-needed building basics notcovered by the E-Rate, such as electri-cal upgrades and hardware purchases,delayed the deployment of informationtechnology.

o Teacher professional developmentneeds increased dramatically.

o Districts were becoming highly depend-ent on E-Rate funding to sustain theirnetworks.

In this second phase of our E-Rate work, theBenton Foundation and the Center forChildren and Technology continued ourinvestigation into the new program anddeveloped new tools to assist teachers,administrators and policymakers. In earlyNovember 2001, E-Rate funding commit-ments were still being made for year four of

the program and the year five applicationwindow opened. Two overarching concernshave emerged in the current policy climate:It is imperative that the E-Rate program isstructured in such a way as to maximizeimpact and it is critical to be able to measurea return on the nation's massive educationaltechnology investment. The Benton Found-ation and CCT have compiled our findingsand observations into this new report, GreatExpectations, Leveraging America's Investments inEducational Technology, with different chapterswritten by experts in the field of educationaltechnology.

The first chapter, "E-Rate 102," by NorrisDickard, recounts some of the E-Rate pro-gram's growing pains and new policy chal-lenges facing it. Drawing on a dialogue thattook place at two national policy roundtableevents in May 2001, and through interviewswith key policymakers, Dickard suggestsfuture improvements to program structureand administration. The chapter will be ofgeneral interest to those interested in

telecommunications policy and of specialinterest to those policymakers contemplat-ing program improvements.

In the second chapter, "The E-Rate TakesHold, But Slowly," Donna Harrington-Lueker,of the Education Writers Association (EWA),reports on lessons that emerged from inter-views with administrators, teachers andtechnology researchers in the same fourcities studied in The E-Rate in America. Thischapter, reprinted with permission fromEWA, was part of a larger report on technol-ogy in urban schools also commissioned bythe Joyce Foundation. It provides an updateon the progress in Chicago, Cleveland,Detroit and Milwaukee since the publicationof The E-Rate in America and makes the casethat building computer networks may be eas-ier than helping schools use the technologyto its maximum potential.

E3

The third and fourth chapters, by MargaretHoney and her colleagues at the Center forChildren and Technology, address the timelyissue of measuring return on investment in apolicy climate focused on accountability.She writes that we must move beyond seek-ing a direct correlation between technologyinvestments and rising standardized testscores. Rather. Honey argues, it is importantto create assessment frameworks thatdirectly correspond to the unique teachingand learning opportunities that technologiesmake possible.

The fourth chapter presents case studies inwhich CCT researchers worked in individualclassrooms with teachers in the cities ofChicago and Milwaukee. Together, theydeveloped assessment tools and guidelinesto help teachers measure learning gains asstudents learned to use various informationtechnology tools for research and presenta-tion compiled into an "evaluation toolkit,"available online and as a supplementary pub-lication to this report. This toolkit will be ofparticular use to classroom teachers andresearchers thinking about evaluation. Theevaluation toolkit is available online atwww.benton.org/e-rate/execsummary.html.

In the last chapter, Harvard University'sChris Dede introduces his "state policyframework" and outlines the need for amore coordinated and systematic approachto policymaking around educational technol-ogy. He explains how educational technolo-gy investments impact, and are impacted by,the larger educational reform context in

which they are made. With greater decision-making being driven down to state depart-ments of education through block grants andother consolidation of federal funding, thematrix will be a useful tool for both state andlocal administrators.

1 0

CHnPITIER s

E-RATE 902: POLITICS, POLICY AND PROGRESS

As a result of the E-Rate and other federaleducational technology investments, greatprogress has been made in bringing multi-media computers, high-speed networks andInternet connections to America's schoolsand libraries.

The Benton Foundation and CCT's report,The E-Rate in America: A Tale of Four Cities,explained how the E-Rate program worked,and in a chapter appropriately titled "E-Rate 101," reported on the passage of thelegislation authorizing the E-Rate anddescribed the initial start-up phase of theprogram. This chapter updates thatinformation by describing progress to dateand programmatic growing pains andsuggesting areas of possible improvement.This chapter, like others in this report,benefited from a dialogue among federal,state, local, foundation and corporatestakeholders who participated in twoBenton E-Rate/ed tech roundtables, heldin May 2001 in Chicago and Milwaukee.

ECAP: FROM &NOME-ROCKEFELLER TO FCC PROGRAIMilFrom the beginning, the creation of the E-Rate was by no means assured. TheMarkey-Fields bill, calling for an "E-Rate,"or education rate, passed the House by avote of 423-4 in June 1994 but died whenthe Senate did not take their bill to the floorin the waning days of the 103rd Congress.As part of the 1995 deliberations over thereauthorization of the Communications Actof 1934, the bipartisan Snowe-RockefellerAmendment, calling for the creation of theE-Rate, barely made it out of the SenateCommerce Committee winning approvalby only one vote. Late in the legislativeprocess, one member of Congressdescribed the entire telecom bill as "deadas Elvis." The amendment, however, soonbecame a rallying cry for the schools andlibraries community who argued that suchfunding was needed to bridge a digital dividethat existed in their institutions. The Houseand Senate ultimately passed a compro-

BY NORRIS DICKARD

mise bill that was signed into law, with theE-Rate included.

The Telecommunications Act of 1996expanded the traditional definition of univer-sal service nationwide phone service at areasonable cost to consumers to includebroader telecommunications assistance toschools and libraries. The act authorized theFederal Communications Commission(FCC) to create a program offering dis-counts to these institutions on eligibletelecommunications services (such asphone service, Internet access, internalconnections and related equipment). The E-Rate provides discounts ranging from 20 to90 percent to applicants in urban and ruralareas. Larger discounts go to thosedeemed economically disadvantaged basedon their service to students eligible to par-ticipate in the federal school lunch program.Local and long-distance telephone compa-nies contribute funding. In the first twoyears of the program, the E-Rate committednearly $4 billion in telecommunications dis-counts.

The Schools and Libraries Corporation(SLC) was set up to administer it, and inNovember 1998, mailed its first wave ofcommitment letters to successful appli-cants. On January 1, 1999, the SLC laterbecame the SLD, the Schools and LibrariesDivision, of the Universal ServiceAdministrative Company.

ThE E-RATE TUREAS TWOIn September 2000, shortly after the publi-cation of the E-Rate in America, the U.S.Department of Education released its find-ings from a preliminary study on who andwhat was funded in the first two years of theE-Rate.

Among the key findings from this study:

o Eighty-four percent of E-Rate discountswent to the nation's public schools,despite private schools and librariesbeing eligible.

11

o E-Rate per-student funding to schooldistricts increased dramatically withpoverty, and the most disadvantageddistricts received almost 10 times asmuch per student as the least disadvan-taged.

o Urban schools and libraries, typicallywith greater concentrations of poorchildren and tending to be larger in size,received larger average funding levelsand higher funding per student.

Because program funding is stronglytied to poverty and minority concentra-tion is highly correlated with poverty,total and average per-student E-Ratefunding generally increased with higherconcentrations of minority (nonwhite)students.

o In the first two years, the largest shareof E-Rate funds, 58 percent, was usedto support the acquisition of equipmentand services for internal buildingconnections; 34 percent was used fortelecommunications services; 8 percentwas allocated for Internet access costs.

the Houston Independent School District, tobe his education secretary. Paige was high-ly regarded and hailed as a "tech champ"who knew how to move districts from "oldschool to e-school." For example, after areport on the deplorable state of manyHouston school libraries, Paige announceda plan to spend millions to bring new tech-nology resources into school libraries andordered the purchasing of new electronicreference materials.

When making his first domestic policy prior-ity announcement as president, Bushissued an education manifesto titled "NoChild Left Behind." The policy blueprint,calling for the revamping of the Elementaryand Secondary Education Act (ESEA), pro-posed to consolidate existing educationaltechnology programs, arguing that:

Schools should use technology as a tool to

improve academic achievement, and that

using the latest technology in the classroom

should not be an end unto itself.

E-RATE FUNDING (in billions of dollars)Year #Applications Requested Commitments Amount1 30120 $2.4 24967 $1.72 32000 $2.4 29871 $2.133 36000 $4.7 26307 $2.124 35300 $5.2 24509 $1.6*

* As of 10/30/01.

THE E- TE D THE USHDIVIONOSTER TOON

With the inauguration of George W. Bush aspresident in January 2001, many wonderedwhat the future would hold concerningfederal education technology investments.In a positive first move, Bush tapped RodPaige, who had served as superintendent of

This proposal begins

to accomplish that

goal by streamlining

duplicative technology

programs into a per-

formance-based tech-

nology grant program

that sends more money to schools.

Consolidating the technology grant pro-

grams and allocating with E-Flate funds by

formula ensures that schools will not have to

submit multiple grant applications and incur

12

the associated administrative burdens to

obtain education technology funding.

President Bush proposed a fiscal year 2002Department of Education increase that hesaid would be, at 11 percent with $4.6 bil-lion in new funding, the highest for any fed-eral agency. "Education is my top priorityand, by supporting this budget, you'll makeit yours as well," Bush told Congress duringhis State of the Union address. The detailson funding for specific programs wouldcome several months later.

The Bush administration proposal to mergethe E-Rate with other Department ofEducation technology programs created afirestorm. Senator Jay Rockefeller (D-W.Va), one of the program's key congres-sional champions, immediately announcedhis opposition to President Bush's proposal,calling it "a grave mistake...a major stepbackwards." He promised to fight the con-solidation, arguing:

Each school (under the E-Ratel gets to apply

for the telecommunications services they

want and need. Under the Bush block grant

approach, local schools would have less flex-

ibility, not more. Under the Bush block grant,

private and parochial schools would have to

negotiate with state education agencies and

worry about entanglements of federal regu-

lations. Most importantly, the secure funding

for the E-Rate and investments in technology

would be jeopardized.

Senator Rockefeller also said that consoli-dating the E-Rate would be breaking a"deal" that was cut as part of the 1996Telecommunications Act, a national quid proquo between telecom companies and theAmerican public:

The telecommunications companies wanted

more competition and the ability to expand.

In exchange, we insisted on a strong, contin-

ued commitment by the telecommunications

companies to "preserve and advance" uni-

versal service, including access to advanced

telecommunications services for schools,

rural health care providers and libraries.

The Consortium for School Networking(CoSN) and the International Society forTechnology in Education (ISTE), in a jointpolicy statement entitled "Preparing theClassroom for the 21st Century," weighedin with the new Bush administration as well.They noted that the ESEA reauthorizationwould likely include the consolidation ofsome ed tech funding, giving more discre-tion to states and local communities overthe use of those funds, but they admon-ished, "The gains that have been achievedthus far will be imperiled if the federal gov-ernment simply cedes its leadership role inthis area."

In particular, CoSN and ISTE urged the newBush Administration to:

Maintain the emphasis on equity in

federal ed tech programs

o Increase the focus on professionaldevelopment

o Connect federal ed tech support torigorous assessment

o Examine the utility of technologies asassessment tools

O Expand the federal research agenda ined tech

O Create a national ed tech clearinghouse

O Preserve federal leadership activities

O Maintain the E-Rate program

In the context of the E-Rate, the specter ofblock grants as a prelude to future cutsfueled much opposition to the proposal. Intheir ed tech campaign proposals, the Bushdomestic policy team had been careful tocombine ed tech funding into one block,without proposing an overall cut. The $3 bil-lion called for in the Bush campaign ed techproposal was roughly the same amount asthe E-Rate and ESEA Title III programs com-bined. But many in the education communi-ty feared that a move to fund the very largeE-Rate investments ($2.25 billion) as part ofthe U.S. Department of Education budgetand federal appropriations process wouldopen up the program to cuts immediate orin the future.

Indeed, with the warning volleys fired, theBush Administration quietly backed off oftheir proposal to combine the E-Rate withother Department of Education funds. In

testimony on ESEA reauthorization beforethe House Committee on Education and theWorkforce on March 7, 2001, EducationSecretary Paige said President Bush wouldnot try to change the E-Rate. In an eSchoolNews article, White House spokeswoman

Lindsay Kozburg confirmed Paige's state-ment and is reported to have added, "It'snot something that's happening with thisround of consolidations ... what we are pur-suing right now is [the consolidation of] pro-grams that are currently [administered by]the Department of Education ... we are cer-tainly reviewing whether we can, andshould, consolidate the E-Rate."

Critics felt their staunch'. opposition hadbeen justified when President Bush'sdetailed fiscal year 2002 budget emerged inspring 2001. Bush called for a 6 percentreduction ($817 million, down from $872million in the prior level of appropriations) inhis proposed state block grant to replacethe existing ESEA Title III educational tech-nology programs. Defenders of his budgetpointed out that $817 million was the sameas the Clinton Administration's fiscal year2001 budget request and simply eliminatedearmarks added in the final scramble to getan appropriations bill passed (see Chart 2).

President Bush's budget also called forimproving the E-Rate program. "The admin-istration is seeking administrative improve-ments in the E-Rate to ensure that this pro-gram provides greater flexibility to schoolsand libraries in how they use their E-Ratediscounts, while reducing the administrativeburden they have faced in applying for edu-cational technology funds," the budget doc-ument read.

Specifically, the administration called on theFCC to complete, no later than September2002, a rulemaking to revise the E-Rate pro-gram to:

O Allocate funds for discounts on a needs-based formula

Define eligible services to include thosethat promote the effective use oftelecommunications, such as teachertraining and software

14

$1,000

$900

$800

$700

$600

$500

$400

$300

$200

$100

ESEA (Title Ill) Ed Tech Investments

$0

FY93 FY96 FY99

Ed Tech Appropriations(in millions of dollars)

FY02(proposed)

Chart 2: ESEA Ed Tech Appropriations

Assess how to institute performancemeasures for the program to gauge theeffectiveness of educational technologyin promoting student achievement

ThE E- ATE AT FOUR: FocusTU NS TO Mil ROVEIMEINIT"The E-Rate program, we think, is workingwell," said a lobbyist for the NationalEducation Association (NEA) in a January2001 eSchool News interview. "It's an innova-tive way of getting resources to schoolswithout competing for other education fund-ing ....Tampering with the program nowwould be a huge mistake, and I don't thinkCongress will agree to do that."

Similarly, Barbara Pryor, an aide to SenatorRockefeller, reminded participants at oneBenton stakeholder roundtable thatchanges as simple as expanding eligibleservices, especially beyond those related totelecommunications, could result in a newlawsuit. She told participants that in 1996

there was a great deal of debate over eligi-ble services and, even then, lawsuits werefiled questioning the legality of the FCC'sdecision to implement the E-Rate. She stat-ed, "There are some people who don't likethe E-Rate and would like to see it end up incourt again."

But as the months passed in 2001 and itappeared the E-Rate was "safe" from beingmerged with appropriated programs at theDepartment of Education, thoughts turnedfrom protecting to tinkering, to makingneeded reforms. Frustrations had beenmounting for years. While praise had beennear universal over the resources the E-Rate provided, for many the process ofapplying and complying left much to bedesired. Bush's proposals sounded like asensible alternative. Ricardo Tostado, anIllinois State Department of Education offi-cial, said at one Benton roundtable, "the E-Rate program has some badly squeakingwheels that need to be oiled." He went on

1 5

to recount that he had spoken with 300 to400 schools in Illinois, most of which hadtold him they would be willing to take 30percent less in E-Rate discounts in a blockgrant scenario, if they could count on thediscounts as a reliable source of fundingand not have to deal with all the hassles ofapplying and complying.

A special insert called "Rating the E-Rate,"in a September 2000 edition of EducationWeek, included articles in which those inter-viewed expressed a similar love/hate rela-tionship with the program. One article wasfittingly titled "A Bureaucratic Hassle, ButWorth It." In an introduction to the series,Andrew Trotter wrote:

But if the E-rate is now speeding many

schools toward new technologies for learn-

ing, it's been a difficult journey for the nerve-

racked passengers: the school officials who

have had to learn the program's intricate

rules, react to a seemingly inexplicable

series of changes on the fly, and then wait,

and wait some more, for crucial decisions to

be made. Even ardent supporters bemoan

the persistent problems in the program.

Many of the "bureaucratic hassles" couldbe traced to administrative reforms thatfollowed the General Accounting Office's(GAO) 1998 criticism of E-Rate administra-tion. A series of GAO reports released justbefore and after the election of PresidentBush seemed to bolster the new adminis-tration's arguments for changes. One prob-lem was related to demand for discountsexceeding the supply. For the third andfourth program years, requests substantial-ly exceeded the program's funding cap of$2.25 billion (see Chart 3). For complicated

reasons related to E-Rate funding priorities,this especially affected the funding for inter-nal connections and caused the FCC toconsider new rules ultimately not adopted

to make sure more applicants couldreceive discounts in this area.

New challenges, and perhaps anotherhassle, came in the form of implementingthe E-Rate portion of the Children's InternetProtection Act (CIPA). CIPA places restric-tions on those receiving E-Rate discounts,requiring Internet safety policies and tech-nology solutions that block or filter certainmaterial from being accessed through theInternet. Items to be blocked were thosethat "include visual depictions that are (1)obscene, (2) child pornography or withrespect to use of computers with Internetaccess by minors, (3) harmful to minors."

The FCC did not define a "technology pro-tection measure" but wrote that it was pro-hibited from extending universal service dis-counts to technology protection measuresor other costs associated with implement-ing the law. The FCC issued regulations thatwent into effect in April 2001, requiringschools currently receiving E-Rate funds tocertify that they were in the process of"complying." Year five recipients will haveto be in full compliance when service startsin July 2002.

One participant at a Benton roundtable wasparticularly irked by this "unfunded man-date," but a congressional staff memberattending the same meeting reported thatCongress had held hearing after hearing onfiltering, with screen after screen showingthe worst sites on the Internet that childrencould view, and members arguing that fed-eral funds were making this possible. Evenmembers who did not want to mandate fil-tering, the participant added, felt they had tosupport the legislation because the issuewas politically so hot that, if not addressed,could jeopardize the program.

$6

$5

$4 FundingE-RateRequests

$3 - - -E-Rate FundingCap

$2 (in billions ofdollars)

$1

$0

Year Two (1999-00) Year Three (2000-01) Year Four (2001-02)

Chart 3: E-Rate Demand Exceeds Supply

The tragic events of September 11, 2001,and the subsequent bipartisan rallyingaround the national priority of fighting inter-national terrorism, have taken many issuesoff the legislative agenda. Optimistically,this unified" Congress could affordPresident Bush new leverage in pushing for-ward his domestic policy agenda. Theimpact on E-Rate policy remains to be seen.

CONICILLPSOON: THE FUTURE OFTHE E-RATEThe SLD began the year five applicationperiod, which opened on November 5,2001, by urging schools to file early andonline. SLD officials also touted new "bellsand whistles" that were meant to makethe process easier. Those improvementsinclude: a new electronic certificationprocess, which saves the SLD from han-dling 30,000 signed pieces of certificationpaper; an electronic application which helpsapplicants avoid simple mistakes; and acommitment to not making any last-minuterule changes.

Drawing on input from the dialogue atBenton roundtables and conversations withkey stakeholders, below are some othersuggestions related to improving this impor-tant program.

o Keep the E-Rate at the FCC and thefocus the same.

Calls to move the E-Rate to theDepartment of Education as part of ablock grant are misguided. Not only isthe legal authority for such an actionquestionable but ending the program atthe FCC and making it part of the annualappropriations process could jeopardizethe funding. Telecommunications carri-ers' contributions currently support E-Rate discounts, not the federal treasury.The program's priority needs to remainconnecting schools and libraries in pooror rural areas. Moving the E-Rate out ofthe FCC would also place the E-Ratebenefits to private schools, as well aspublic libraries, at potential risk. Acorporate representative at the Benton

Milwaukee roundtable expressedconcern that the "E-Rate program willbe ended and people will claim that itsobjectives have been achieved. There islots of work left to do."

2 Lift the funding cap from the currentlevel of $2.25 billion.

As data on the connectivity of thenation's schools indicate, the work ofthe E-Rate is far from over. Of course,technology is also continually changingand schools will have recurring and newcosts. As Bob Nelson, director of tech-nology for Milwaukee Public Schools,said at a Benton roundtable, "there is apropensity to declare victory too earlyin this work." Year three and fourrequests demonstrate that demand fordiscounts far exceeds supply. Theunmet needs are especially acute in thearea of internal connections. Some ana-lysts have suggested that if the programpriorities were changed, to place agreater emphasis on internal connec-tions for example, such needs could bemet. At the Benton policy roundtable inMilwaukee, one participant provided hisanswer to the policy dilemma: "Wehave to raise the caps, and we have toprioritize." Increasing the funding capcould also leverage new resources, asE-Rate discounts free up school tech-nology funds for other critical needssuch as computer and software pur-chases, technical support, electricalupgrades and teacher professionaldevelopment.

Reduce the paperwork burden onapplicants.

A Latino Issues Forum Report, releasedin summer 2001 and called ConnectingCalifornia's Children: Is E-Rate Enough,

focused on evaluating the impact of theE-Rate on that state's most disadvan-taged schools. Researchers found that

c-

many California underserved schoolsthat desperately need E-Rate funds donot feel they can negotiate the laboriousand technical process. They simply do nothave the time or staff with the grant-writ-ing expertise necessary. The SLD hasgreatly simplified the process of applyingfor the E-Rate. The SLD should investi-gate in particular changes that couldmake it easier for smaller schools and dis-tricts to participate at higher levels.

ga Conduct outreach and assistance toschools in low-income communities.

Connecting California's Children reportedthat a surprising 43 percent of the dis-advantaged schools participating in itsstudy did not even know about the E-Rate. In trying to keep administrativecosts low for the program, the FCCavoided creating a pool of resources forstate leadership activities. Some largerstates (such as California, Florida,Illinois and Pennsylvania) and smallerstates (such as Iowa, Mississippi andNorth Carolina) have committed sub-stantial resources to assisting localitiesto apply and comply. Schools outside ofsuch states must often go it alone. TheSLD should examine ways to improveoutreach and assistance to low-incomecommunities, including possibly creat-ing a fund for state assistance. Statesshould be engaged more broadly as fullpartners in the program.

Investigate ways to improve programadministration and structure.

A participant at the Chicago Bentonroundtable, representing a major educa-tion association, said, "There are seri-ous operational problems with the E-Rate. The way the program is managedwill be radically changed." Despiteimprovements to date, it is critical thatthe SLD engage in a comprehensiveaudit of its procedures to investigate

IS

ways to further speed up the process-ing of commitments and appeals, aswell as to eliminate conflicting rules andregulations. A more fundamental ques-tion is whether the SLD is structured insuch a way as to be most effective inresponding to local needs. Is its central-ized, federal application and appealsprocess the most efficient way to deliv-er services? Would a more decentral-ized program, with policy driven fromthe federal level and administrationdevolved to a regional or state level, bea more appropriate structure for makinggrants? It is common for large federalgrant programs of over $1 billion todeliver sub-state grants via a state enti-ty, which is ostensibly more attuned tolocal and regional needs? Would astate-by-state allocation of E-Rate dol-lars, based on income, need and popu-lation, be more appropriate than the cur-rent application-driven system?

Reassess the appropriateness ofcurrent discount levels and priorities.

A critical question for the FCC is whetherthe current matrix of discount levels (20to 90 percent) is most appropriate andthe priorities reflect the most pressingand strategic needs. Moreover, is thestudent free and reduced lunch count ofa school or district (currently the basisfor determining discount levels) the mostaccurate measure of the need forresources? As the Bush administrationhas requested, perhaps the SLD shouldlook at how the program can be moreresponsive to schools in high-cost areas.A director of educational technology for alarge urban school district and participantat Benton's Chicago policy roundtableput it this way, "Maybe we should bedoing fewer schools, but doing themfrom A to Z."

Expand the list of eligible products,services and vendors.

The FCC announced new services,such as Internet2 and wirelessnetworks, being eligible for E-Ratediscounts in year five. Despite thisprogress, are the current definitions ofeligible hardware too limited? Are dis-counts for phone service still relevantgiven the goals of the program? Is theprogram keeping up with new technolo-gies and advances? How much band-width is enough? Should a panel be setup by the FCC to study the issue andmake recommendations that wouldapply to schools? Steve Kohn ofVerizon, participating at the Chicagoroundtable, asked, "What are the otheractivities you need to make the wholepackage workable?"

To conclude, because of the E-Rate andother federal educational technology invest-ments, multimedia computers, high-speednetworks and sophisticated new softwareprograms are becoming commonplace inAmerica's schools and libraries. Since theE-Rate is viewed as such a vital resource toconnect disadvantaged schools to 21st-century digital opportunities, proposing toeliminate the program elicits a reaction sim-ilar to calling for cuts in the school lunchprogram. Former critics are now in agree-ment that the E-Rate is a critical investment,even as they call for "bureaucratic hassles"to be eliminated.

Norris E. Dickard is a Senior Associate andDirector of the E-Rate Project at the BentonFoundation.

19

mni

CHAPTER 2s THE E-RATE TAKES HOLD, FUTSLO LY: VDEIN F'q1M FOUR COTDES,11\

BY DONNA HARRINGTON-LUEKER

Two years after receiving their first round ofE-Rate funds, school officials in Chicago,Milwaukee, Detroit and Cleveland havefound that building a multimillion-dollar com-puter network may be easier than helpingschools learn to use that technology well.

That's one of the lessons to emerge frominterviews with administrators, teachers andtechnology researchers about the currentuses of technology in the districts and fromvisits to the districts' schools.

It also comes as the Federal Communi-cations Commission prepares to award$2.25 billion to wire the nation's schools,bringing the total funds awarded to $8 bil-lion. The largest share of those funds, whichcovers telephone service, Internet accessand internal network connections, has goneto high-poverty districts.

For the four cities in this study, E-Rate fund-ing has been a boon. In the first year ofthe program, Chicago, Milwaukee andCleveland were among the top four recipi-ents of E-Rate funds nationwide. AndDetroit, which lagged behind the others inits requests for funds, asked for $68.4 mil-lion this year. That's more than the districtreceived in the three prior rounds of fundingcombined.

The districts have used those E-Rate fundsto build powerful telecommunicationsnetworks that rival or exceed those in moreaffluent suburban districts, but most haveonly begun to address transforming thoseinvestments into gains in teaching andlearning.

The cost of updating electrical systems, theneed to train massive numbers of teachers,

the presence of competing reforms andcontradictory district mandates, the pres-sure of high-stakes assessments: All affectthe way technology is and isn't used inschools in each of these cities.

A further complication: This year, the FCCreceived $5.2 billion in E-Rate requestsmore than twice the $2.25 billion availablefor distribution. Given that gap, in early May,the FCC proposed giving priority for fundsfor internal network connections to districtsthat did not receive such funding last year.That change could jeopardize the progressof all four districts in this study.

WORLD-CLASS NETWORKSAt the district level, the most obvious signsof progress are the new high-speed net-works that crisscross the school systems.

Only two of Cleveland's 118 schools hadInternet access at the time of EWA's firsttechnology report. Today, after three wavesof E-Rate funding totaling $74 million, theschool system has built a web of high-speedconnections that allows the transmission ofvoice, video and data to all schools.

With the exception of rooms in a few olderschools recently reopened, every class-room in the district is wired for the Internet,says Frank Detardo, Cleveland's director ofinstructional technology.

This compares to 60 percent of classroomsin high-poverty districts nationwide in theyear 2000, according to the National Centerfor Education Statistics, and 82 percent ofclassrooms in districts with the lowest lev-els of poverty.

(This chapter, originally published in June 2001, is reprinted with permission. It was produced for the EducationWriters Association as part of a larger report, New Networks, Old Problems:Technology in Urban Schools. This reportwasa follow-up to EWA's Barriers and Breakthroughs:Technology in Urban Schools (1999) and charts both the progress fourschool systems have made in the last two years and the new challenges they face. To measure any differences inteaching and learning, the report also profiled four schools that reporters first visited in 1998-1999. Both reportsmay be obtained for $12 each from the Education Writers Association, 2122 P St. N.W., Suite 201, Washington,DC 20037; (202) 452-9830: ewa@ewa.org. See www.ewa.org for ordering information. The Joyce Foundation hassupported both EWA and Benton's work on the E-Fiate.}

Only 38 percent of classrooms in high-poverty districts were connected to theInternet in 1998, the year before receipt ofthe first E-Rate funds, and 57 percent werewired in more affluent districts.

Milwaukee has made similar gains. Before E-Rate, most schools in the city had dial-upInternet access, and only one-fifth of the dis-trict's 5,000 classrooms had Internet connec-tions. "I'd be constantly saying, 'If we couldgo on the Internet, this is what we'd see,"says James Furness, technology coordinatorat Westside Academy I and II.

As of spring 2001, Milwaukee had connect-ed all schools to a district-wide network ofhigh-speed lines and fiber-optic cable. Sixty-five percent of its classrooms also had beenconnected, and district officials say theyexpect to have all classrooms online in thenext 18 months.

Gains have been slower in Chicago andDetroit. Thanks to E-Rate funds, nearly halfof Chicago's 600-plus schools now have T1lines, says Chanda Davis, the district's E-Rate project manager. In 1999, only 63schools had such access. In addition, about40 percent of the city's classrooms are con-nected to the Internet, including every class-room at the high school level. Still, about athird of Chicago's schools are "flatliners"with no Internet connections at all, saysDavis. These schools will be given priority inthe 2001-2002 school year.

Schools that had Internet connections orschool-wide networks have faced obsta-cles. According to Davis, some schoolswith networks in place before E-Rate had toredo their lines to meet new district stan-dards. Others had to reconfigure newlyinstalled networks because inexperiencedprincipals had no way of judging a contrac-tor's work early in the E-Rate program,Davis says.

But Chicago's size has been the biggestchallenge. "Very few organizations in the

world have more than 640 sites to dealwith," says Davis.

Though smaller, Detroit also has madeslower progress. As of spring 2001, the dis-trict had provided all its high schools andnearly all its middle schools with T1 lines.But less than half of the district's elemen-tary schools have such connections, saysThomas Diggs, the district's technologydirector. At the same time, Diggs says it'sunclear how many of the district's 8,800classrooms are wired for the Internet.

All four districts have spent millions in localfunds to upgrade electrical systems thatcan't support computers, printers and otherperipherals. E-Rate funds cover networkconnections within schools, but not electri-cal upgrades.

With some buildings more than 100 yearsold, Chicago has committed $199 million tobringing the electrical systems in half itsschools up to the minimum requirementsspecified in the E-Rate, the district's officeof operations reports. Those funds havecome from a $2.6 billion capital improve-ment program the district began in 1996.Another 200 schools are targeted for elec-trical upgrades in 2001-2002.

But even with the capital improvementfunds, the district has had to scale back itsinitial plans. According to Davis, 10 roomsin each Chicago elementary school werewired for network connections in earlyrounds of E-Rate funding. But next year thedistrict can only fund electrical upgrades forthree rooms in each school, Davis says.

In Milwaukee, technology director BobNelson estimates that one-third of theschools require upgrades to support theincreased demand for electricity at a totalcost of $31 million.

Districts have found, too, that powerful net-works can stall when classrooms lack suffi-cient hardware. Thanks to Ohio's School Net

21

program, which has targeted elementaryschools throughout the state, Cleveland's ele-mentary schools have between four and fivecomputers in most classrooms. But hardwareis less plentiful in the district's middle and highschools, say teachers and technology spe-cialists familiar with the district.

Lack of hardware has limited what teacherscan do with the district's high-speed net-work. "It's a great leash, but there's nodog," says Shane McConnell, chair of theEnglish department at Cleveland's EastTechnical High School.

ONLINE CONTENTAs the four districts put new technologies inplace, other trends are emerging. For thefirst time, most of the districts have begunto provide online resources such as ency-clopedias and information databases via dis-trict-wide networks. With more schoolsand more teachers having access to theInternet, the districts also have put local andstate learning standards, sample lessonplans and advice on integrating technologyinto the classroom online.

In each district, too, there are efforts to spurschools to use the new technology for Web-based projects that incorporate criticalthinking and high academic standards.

Chicago has invested more than $1 millionin developing a suite of Web-based toolsthat teachers use to create projects stu-dents use online. The tools are part of thedistrict's Technology Infusion Planning pro-gram, or TIP, an effort to train teachers tobetter use the Internet. The program cur-rently reaches more than 200 Chicagoteachers.

The projects, which address specific districtlearning standards, vary by grade and sub-ject. A third-grade unit on the solar systemasks students to select a planet and thencreate an online brochure that would per-suade someone to become a colonist. To

complete the assignment, students consultWeb sites for information about the sizes ofthe planets, their distance from the sun,their geography and their natural resources.Once students have collected information,they post what they've learned online.

Another project on global warming asksseventh- and eighth-grade students to usePalm Pilots equipped with probes to collectair temperatures in the schoolyard.Students then enter their data into graphingsoftware and compare their findings withreal-world data on global temperature shifts.

The goal in each project is to encourage stu-dents to investigate a topic on their own,using unique resources available on theWeb, and to encourage teachers to look dif-ferently at the content of lessons and theway they assess their students' work.

Bernard Bradley, a science teacher andtechnology coordinator at NewberryAcademy, a K-8 public school on Chicago'sNorth Side, exploits just such resources inhis global-warming unit. A mentor for theTIP program, Bradley contacted a number ofscientists via e-mail, asking the researchersif they would provide copies of their data forhis students to use. All agreed, and thatdata became the core of Bradley's unit.

This year, the teacher has developed a sim-ilar project involving the effect of variousenvironmental factors on the developmentof frog eggs.

In both units, he emphasizes inquiry andexploration rather than lecture. "The kidsjust come into the lab and go to work," theteacher says. Working in groups, studentshave access to 16 iMacs, all connected to aT1 line, and to four iBooks with accompany-ing sets of lab probes.

With new networks capable of transmittingaudio and video as well as data, the districts

22

also are developing curricula that use video-conferencing.

Currently, six Cleveland middle schools areworking with the Ohio Consortium forConceptual Learning (OCCD, which hasreceived $6 million from the state to devel-op units that target specific Ohio-learningproficiencies. Schools in Akron, Columbusand Cincinnati also are involved.

The Cleveland schools are piloting an eight-week unit on fitness and nutrition thatconnects students with doctors and medicalresearchers at teaching hospitals in thearea. Next year, 41 new schools will jointhe statewide consortium, including 18

Cleveland high schools, says GeoffAndrews, the OCCL's director.

At the same time, sChools in each district arealso using their networks to deliver programsthat target basic skills. Nine schools in

Cleveland are working with CompassLearning, a commercial program that coversthe basic math and literacy skills studentsneed to do well on Ohio's proficiency exam.Sixth-graders in the pilot schools use thesoftware for between 120 and 160 minutes aweek. In Chicago, 14 schools are pilotingFast For Word, a networked reading program.

Still others acknowledge that in the face ofpressures to improve test scores, they'vesimply put technology on the back burner."Books are just more important," saysBruce Allman, principal of Chicago's ElizaChappell School. Eleven of Chappell'sclassrooms have been connected to theInternet as part of the district's ongoing cap-ital improvement plan, but the connectionswere not something the pre-K-8 school hadsought, says Allman. And the principal sayshe does not plan to increase the school'scommitment to technology. Chappellalready has a 36-station computer laborato-ry and a minimum of one computer in everyclassroom.

"We're still worried about reading and doingmath at national norms," says Allman.Forty-nine percent of Chappell's studentsscore at that norm in reading, and 67 per-cent do so in math.

TEACHER TDOG,A

With considerable help from federal and pri-vate sources, the four districts are workingon new strategies for providing professionaldevelopment.

Cleveland is part of Alliance+, a Web-basedprogram that helps teachers use Internetresources in the classroom. Funded by theU.S. Department of Education's TechnologyLiteracy Challenge Fund, Alliance includes10 weeks of workshops geared to ateacher's grade level (elementary, middle orhigh school), with each workshop address-ing progressively more difficult applicationsof the Internet.

Elementary teachers begin with an introduc-tion to the World Wide Web and to the ideaof using telecommunications technology inhands-on science classes. During that intro-ductory lesson, they also link to the ScienceLearning Network, an online communityfunded by the National Science Foundation,and to schools across the country that areinvolved in projects that require the sharingof data. Still other Web links take teachersto museums, including San Francisco'sExploratorium, where archived webcastsprovide access to portable laser light showsand fiber-optic sculptures.

In subsequent weeks, teachers learn to setup an e-mail account on Yahoo and investi-gate Web sites such as the Volcano Worldor the Rainforest Connection. (The formeris supported by NASA; the latter by theSmithsonian Institution.) Other workshopsdiscuss how to incorporate real-timeInternet resources, such as databases withupdated sea-surface temperatures, into les-son plans.

2 3

In the third year of a five-year grant, Alliancehas trained 820 Cleveland teachers, mostof them at the elementary school level. Aseparate grant from the Joyce Foundationprovides an additional 30 hours of trainingfor middle-school teachers.

Federal funds have driven professionaldevelopment efforts in Milwaukee. With $4million from the Department of Education'sTechnology Literacy Challenge Fund, thedistrict worked with Marquette University todevelop a three-credit online graduatecourse on integrating technology into thecurriculum. According to district officials,more than 450 teachers have completedthe course.

The district also received $1.9 million fromthe federal government's PreparingTomorrow's Teachers, or P13 program,which targets students in teacher prepara-tion programs. As part of that grant,Milwaukee is developing a video library ofbest classroom practices that can be deliv-ered over the Internet and is pairing studentteachers with tech-savvy teachers. The dis-trict is also working closely with facultymembers in schools of education.

To help teachers learn to use technology,Milwaukee schools also have used Title Ifunds to hire technology specialists.

But while teacher training is critical, it's alsotime-intensive. "It's like a graduate coursein educational measurement or disruptivebehavior," says Jonathan Fairman, a highschool English teacher and school technolo-gy coordinator who works with the Allianceprogram. "You'll do six hours of homeworkfor every three hours of coursework."

And even that commitment might not beenough. A preliminary evaluation of theAlliance program by _researchers at KentState University showed that teachers with30 hours of training didn't use computers

more frequently than teachers with no train-ing. Teachers who had received 60 hours oftraining, plus other support, reported morefrequent use of computers in the class-room.

Ongoing help at the school site is essentialas well, says Jim Sweet, Chicago's directorof online learning. "We assumed initiallythat once teachers had the connection andsome basic skills, they'd use the technolo-gy, but that wasn't the case," says Sweet,who came to Chicago from the NationalCenter for Supercomputing Applications atthe University of Illinois-Urbana.

"When you look at the research on innova-tion and effective instruction, the answeralways is tools, training and follow-up,"Sweet says. As part of the TIP program, acadre of seven experienced technology-using teachers visit participating schoolsregularly to coach other teachers and helpthem refine the online lessons they'vedeveloped for the program.

Even then, Sweet acknowledges, not allteachers complete their lessons, and not alllessons make the most effective use oftechnology. Further, the seven mentors canonly reach a small number of Chicagoteachers.

MAKING TECHNOLOGY COUNTOther challenges remain. Central offices ineach of the districts continue to define therole they must play in deploying and main-taining school technology. Detroit has hireda private firm to oversee technology servic-es the district previously provided, includingcomputer repair and maintenance. AndChicago has developed a purchasing pro-gram that takes advantage of the district'sability to buy in bulk and offers computers toschools at reduced rates from designatedvendors. But schools still report frictionover the central office's need to standardizehardware and network connections and

24

their need for flexibility to grow their ownprograms. Milwaukee, too, is piloting aninnovative technology advisory board com-prising technology experts from across thenation.

A number of lighthouse schools, whoseprincipals have proven themselves adept atleveraging school funds and writing grants,also continue to rocket ahead of otherschools, accumulating even more resourcesthan they had two years ago.

But such progress often demands persist-ence. "I'm usually a little hard to turndown," says Linda Pierzchalski, principal ofChicago's William J. Bogan ComputerTechnical High School, chosen this year bya national magazine as one of the nation'smost-wired schools.

Bogan, which has a poverty rate of about 83percent, received $600,000 in year two ofthe E-Rate program, and Pierzchalski saysthat the funding moved the school forwardquickly with its network. (Before E-Rate, theschool relied extensively on Title I funds forits technology program and had to deployits use of technology in stages.)

Another of Pierzchalski's strategies: agree-ing to become a test site for various tech-nology programs. "My middle name ispilot," the principal says. "I simply tell peo-ple: You can try it here, but we're not payingfor it."

But effective technology leadership remainselusive in many urban schools, advocatesand others say. Districts and schools strug-gle with the technical demands of maintain-ing sophisticated high-speed networks, atask that is much more complex than keep-ing a stand-alone computer running or fixinga printer that has jammed. In most of thesedistricts, the job of keeping the network run-ning falls to school-level technology coordi-nators or technology teachers, virtuallynone of whom is a network specialist.

The job can be both time-consuming andfrustrating. One Chicago principal notesthat his technology coordinator a teacherwith a master's degree in instructional tech-nology has become "an electronic janitor"who spends her time keeping the serversrunning.

A technology coordinator at a Milwaukeemiddle school reports the same pressure."It's not as much fun as it used to be," shesays. "I spend a lot of time alone in a noisy,hot little room and less time with teachers."

Perhaps most tellingly, while the access totechnology has increased in each of the dis-tricts, so have competing pressures. Stateaccountability tests, an array of reform ini-tiatives, the fear of being put on a warninglist, can all push technology onto the backburner.

Combating skepticism about yet anotherreform is a challenge. "Teachers and admin-istrators don't really believe that technologyis the answer to their problems," saysElliot Soloway, a University of Michiganresearcher who has worked in Detroit andother urban systems.

But until they do, the billion-dollar networksE-Rate has put in place are in danger ofbecoming the latest, and perhaps mostexpensive, unused instructional technologyto be put in schools.

Donna Harrington-Lueker is a freelance educationwriter and an assistant professor of journalism atSalve Regina University in Newport, RI

25

CHAPTER 3: NEW APPROACHES TO ASSESSINGSTUDENTS' TECHNOLOGY-BASED WORK

BY MARGARET HONEY

After more than two decades of research onthe benefits of educational technology, evi-dence that demonstrates the positiveeffects technology can have on studentachievement is mounting. Specifically, stud-ies have shown that:

Large-scale statewide technologyimplementations have correlated use oftechnology with increases in students'performance on standardized tests.

Software supporting the acquisition ofearly literacy skills including phonemicawareness, vocabulary development,reading comprehension and spellingcan support student learning gains.

Mathematics software, particularlyprograms that promote experimentationand problem solving, enable students toembrace key mathematical conceptsthat are otherwise difficult to grasp.

Scientific simulations, microcomputer-based laboratories and scientific visuali-zation tools have all been shown toresult in students' increased under-standing of core science concepts.

We have also learned that if technologiesare to be used to support real gains in edu-cational outcomes, then five factors mustbe in place and working in concert.

1. There must be leadership aroundtechnology use that is anchored in solideducational objectives. Simply placingtechnologies in schools does little good.Effective technology use is alwaystargeted at specific educational objec-tives; whether for literacy or sciencelearning, focus is the key to success.

2. There must be sustained and intensiveprofessional development that takesplace in the service of the core vision,not simply around technology for itsown sake; moreover, this developmentmust be a process that is embedded inthe culture of schools.

3. There must be adequate technologyresources in the schools, includinghardware and technical support to keepthings running smoothly.

4. There must be recognition that realchange and lasting results take time.

5. Finally, evaluations must be conductedthat enable school leaders and teachersboth to determine whether they arerealizing their goals and to help themadjust their practice to better meetthose goals.

PLANNING FOR MEANINGFULTECHNOLOGY INTEGRATIONThe singular and dominant benefit of tech-nology in education is its capacity to sup-port all aspects of the learning process fromthe classroom to the central office. Weknow that technologies offer teachers andstudents opportunities that would otherwisebe extremely difficult to realize in classroomcontexts. School reformers generally recog-nize four areas as centrally important toimproving student achievement: assess-ment, information access, collaboration andexpression. Educational technologies havebeen shown to demonstrate particularpromise in all four.

There is much work to be done, however,before we can realize this promise.Educators are only beginning to take advan-tage of the genuine potential of educationaltechnologies. As the preceding chaptersmake clear, the E-Rate has enabled schoolsto make substantial progress in buildingtheir technical infrastructures; schools mustnow decide how this technology can bestbe used. The challenge is twofold educa-tors must embraoe a coordinated vision oftechnology integration that capitalizes ontechnology's unique affordances, and con-struct a coherent approach for their schoolsand districts to integratepe various piecesof this vision. This meanSdeciding what stu-dents should be learning with technology,and then applying those learning goals to0 fpis 0

every level of the educational processfrom professional development and curricu-lum design to classroom implementationand student assessment.

In order to improve current teaching strate-gy with technology, for example, ongoingprofessional development must beaddressed. Recent national surveys indicatethat the increased technology infrastructurein schools is not being matched withincreasing, or even adequate, professionaldevelopment for teachers. Although"teacher usage and skill level has increasedin the past year ... only 8 percent of schoolssay that the majority of their teachers are atan advanced skill level and able to integratetechnology use into the curriculum. Morethan 45 percent of schools report that over50 percent of their teachers are at the inter-mediate skill level able to use a variety ofcomputer applications but not adept at inte-grating technology into the curriculum."

Curriculum design is another area in whicheducators are only beginning to connecttheir effort to a larger vision of technology'srole. Current basic teaching strategy doesnot yet take advantage of the extent towhich technology can add substantive learn-ing learning that goes beyond the rotepractice of basic technology skills and drills

to classroom curriculum. Teachers gener-ally do not view technology as a means toexpand their curriculum beyond basic skillsinstruction, but simply as another means a

requirement, in fact to accomplish thebasic work they are already doing. Theytend to see the technology componentsthey have added to curricula as "motiva-tional" and "skill-building," but have noclear standards or expectations for evaluat-ing how well a student uses particular tech-nology tools.

Relatively few teachers who regularly usetechnology in their curriculum make use ofanalytic and project-oriented software, andthe few who do use interpretive software

use it only infrequently. Instead, they useprograms such as Power Point or other mul-timedia authoring tools to support work thatis not significantly different from what theywould be teaching without the technologythat is, to conduct research, write papers,make oral presentations and so on. Whilethere is evidence that use of such programsis indeed motivational for students, we haveyet to realize the potential of these tools tosupport the acquisition of critical 21st-cen-tury skills such as information literacy aswell as the ability to communicate effective-ly and think critically. Multimedia software,for example, offers an expanded, morecomplex arena than pen and paper in whichstudents must constantly self-edit theirideas and their presentation of the ideas.There is thus a clear need for new curricu-lum that accesses the unique learningopportunities that commonly used technolo-gies can offer to students.

As they attempt to address these new chal-lenges in technology integration, schoolsand districts also face the demands of grow-ing accountability measures as well as align-ment with local, state and national stan-dards. Determining how to use technolo-gies well and deeply to support richer learn-ing and teaching is a tall order unto itself;having to meet the requirements of testingand standards raises the stakes even high-er. These growing accountability pressures,particularly in urban environments, often rel-egate technology to the role of enrichmentrather than viewing it as a resource that cansupport core teaching and learning goals.We know that continuous improvement, aswell as regularly measurable informationthat marks improvement, are two elementsof effective technology use for schools thatwould speak to these new challenges.Actually designing strategies to yield theseelements, however, remains an elusive goalfor most school systems.

Between September 2000 and June 2001,the Benton Foundation and the Center for

27

Children and Technology (CCT) establisheda collaboration with two Joyce Foundationcities Chicago and Milwaukee to helpthe districts develop locally relevant butbroadly generalizable programs of workaround the coordinated use of technologyto support student learning. Because of itsunderlying importance to areas of educa-tional improvement, we chose evaluationresearch to organize the ways in which cre-ate-a-vision technologies can be used tosupport high-quality teaching and learning.

Researchers from CCT focused on improv-ing basic teaching strategy by developing anevaluation toolkit that would complementwork already taking place in the districts,while at the same time seeking to raise theteaching and learning bar. We did thisthrough joint work on the creation of anincreasingly sophisticated framework forthe use of technology. We sought to build auniversal toolkit that would work acrossmultiple content areas, and that would moveteachers and their students toward anincreasingly sophisticated perception ontechnology use. By providing teachers witha sense of beginner, novice and expertstrategies, the toolkit sets forth a develop-mental trajectory for the process of learningto use technology well.

We also designed a framework to be usedby districts in teacher professional develop-ment programs, making it possible to alignthe toolkit with support for teachers at vary-ing levels or phases of integrating technolo-gy into the classroom. Finally, we wanted todevelop a tool that was responsive toaccountability measures; the toolkit is struc-tured to yield information on student learn-ing that teachers can put to immediate use,and that district administrators can usemore broadly to determine whetherstudents across the district are on track.

RECENT WORK IN TECHNOLOGYA recent report released by the CEO Forumon School Technology and Readiness

makes recommendations to ensure that thenation's investment in education technologyimproves student achievement and benefitseducation. Among these are four recom-mendations that speak directly to theapplied research and development work weundertook in this collaboration.

Focus education technology investmenton specific educational objectives.

Make the development of 21st-centuryskills a key educational goal.

Align student assessment with educa-tional objectives and include 21st-century skills.

Adopt continuous improvement strate-gies to measure progress and adjustaccordingly.

In addition to the CEO Forum report, anumber of other reports have underscoredthe importance of developing 21st-centuryskills. The North Central Regional Edu-cational Laboratory, together with MetiriGroup, has developed En Gauge, a Web-based framework addressing the value oftechnology. This framework defines 21st-century skills as digital age literacy, inven-tive thinking, effective communication andhigh productivity, among others. Our evalu-ation toolkit addresses several elements ofEn Gauge's digital age literacy, as describedbelow:

Basic Literacy: The definition of basicliteracy reading, writing, listening andspeaking has expanded in the digitalage to include media other than paperand pencil. Literate people must be ableto read, write, listen and speak fluentlythrough the use of text, images, motionvideo, charts, graphs and hypertextacross a range of media.

Visual Literacy: Students must be ableto communicate through visuals; theymust, in other words, be able todecipher, interpret and express ideas

()8

through images, graphics, icons, charts,graphs and videos.

o Information Literacy: Literacy withinformation entails research with onlineresources, as well as an understandingof the need for critical evaluationand appropriate application of suchresearch.

Finally, the U.S. Department of Education'srecent report on e-learning argues that stu-dents must acquire the skills needed to par-ticipate in an increasingly technological soci-ety. The report states, "[a] meaningful, uni-fied approach to providing students with theskills they will need for their futures must bemore than a checklist of isolated technologyskills; rather, these skills are only a first stepin assuring all our children become profi-cient information and technology users."

In our work in Chicago and Milwaukee, wesought to map out the next step in providingsuch digital age skills. We have focused onfive core areas of technology use: Internetresearch; data collection and representation(e.g., spreadsheets and graphing tools); livepresentations; Web design; and illustratedreports. These areas are among the tech-nology uses most commonly employed inclassrooms. Yet while educators areincreasingly teaching students how to usesoftware in these areas, they are just begin-ning to think beyond providing "isolatedskills" and toward making technology usecompatible with their more fundamentalgoals of promoting literacy, critical thinkingand sound communication skills. Our intenthas been to push this thinking forward.

DEVELOPING THE TOOLKIITOur district partners in Chicago andMilwaukee are similarly concerned with thequestion of how to move to the next level oftechnology integration. Milwaukee's direc-tor of technology, Bob Nelson, andChicago's chef officer of learning technolo-gies, Richard White, are eager to pursue

new strategies for assessing the broadimpact of technology integration on studentlearning in their districts. Their reasons mir-ror those of educational policymakers at alllevels: they want to measure the impact oftheir technology investments but have fewtools to do so.

Assessment tools such as norm-basedstandardized tests, the currency of educa-tional "accountability," were not developedto assess the skills and knowledge thattechnology-based instruction can bring tostudents. Student outcomes from suchassessments drive district, state and nation-al education policy in many arenas tech-nology implementation is one of these.Although these tests can effectively assessa set of narrowly defined skills, they do notaddress the kinds of learning taking placewhen classroom technology is used to itsmaximum potential. Thus, finding new waysto identify learning goals for technology andto assess progress toward those goalshave been of paramount importance toNelson and White.

An important component of our work withthe districts has been paying attention tolocal, state and national technology stan-dards. National standards, such as thosecreated by the International Society forTechnology in Education (ISTE), state pack-ages in Illinois and Wisconsin, and localMilwaukee and Chicago standards, wereamong the materials we referenced. Thesepackages all include useful thinking aboutthe technology skills students shouldacquire, and provide age-based bench-marks for student achievement with tech-nology. In some cases, standards packagesprovide teachers with supplemental materi-als, such as sample lesson plans and cross-referencing with curriculum standards in

other subject areas. Yet standards provideonly one piece of the puzzle.

The challenge facing the standards-writersis obvious because standards must be

2 9

3

ME13

applicable to all kinds of student work, andacceptable to all kinds of audiences, theycan suggest a direction but cannot map outa path. On the subject of "TechnologyCommunications Tools," for example, theISTE standards suggest that students beable to "use a variety of media and formatsto communicate information and ideaseffectively to multiple audiences." In thesame area of "communicating information,"Wisconsin's state standards for Informationand Technology Literacy provide a long listof skills that students should master "by theend of grade 8," among them the ability to"plan and deliver a presentation usingmedia and technology appropriate to topic,audience, purpose or content."

While prescriptions like these help to estab-lish goals for technology use, it is still up todistricts and teachers to create the meansof reaching those goals. In working with theChicago and Milwaukee school districts, wesought to move these general guideposts toa concrete level to help teachers andadministrators determine where studentsare in their technology use, give teachers aneasy and flexible strategy for defining clearlearning objectives and provide concreteexamples of what such objectives wouldlook like in students' work.

The task was to distill some of our knowl-edge about the unique affordances of tech-nology (and to gather new knowledgethrough collaboration with teachers and dis-trict personnel) into a set of professionaldevelopment tools for teachers a class-

room-based evaluation toolkit for assessingthe impact of technology-rich activities onskills development, student learning andmedia literacy. This product would buildupon the standards' prescriptions for gener-al technology use by helping teachers deter-mine how and why specific types of technol-ogy tools are useful for promoting certainskills.

Because the toolkit starts with evaluation,rather than curriculum development, it servesmultiple needs. First, it provides a means ofidentifying educational objectives for tech-nology use in classrooms. Getting teachersthinking about assessment may be theclearest avenue toward helping them identi-fy and articulate their goals for teaching withtechnology. The question "What would I

want to see in a good piece of student workwith technology?" necessarily leads to thequestion "What do I want my students to belearning with technology?" Second, thetoolkit aids in aligning learning objectiveswith corresponding assessments. Byemphasizing the identification of learninggoals that must underlie lesson planning,focusing on appropriate assessments candrive good curriculum design. Finally, andmost concretely, the toolkit offers a way tomeasure students' progress in technologywork.

Margaret Honey is Vice President of theEducational Development Center, Inc., andDirector of the Center for Children andTechnology.

30

CHAPTER (3.g ASSESSMENTS OF STUDENTS'TECHNOLOGY-BASED WORK:

AN OVERVIEW OF THE TOOLKIT AND CASE STUDIES

BY ANDY GERSICK, CONSTANCE KIM, JULIE THOMPSON KEANE,WENDY FRIEDMAN, KATIE CULP

The evaluation toolkit, a set of instruments skill areas: digital skills, media and meaning,that help teachers take advantage of tech- point of view and audience.nology affordances in their teaching, wasdeveloped in collaboration with teachers and The first category, digital skills, covers theadministrators in Chicago and Milwaukee. mechanics of technology use. The role ofThis chapter presents the content and struc- digital skills is like that of grammar, spellingture of the toolkit, and case studies that and vocabulary; they are the basic means ofillustrate how the ideas contained therein production for multimedia work. A studentaffected technology use in our collaborators' must have the digital skills to manipulateclassrooms, information within a given program cutting

and pasting, formatting text, importingTHE TOOLKOT: CONTENT graphics in order to make a coherent com-The objective when creating a multimedia position. Without knowledge of how to useproduct is no different than with any other software, students cannot read or composeform of communication: A product created technology products, but attaining digitalwith technology should communicate an skills is a means to the end of communica-idea or a set of ideas. Thus, the basic ques- tion, not an end in itself.tions to be addressed in the product arecommon to all forms of communication: Once these basic tools of communication

have been acquired, the author is ready too What is the message being communi- consider her message and the most effec-

cated? tive way to convey it. Having learned the

o Is the message clear? strengths and limitations of the availablemedia options, the author can choose a

o Who is the sender? medium that most enhances her message,o To whom is this message directed? thereby developing the capacity to bringo Who is the receiver? media and meaning together. In this catego-

ry of skills, the author learns how to conveyideas in the various forms made available byAlthough these questions may seem like antechnology.obvious starting point, they are often

obscured by considerations of mere techni-Choosing media is also an important way forcal proficiency. Communicating well withthe author to express her point of view. Alltechnology demands that students knowdecisions about media including decisionsmore than simply "how to use" software.about text, visuals, sound effects and over-They must understand how to take advan-

tage of its unique benefits; to convey ideas all presentation format are directed by theauthor's opinion and intent. Thus, in choos-

via an integrated mix of sound, visuals, ani-ing the most effective media for all aspectsmation, interactivity and all the other ele-of communicating her message, the author

ments that technology, tools make availablemust be conscious of her own opinion andto them. The toolkit provides a frameworkthe differing effects of specific media on her

with which educators can juggle, understandm

and evaluate the elements of teaching and essage. The author must, in other words,recognize herself as the sender of her ownassessing student work with technology,

without losing sight of core learningpersonalized message and make decisions

objectives, regarding media that uphold the integrity ofthat specific perspective. Creating a live

To organize the myriad facets of technology presentation that gives equal weight tosounds, picture images and text, for exam-use into a manageable perspective, theple, lends a decidedly different tone to an

toolkit divides technology use into fourba%"icauthor's message than a text- and chart-

31

heavy live presentation does. Dependingupon how the author chooses to use andcombine different media, she can empha-size, strengthen or undercut a given view-point.

Finally, in order to make an impact, theauthor must consider the audience for hermessage. While it is important to choosemedia for its compatibility with content andthe author's point of view, it is equally impor-tant to choose a medium that will communi-cate with its receiver. Thus the author mustconsider which media would be most effec-tive and compelling for the components ofher message as they relate to particularaudiences. For example, streaming videowith fast pop music in the backgroundwould be more effective when presenting tohigh school students than when presentingto the parents of those students.

Within each of these skill areas, the toolkithelps teachers teach and evaluate concreteskills that students must master to be effec-tive communicators.

THE TOOLKIT: STRUCTUREThe toolkit comprises a multi-layered portfo-lio of interrelated tools, including a technol-ogy affordances matrix, conceptual frame-works, checklists and sample assessmentsof student work for five categories of work:web design, data collection and representa-tion, information gathering, live presenta-tions and illustrated reports. The tools forlive presentations were developed and com-pleted during the span of our work withteachers in Chicago and Milwaukee.

Each component of the toolkit serves adistinct purpose.

m Technology Affordances Matrix: entry

point to the rest of the toolkit. The matrixlists the most common forms of studentwork with technology, and broadly intro-duces the learning opportunities thateach form affords students.

o Conceptual Frameworks: diagnostictools. For each type of work (i.e., illus-trated reports or Internet research), theconceptual frameworks divide each ofthe four skill areas (digital skill, mediaand meaning, point of view andaudience) into basic, intermediate andadvanced skills, and provide essentialdescriptions of each skill level. Teacherscan use the frameworks as roadmaps todetermine where their students are interms of their technology use, and whatkinds of skills they must learn tobecome more sophisticated users.

Checklists: The "meat" of the toolkit.

While the conceptual frameworksprovide general descriptions of skillareas, the checklists give teachersconcrete descriptions of specific skillsfor students to learn. For example,where the frameworks say broadly thatan intermediate-level live presentationwould be one in which "each medium(text, images, sound) brings somethingdistinct to the meaning of the presenta-tion," the checklists direct teachers tolook at every component of a livepresentation, such as spoken narrative,text, illustrations and sound effects.The checklist is a flexible tool; teacherscan pull from the checklists whenplanning lessons or when designingrubrics to assess student work.

Sample Assessments of Student Work:A user's guide to app/ying the tools. Thesesample assessments walk teachersthrough a step-by-step process of usingthe frameworks and checklists toassess a sample piece of student work.

Working together, the four pieces of thetoolkit can guide a teacher through theprocess of planning, implementing and eval-uating technology-based lessons. She candecide what medium to work with using thetechnology affordances matrix; locate herstudents' level of sophistication on the con-ceptual framework; devise a lesson plan

3 2

referencing the checklists and pull from thechecklists to create a grading rubric; anduse the sample assessment as a guide toevaluating her students' work.

CASE STUDOES: COLLABORATORGWITH TEACHERS ON CD-OOCAGOAND MILWAUKEEThe toolkit was developed through a collab-orative process of work with teachers andadministrators in Chicago and Milwaukee.Center for Children and Technology (CCT)researchers began by conducting class-room observations to assess partner teach-ers' current levels of technology use. CCTand partner teachers then collaboratedthroughout the school year to develop tech-nology-rich classroom projects that wouldincorporate the concepts we planned tobuild into the evaluation toolkit. CCT andpartner teachers held frequent discussionsabout the format and content of the toolkit,and about how the core ideas were playingout in classes. Researchers also conduct-ed secondary classroom visits to helpteachers implement technology projectsand to observe teachers' evolving class-room practice as they grappled with newtechnologies and new goals for technologylearning.

Both Chicago and Milwaukee are large,complex districts, where administratorsmust consider and manage concerns rang-ing from standardized testing pressures tobudget shortages. Both have pursued sig-nificant amounts of E-Rate and other tech-nology funding. Both are in the thick of anevolving process of technology implementa-tion, charged with simultaneously develop-ing plans to build and maintain infrastruc-ture, creating technology-related profes-sional development, tracking the impact onstudents and justifying all of these under-takings to both local constituents and statepolicymakers.

MILWAUKEE: The Milwaukee Public School(MPS) system is the thirteenth-largestschool system in the country, servingover 105,000 students in 160 schools.Milwaukee's High Standards reformincludes eighth-grade completion stan-dards, benchmark content standards foreach grade, district performance assess-ments and school improvement plans.

CCT's work in Milwaukee involved two col-laborators: the Wisconsin Conservatory forLifelong Learning (WCLL) and the JohnAudubon Middle School (Audubon). WCLLhas an enrollment of 980 students in pre-Kthrough twelfth grades. Of those, 71 per-cent receive free or reduced lunch, and 15percent have exceptional education needs.According to WCLL, "Technology is thekernel from which much of the teaching andlearning that occurs at WCLL emanates."Every WCLL classroom is equipped withPower Macintosh computers. The schoolalso boasts interactive video (distancelearning) and school-wide broadcastingcapabilities, including a technology-enrichedcomputer suite, and an InformationResource Center with numerous fiber dropsand computers. Currently, there are twoprimary sets of standards that shape theircurriculum development process the MPSK-12 Teaching and Learning Goals.

The Audubon school is a board-approvedspecialty school in the area of communica-tions. According to the school's missionstatement, "Communications Technology isa key element of Audubon's goals, its stan-dards, assessments and its curriculum,which are driven by the District's HighStandards." Audubon has three computerlabs, with various software programs thatsupport desktop publishing, computer-aideddesign, digital photography, graphic arts,multimedia and other forms of technology-based activities. Audubon is additionallyequipped with two video (distance learning)rooms, a math and science technologyresource lab and a video production lab.

33

CHICAGO: With approximately 500,000students enrolled in 596 schools, ChicagoPublic Schools (CPS) is the fourth-largestschool district in the nation. With 86 per-cent of its students coming from low-income families," CPS has been one of thelargest recipients of E-Rate funds, totalingover $75 million. To ensure that teachersand students benefit from these largeinvestments in the technology infrastruc-ture, the district's Department of LearningTechnologies is providing professionaldevelopment opportunities that support thealignment of technology standards to theCPS framework and to Illinois LearningStandards.

The Center for Children and Technology col-laborated with two schools in Chicago, theNorth Kenwood Oakland Charter School(NKO) and the Nathanael Greene Elemen-tary School. Each school was selected withthe help of district administrators. TheGreene school has an enrollment of 770students. Of those, over 95 percent areHispanic and over 80 percent of the studentbody receives free or reduced lunch. TheGreene school had a pre-existing relation-ship with CCT: last year, the school collabo-rated with CCT on a Joyce Foundation-funded project to develop a TechnologyTraining Center for one of the district'sregional divisions. Greene teachers act asmentors to visiting teachers and demon-strate technology integration in the class-room. The Training Center provides teach-ers the opportunity to see technology inte-gration in real classrooms with real teachersand students. The Greene school has twonetworked PC and Macintosh G3 labs.Each classroom is equipped with one net-worked teacher station and four Oraclecomputers.

The North Kenwood Oakland School (NKO)has an affiliation with the University ofChicago's Center for School Improvement.NKO is a K-8 school with 306 studentsenrolled. Over 95 percent of the student

body is African-American, and of those, 62percent receive free or reduced lunch. Theschool has developed a technology planthat will fully integrate technology with itscurrent project-based curriculum. Eachclassroom is equipped with four or five net-worked PCs, and the school has one net-worked PC lab.

TEACHERS' CHALLENGES ANDPOINTS OF VIEWLike teachers nationally, our partners atAudubon, Greene, the North KenwoodOakland School and the WisconsinConservatory for Lifelong Learning share abelief in technology's potential to bring newlearning opportunities to the classroom.They note that when using technology theirstudents are more "motivated" and theyhave a general sense that technology is a"powerful" learning tool. As is so often thecase, however, going beyond this feeling toidentifying and articulating where that powerlies and how it is best tapped was a strug-gle for all the teachers in this project.Without the time or training to think in detailabout teaching with technology, few hadused the resources available to them, andmany tempered their faith in technology'spotential with a healthy skepticism about itspractical value.

"Why should I spend time teaching my kidsto do something on the computer that theycould do just as well the old way withposterboard," one teacher asked. It is aconcern for most teachers, and it illumi-nates the current state of technology use inschools. The challenge facing teachers isnot how to teach content with technologythey know how to do this just as they knowhow to teach content through traditionalmedia but how to identify and develop theunique skills associated with new media andtechnologies.

What attracted teachers to this collabora-tion was the prospect of developing toolsthat would help them and other teachers

3 4

make technology meaningful in their class-rooms. Specifically, what could technologybring to the learning experience that wasunique to technology use?

The question for our collaborators was notwhether students should be using thesetechnology tools, but how educators canmake technology use compatible with theirdeeper goals of promoting literacy, criticalthinking and sound communication skills.To begin to move in this direction, teachersneed help guiding their students past thebells and whistles of technology tools thenovelty that can motivate students in theshort term but does not contribute to realsustained learning over time. The justifica-tion for technology in the classroom lies notin simply teaching students to use technolo-gy, but in teaching them to use it well. Withthat step, classroom technology could blos-som from an expensive distraction to a vitalpart of the learning experience.

COLLA ORATEION ON THE CLASS-00MS

In repeated planning discussions, teachersand CCT researchers chose a number ofclassroom projects that were consistentwith teachers' content goals for the schoolyear and would also provide opportunitiesto work with technology tools. These proj-ects spanned a wide range of work, includ-ing science experiments and class presen-tations, oral reports on history, onlineresearch and web page creation. In thecourse of these projects, students usedInternet search engines, word-processingsoftware, drawing and oral presentationprograms, spreadsheet software and webdesign programs the same tools thatresearch indicates are the ones most com-monly employed in classrooms.

For some of our collaborators, fixed curricu-la and testing pressures allowed time foronly one major project incorporating newconcepts about technology. These teach-ers went through a process of talking to

researchers about their own emerging goalsfor technology use, working to incorporatenew ideas from the evaluation toolkit intotheir project work with students, and con-ferring afterwards with researchers aboutthe changes they saw in student work, andabout the challenges they faced duringthese first attempts to enhance their teach-ing with technology. Their experience isbest exemplified by the work of "TabithaJames," a sixth-grade science teacher.

Ms. James had spent a summer workingwith students in a technology-rich environ-ment, and was motivated by the work herstudents had done there and by her ownexperiences teaching with these new tools.All of the students she worked with overthat summer had in the end achieved profi-ciency, according to the MPS standards,and Ms. James strongly believed that theuse of technology within this program waslargely responsible for their level of suc-cess. She did not, however, have an explic-it sense of how technology had supportedher summer students, or how that experi-ence could be replicated.

In Ms. James's school, teachers andresearchers decided that our work wouldcenter around the sixth grade's annualscience fair project. The semester-longproject engaged students with the scientificmethod by requiring them to design andcarry out a controlled experiment. In thepast, students wrote reports on their exper-iments and presented them to the classusing a posterboard presentation design.This year, technology would be integrated inthree areas of the work:

o Research: Students would use Internetresearch to identify an area for investi-gation and to develop a hypothesis fortesting.

o Data Collection and Analysis: Workingwith their math teacher, students woulduse spreadsheet software to track andchart their data.

G=D

Presentation: Students would developand deliver multimedia presentations oftheir research and findings to class-mates, teachers and parents.

The content of this project remained thesame as in years past students carried outtypical science fair experiments on therelative growth rate of bean plants. Theychose which variables to manipulate: sunvs. shade, water vs. coffee, etc.

At the same time, the means of organizingand presenting information had changed.Students would use the Microsoft Excelprogram to chart their data, and MSPower Point to create their oral presenta-tions. Working with CCT researchers, Ms.James sought to take advantage of the newopportunities that technology tools mightoffer her students. She placed additionalemphasis on the visual component of pre-sentations, making it clear to her class thattheir talk should complement rather thanreiterate what was on their slides, that theywere not allowed merely to read from thescreen, and that visual aids should illustratedifferent kinds of information from the oralcomponent. Further, she encouraged stu-dents to consider the flexibility that technol-ogy tools could afford them. Students weretold in advance to be prepared to answerquestions about their projects from both theteacher and the other students, and that as

a class they should be listening to oneanother and should ask questions aboutareas that needed clarification or make sug-gestions for further research or alternativeexplanations. When students organizedand presented their work, a number ofimportant benefits were quickly apparent.

The first was that students had new choicesto make about how their data should beorganized. Where the time-intensive natureof plotting numbers by hand had meant thatMs. James gave her students limited choic-es in the past, the software her studentscurrently employed offered almost limitlessoptions. Students were now faced with theneed to make qualitative decisions aboutwhat formats would best convey the mean-ing of their data. For the first time, studentshad the opportunity to learn by looking at"wrong" representations to place thesame sets of numbers into bar graphs, piecharts and line plots, and to learn how eachof these formats affected the meaning oftheir information. Having this task automat-ed meant that they could examine a numberof permutations quickly and come to under-stand the manner in which different data arebest represented.

The table below represents the datacollected by "Sheena," a student in Ms.James's class who tested the relativegrowth rates of bean plants watered withdifferent liquids.

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10Kool Aid 0 in. 0.8 in. 1.4 in. 2.2 in. 3.2 in. 4.6 in. 6 in. 6.2 in. 6.4 in. 6.6 in.Water 0 in. 0.5 in. 1.5 in. 2.1 in. 3.3 in. 4.8 in. 6 in. 6.4 in. 6.5 in. 6.7 in.

3 6

After she had entered her data into aspreadsheet, Sheena found herself facedwith a couple of questions:

First, what kind of graph would bestrepresent these numbers? In years past,Ms. James had never needed to teach herstudents how to distinguish the relativemerits of different modes of datarepresentation. She simply taught themhow to make one kind of graph, and all stu-dents used the same one. However, theflexibility afforded by the spreadsheetprogram meant that students like Sheenacould cycle through numerous graphingoptions: in this case, Sheena's goal was torepresent the comparative growth rates ofplants she had watered with Kool-Aid andwith water, and she was considering bothpie charts and line graphs.

A discussion naturally followed from thischoice in which Ms. James and her stu-dents talked about the different qualities ofeach type of graph. The pie chart would beperfect if you wanted to know when theplants had grown the most it enabled theviewer to see which weeks had featuredsignificant plant growth and which had not.But it did not make it easy to compare thegrowth rates of the two plants. The linegraph, on the other hand, was less precisein its representation of growth in each indi-vidual week, but made it easy to make asimple visual comparison between the twoliquids' relative growth rates. All of the stu-dents learned from Sheena's dilemma aboutthe ways in which different graphs can com-municate different information, even whenthe data represented are the same.

A second question arose when Sheena wasbuilding a slideshow in PowerPoint toaccompany her oral presentation. Whichshould come first the table or the linegraph? This apparently simple questionactually required a critical consideration ofthe different types of representation. (Wasthe graph a visual aid for explaining-. the

17%

Bean Growth with Kool AidOs

2% 4%17% 6%

9%

17%16%

12%

Week I0 Week 20 Week 30 WeekO Week 5

Week 60 Week 70 Week 80 Week 9

Week 10

17%

Bean Growth with Water0%

1% 4%6%

3%

Week I

D Week 2

Week 3

0 Week0 Week 5O Week 6O Week 70Week 80 Week 9El Week 10

Bean Growth Kool Aid vs. Water

Aid

Water

Week I Week 2 Week 3 Week v Week 5 Week 6 Week 7 Week 8 Week 9 W O

3 7

table, orwith thetion?

was the table a mere appendix,graph being the key representa-

Questions such as these how to presentdata to an audience so that the most signif-icant points will be clear are importantconsiderations for real scientists. But theyhad never come up in Ms. James's class

bit

before. Only with the introduction of tech-nology tools did students have the power tomake real choices about how to representtheir data.

An additional benefit of adding technologyto the science fair project became evidentduring students' class presentations.Presenting data visually in a flexible, tech-nology-based format meant that the presen-ters, the class and their teacher could allwork with the data together during thepresentation something that would nothave been possible if the data had been pre-sented in a more traditional medium, suchas a poster. Students therefore had theopportunity to become aware of miscon-ceptions and misinterpretations embeddedin their work, and to rethink with their class-mates how they could correct thosemissteps.

While these new opportunities were readilyapparent in Ms. James's classroom, theend of this project found her still in the mid-dle of her confrontation with new ideas anddilemmas around technology. On the onehand, Ms. James's teaching with technolo-gy was opening up new possibilities in herclassroom: the flexibility afforded by spread-sheets and visual presentation tools gaveher students new options and new things tothink about in organizing and presentingtheir data; the need to think critically aboutvisual choices in constructing theirslideshows broadened students' conceptsof what it meant to communicate effective-ly; and the oral presentations themselvesprovided an opportunity to engage studentsin one another's work and thus help themlearn how to anticipate their audience'sneeds and communicate their pointseffectively.

At the same time, teaching these new ideaswas not easy for Ms. James. She herselfstruggled at times to figure out the best wayto present a student's data, or to advise her

students on decisions about the style or lay-out of their presentations. Further, Ms.James often limited her interaction with stu-dent presenters to questions about content.She asked questions almost exclusivelyfor the purpose of seeing if the studentunderstood his own experiment yet toassess her students' grasp of new commu-nications skills, she knew she needed toask questions about how they had chosento convey that understanding to others.

Though Ms. James had not yet fully inte-grated new ideas about technology into herteaching, the groundwork was laid in thisfirst project. Introducing technology into anestablished lesson had added new dimen-sions to her students' learning. Now thatMs. James is more comfortable using thesetechnology tools with her students andunderstands the particular opportunitiesthat technology brings to the classroom,she feels more prepared to tackle thesenew challenges in the upcoming schoolyea r.

EVOLVING TECHNOLOGY USE:MULTIPLE PROJECTS IN CHICAGOTeachers who had the chance to use CCT'stools with multiple student projects begandeveloping a new relationship to technologyuse. Their students' work with technologyreflected the growing palette of ideas beingaddressed in their classrooms.

"Susan Moran" had her fifth-grade classwork on three oral-report lessons during thecollaboration with CCT researchers; thefirst on the Revolutionary War, the secondon explorers and the third on America'sphysical expansion in the 19th century. In

each unit, students showed a markedimprovement in both their technical masteryand critical use of multimedia presentationtools. This steady improvement evinced notonly the students' progress, but also thedevelopment of Ms. Moran's thinking abouthow best to teach with these tools.

38

The following two pages lay out some of thechanges that occurred as Ms. Moran andher students progressively enhanced theirapproach to multimedia, folding in new skillsand concepts in a yearlong process of work-ing with CCT researchers.

The slide below is from a presentation stu-dents created using Microsoft Power Point.In its strengths and weaknesses, it is by nomeans atypical of the class's early work withthe program.

The slide illustrates a number of commonthemes we encountered in students' earlywork with technology tools of all kinds,before they had had the chance to thinkabout how different media are qualitativelydifferent from paper and pencil.

Most obvious is the pure technical inadequacythe fact that the text runs off the page.

Conceptual issues are also abundant, butharder to qualify:

o The student author does not yet under-stand the requirements of this medium:that material be divided across slides ina way that will correspond to his oralnarration.

o He has not taken his audience intoaccount when formatting his text (thetext's size and length make it inaccessi-ble).

O The student does not use visual aids tocarry part of his information. He ispresenting an essay on a slide.

Finally, this student author has not learnedto make aesthetic or stylistic choices. He hasnot exploited the expressive qualities thatmultimedia can add to a report. He doesnot employ color, sound or layout to empha-size any one piece of information overanother.

(')(0).J

On March 29,1929 three dying men lay on the frozen Antarctic continent-three men whohad made their way to the South Pole. On their way, they had been in a terrible blizzardwith in II miles( 18 kilometers). ( Robert was born on June 6, 1868 and died on March29, 1929). Robert died when his boat sank in the middle of the Arctic Ocean with three ofhis crew members.

Scott led two expeditions to the south pole and died on the second trip. His expeditionwas the second to reach the south pole. Scott led his first British Antarctic expedition onthe HMS Discovery. They sailed along the northern Ross Island to Mount Terror. Scottfound some area and named it King Edward VI land. Scott went in a hot air balloon to theAntarctica. Scott and two members tried cross the Ross Ice Shelf on a sled on November1902 throw January 1903. Tried to cross a lake of Vitamin C and mad them ill and wereforced to return. Most of the crew returned to England but Scott and a few others stayed toexplorer until September 1904. On his return Scott was promoted as captain and becamepopular with the public and wrote "The Voyage of Discovery."

In 1910 Scott went on his second expedition on a ship named Terra Nova. He was

3 9

rut

z

6=1

Ths

This first piece of work represents the verybaseline of student work with technology, inwhich the student has yet to tackle the newelements that the technology brings to theprocess of composing.

The next three slides also come from Ms.Moran's classroom, later in the school year.By this time, the students were far moretechnically proficient than they had been.Their slides were competently composedwith little use of distracting or superfluousanimation or sounds. More important, stu-dents' multimedia presentations werebecoming meaningful narrative composi-tions. Students were becoming "writers" inthis medium. To a far greater degree, stu-dents in this class used both text and pic-tures to contribute to the meaning of theirwork. Far fewer students were seduced byflashy but distracting graphics, sounds oranimation, and more students were usingdifferent aspects of the medium to addmeaning to their presentations.

The slides come from a student report onthe Wright brothers. They demonstrate thecreator's new mastery of a number ofimportant skills outlined in the ConceptualFramework for Live Presentations:

Narrative building The presentationtakes advantage of the slideshowformat to present a clear, well-sequenced storyline.

Layout Each slide is laid out tomaximize the impact of key visual andtext elements.

Making meaning with visual materialThe photos presented are not merelypictures; they add to the audience'sunderstanding of how the Wright broth-ers' invention evolved over time by illus-trating those new developments tomatch their descriptions in the text.

Creation of aesthetic tone Theauthors employed a consistent designscheme for the entire presentation, with

all of the elements, from the font to thebackground color (Originally sky blue,depicted here in black and white),contributing to an overall effect in whicheach element builds on the last and allelements contribute or correspond tothe themes of the report.

The report does not merely "look profes-sional." It communicates clearly.

In 1900, the Wright brothers tested their first glider.

Their longest ride that day lasted 10 seconds.

In 1902. the Wrieht brothers went to Kitty Hawka new elider.

They made a new glider with a rudder

With this new idea. they soarcd more than GOO(183 m1 which made a new world record.

In 1901. the Wright brothers went to Kitty Hawk.a new and improved glider.

This glider broke the world record for gliderreachine 389 feet (119 rn).

40

CLASSROOM OM PACTAssessing the effects of technology use inthe classrooms of our teacher-collaboratorsraises two questions: First, how did teach-ers' work with the ideas from CCT's evalua-tion toolkit impact students' use of technol-ogy? Second, how did this approach totechnology affect students' work relative toprevious work created without technology?

The first question is easier to answer. Overthe course of their projects, students' useof software, such as spreadsheet programsand multimedia presentation tools, changedin several observable ways. Students' pre-occupation with the frivolous and distractingaspects of technology cosmetic but con-tent-inappropriate features such as flashycolors and backgrounds, time-consumingbut empty animations and sound effects,and superfluous graphics dropped consid-erably. At the same time, students increas-ingly used technology to enhance the effec-tiveness of their work. They experimentedwith multiple arrangements of data until theyfound representations that conveyedinformation clearly. They created increas-ingly polished and professional slideshows,employing fonts and layouts that enhancedthe clarity and force of their narratives,eschewing distracting sounds and animationand using visual aids that added tothe meaning of their presentations.Furthermore, the levels of student growthincreased in relation to the time their teach-ers were able to devote to work with tech-nology. The more teachers were able totake advantage of their work withresearchers as a professional developmentexperience, the more new ideas theypassed on to their students.

The second question How did improvedtechnology use compare to pre-technologywork? is harder to answer. Using spread-sheets and presentation programs such asPower Point did not "solve" the issues thatteachers had traditionally faced when work-

ing with student projects: students stillsometimes botched their research, madeerrors of judgment or reasoning, mumbledtheir speeches, missed key content areasor failed to construct coherent narratives.Technology was not a panacea for all stu-dents' mistakes, nor did it raise the qualityof all students' work to an equal degree.

What technology did do was allow studentsto draw on a different and more varied skillset when analyzing data and creating pre-sentations. While not every student tookadvantage of the opportunities provided byspreadsheets or multimedia tools, many didcreate "deeper, more sophisticatedprojects with greater ease than would havebeen practical without using technology.Students enhanced their science reportswith clearer and more flexible representa-tions of data. They deepened their presen-tations with sets of maps, charts and con-trasting photographs, which would havebeen unwieldy or unavailable had they notused technology tools to gather and pres-ent their visual content. They engaged, attimes, in livelier discussions with their class-mates because they could refer to thesemore accessible visual aids. They madesophisticated aesthetic choices that height-ened the impact of their points of view.

Even still, these changes seem small whenlooked at in the grand scheme of a schoolyear an increase in students' literate use ofone or two computer programs. But if thesedevelopments represent just one part ofthese students' growing literacy, they alsorepresent only a small fraction of our partnerteachers' total teaching and preparation time.In other words, converting technology into aproductive class activity took up a relativelysmall portion of teachers' time, once theywere provided with tools to aid in lessonplanning and assessment.

Further, all of these teachers were still in theearly stages of rethinking their technology

use. Working with CCT researchers con-sidering technology use in the light of con-cepts like media and meaning, point of viewand audience, as outlined in the conceptualframeworks pushed teachers to build newlayers into their technology-based lessonplanning. The technology tools that hadbeen undistinguished stand-ins for paperand pen began to take on new value asteachers and students engaged the uniquequalities of multimedia. At the end of a year,this evolution was by no means completed,but it was a process that, once begun,became self-perpetuating. Each advance intheir students' technology use raised newquestions for teachers, about how and whystudents could use technology more effec-tively. The completed evaluation toolkit willprovide greater assistance to teachersseeking to answer these questions. Withsimilar professional development tools toaddress every technology-related aspect oftheir curriculum, teachers could turn theirstudents' time on the computer into a vast-ly more effective learning experience.

Perhaps most important, the teachers whocollaborated with CCT made the leap fromseeing technology as merely another meansfor accomplishing the work they werealready doing, to seeing technology as amedium for teaching 21st-century skills, and

CCD

thus creating new learning opportunities inthe classroom. This is the goal of the eval-uation toolkit. It is this fundamental shift inperspective that educators must encourageat the school, district and state levels if weare to capitalize on the gains that E-Ratefunding has enabled. As long as teachersview technology as an "add-on," these mul-timedia tools will be little more than compli-cated means for doing the same things, andthey will be obsolete long before they justi-fy their purchase price. lf, however, schoolsand districts can infuse their technologyplans and professional development with avision of the unique learning that technologycan foster, the E-Rate will be money wellspent.

Andy Gersick is Senior Research Assistant, Centerfor Children and Technology

Constance Kim is Research Associate, Center forChildren and Technology

Julie Thompson Keane is Associate Project Director,

Center for Children and Technology

Wendy Friedman is Research Associate, Center for

Children and Technology

Katie Culp, Ph.D., is Assistant Director forResearch, Center for Children and Technology

4 2

CHAPTER EigENHANCING STATE AND LOCAL POLICYMAKING

BOUT EDUCATIONAL TECHNOLOGIESA

Discussions about distributing E-Rate fundsthrough a formula-driven block grant reflecta changing policy climate in which the focushas shifted from direct federal grantmakingto increased power and responsibility foragencies at the state and local level.Recently, resources historically awarded andadministered directly by federal governmentagencies have instead increasingly "passedthrough" federal channels to state and localgovernments, where decision-makers havegreater latitude in determining how to spendthese funds.

Both the U.S. House and Senate versions ofthe current Elementary and SecondaryEducation Act (ESEA) reauthorization pro-pose moving a substantial amount of educa-tional technology monies from specializedfederal programs into block grants. Whetherthis shift is good or bad, it apparently is anidea whose time has come.

Student assessment is another illustrationof an education policy area in which statesare assuming a greater role. State high-stakes tests, linked to state curriculum stan-dards, are now the single largest influenceon what is taught in many classrooms.Frequently, individual pupils, teachers,schools and districts are rewarded or pun-ished based on the results of these tests.This approach to student assessment maysoon assume even greater prominence,since plans for ESEA reauthorization includefederally mandated, state-selected annualtesting in math and reading from gradesthree through eight. While many expertshave serious concerns about using summa-rive tests as the sole measure of education-al outcomes, both politicians and the publicare endorsing this strategy for educationreform.

In these circumstances, state and local deci-sion-makers face increased responsibilityand accountability to go along withgreater resources and powers. Yet many

BY CHRIS DEDE

state governors, legislators and agencypersonnel, as well as school superintend-ents, school board members and districtofficials have brief spans of time in office, alimited background in technology and educa-tion and numerous other demands on theirattention. Providing a "State PolicyFramework for Assessing EducationalTechnology Implementation" may enhancethese decision-makers' capacity to improvelearning and schooling through informationtechnology.

A STATE EPOLOCY [FRAMEWORKFOR ASSESSONG EDUCATIONALTECHNOLOGY DlitiiIPLEMENTATIONThe State Policy Framework for AssessingEducational Technology Implementation (seeAppendix 1) delineates a menu of ways thatstate policies can enhance educational tech-nology usage to improve student learningand standards-based educational reform.The framework also presents a process forcategorizing and charting the evolution ofstate policy for learning technologies in theoverall context of strategic planning toimprove education. This tool is intended pri-marily as a means of self-assessment forstate department of education staff andother education decision-makers.

The seven categories in the framework spanthe spectrum of potential state policyactions and provide a common template forcomparative policy discussions amongpolicy setters.

CategoriesA. State Technology Standards and

Assessments for StudentsB. State Technology Standards, Assess-

ments, Professional Development andAssistance for Teachers and TeacherEducators

C. Statewide Subsidized Electronic Net-work Linking Districts and OtherStakeholders for Information Exchange,Collaboration and Distance Education

4 3

D. Statewide Program Providing Data orAdministrative Systems to Districts (e.g.,fiscal databases, student assessmentresults)

E. State Guidelines for Technology-RelatedFacilities Design, Equipment, Software,Connectivity and Infrastructure; State-wide Consortium Purchasing Programs(Discounts for Large-Scale Orders) andFunding Support for TechnologyAcquisition

F. State-Sponsored Research and Eval-uation of Educational TechnologyInitiatives; Development of EducationalTechnology Devices, Applications andApproaches; Dissemination and Adapta-tion of Educational Technology

G. State Strategic Plan for EducationalImprovement, Including Technology; StateFunding for Educational Technology Plansand Initiatives

Each category has a set of essential ques-tions that highlight the issues involved inpolicies of this type. These questions arefollowed by indicators that depict an evolu-tionary path for the progression of statepolicy, while allowing for variation amongstates depending on individual circum-stances and political philosophy. For exam-ple, one of the essential questions in theState Technology Standards and Assessments forStudents category is "Are technology toolsand services used as a vehicle for ongoingimprovement of student learning andassessment?" Three indicators related tothis question are:

State provides online resources formodel curriculum units and lesson planslinking state content standards andstudent assessments.

State assessment strategies exemplifyeffective use of technology for assess-ment.

State provides incentives to developvirtual learning environments for

students who have difficulty obtainingaccess to classroom settings.

The more indicators a state and its localitiessatisfy in their implementation of education-al technologies, the more complete andaligned its policies are in ensuring effectiveusage to improve student learning and stan-dards-based reform.

GENESIS OF THE FRAMEWORKThe concept of developing a State PolicyFramework originated as part of a NationalScience Foundation-funded project con-ducted by the Council of Chief State SchoolOfficers (CCSSO). This project centered onexperts in various aspects of educationaltechnology preparing papers on howadvances in computers and telecommunica-tions were creating new opportunities forstate policies that could enhance studentlearning and the effectiveness of schools.These commissioned papers were present-ed and discussed at the CCSSO Technolo-gy Conference in February 2000.

Out of this dialogue came an initiative toprepare an educational technology policyframework for states. The initial draft wasrefined in discussion with policymakers fromseveral states who were advanced in theirthinking about educational technology andthen presented at the 2001 CCSSOTechnology Conference. After that event,the NEIR*TEC federal technology centersponsored an ongoing web-based discus-sion about the State Policy Framework (seewww.neirtec.org/statepolicy).

In May and June 2001, the BentonFoundation hosted two meetings. In atten-dance were federal, state and local educa-tion decision-makers, corporate and founda-tion leaders and other key stakeholderswho provided in the course of their discus-sions feedback on the policy framework.This dialogue was valuable in both broaden-

ing and deepening the framework; it alsoserved as a catalyst for the creation of theconcept maps that follow later in thischapter.

The State Policy Framework complementsinitiatives by the CEO Forum, theInternational Society for Technology inEducation and other groups that havedeveloped visions for the usage of comput-ers and telecommunications in education,as well as assessment rubrics for chartingprogress toward those visions. The frame-work, by focusing on how to achieve a visionrather than on what the components of thevision might include, provides a repertoireof interrelated mechanisms that states andother stakeholders can use to actualizedesired outcomes in educational computersand telecommunications. Formulations ofprocess, such as this framework, are a vitalmeans for accomplishing the substantiverecommendations that a variety of groupshave made for educational technologyusage.

STATES CURRENT USAGEOF POTENTOAL EDUCTDOEVIALPOLICY lifiECD-DANIOSIlfdSRelatively little is known about states' cur-rent implementation of the policy mecha-nisms presented in the framework. Mostsurveys of educational technology in thevarious states have focused on the tech-nologies and teacher training now in placerather than examining their state policiesthat caused the implementation. This is animportant area for further research.

Some fragmentary information about stateeducational technology policies, however, isavailable. The May 2001 issue of T.H.E.Journal published a "Technology ReportCard" for the states. This report indicatesthat one-third of the 31 states reportingrequire students to take a course abouttechnology before graduation, while a num-ber of other states that do not require a fulltechnology course do have technologystandards and performance objectives in

6E)Pi'e'

their state curriculum standards. That is, 90percent of the states responding eitherrequire or recommend the integration oftechnology into the curriculum. Althoughnine of these states have "approved lists"of hardware and software, none of thestates responding regulates or prescribesthe purchase of computer hardware, andonly two of these states regulate or pre-scribe the purchase of administrative soft-ware. Twenty-two states reported that theyhave some sort of system for gatheringinformation on technology usage and tech-nology accountability.

A May 2001 special issue of Education Week,entitled "Technology Counts: The NewDivides," contained state technology imple-mentation data from a survey conductedfrom October 1999 to March 2000. Thirty-four percent of all public schools surveyedresponded, and the data were aggregatedto allow cross-state comparisons:

0 Twenty-six states require teachers tohave technology training before beinglicensed to teach; but only Idaho,Michigan and North Carolina require atechnology test for prospective teach-ers.

O Of the 20 states that require schools ordistricts to set aside time for professionaldevelopment for teachers, just Arkansas,Florida, Tennessee and West Virginiahave such a requirement for technology-related professional development.

O While 38 states have recertificationrequirements for teachers, only Con-necticut, Georgia, North Carolina andVirginia include technology as a part ofthose requirements.

O Thirty-five states have student standardsfor technology, but only Florida, NorthCarolina, Pennsylvania and Virginia teststudents on those standards.

O Fifteen states offer teachers professionalor financial incentives to use technology;11 states offer such incentives to admin-istrators.

45

ruu

Detailed data on states' usage of thenumerous policy mechanisms delineated inthe framework, but not included in thesesurveys, is needed.

111AITERRELATOONSHOPS AMONGEDUCATEION POO-MBESThe categories in the State PolicyFramework are richly interrelated, as poli-cies that affect any aspect of the education-al system have consequences and affectother parts of the system. As an example,the concept map in Figure 1 illustrates theinterrelationships among aspects of theeducational system related to studentassessment.

In addition to improving student learningoutcomes, interventions that enhance diag-nostic assessments of students impactmodels of effective curriculum, teacher ped-

State Plan for, Educational

Improvement

agogy, statewide data collection and analy-sis, and parent and community involvement.Shifts in statewide data collection andanalysis impact state research and evalua-tion, which influences the state plan for edu-cational improvement, which in turn shapesteacher professional development andteacher preparation.

The case studies in this report describe theprocess of implementing a new means ofdiagnostic assessment in select Milwaukeeand Chicago public schools. The outcomesof these case studies illustrate many of theinteractions in Figure 1. For example, evenin the early stages of implementation, teach-ers rethought their approach to instructionand curriculum, and students gained newinsights into their learning. Research stud-ies documented why this innovative,technology-based educational strategy was

Teacher N\Professional

Development

TeacherPreparation

StatewideData

Collection andAnalysis

Teacher'sPedagogy

Models of \Effective

Curriculum

Research,( Evaluation, and

Dissemination

Parent andCommunityInvolvement

StudentLearningOutcomes

Figure 1: Concept Map of Improvements In Student Assessment

46

powerful, suggesting implications forteacher preparation and professional devel-opment. As new approaches to diagnosticassessment and their associated innova-tions mature, broader impacts with implica-tions for local communities and for statepolicies are likely.

Thus, the potential benefit of diagnostic stu-dent assessments is much greater thansimply their direct, immediate effect on aid-ing student learning through better feed-back to teachers and students. Conceptmap analyses that trace the potentialeffects of an innovation such as diagnosticstudent assessment both clarify the overallimplications of this advance for educationalimprovement and highlight other policyareas affected by this intervention. This aids

State Plan forEth,cational

improvement

State-SubsidizedElectronicNetwork

decision-makers in understanding how toalter the overall configuration of state poli-cies in response to an advance in any singlearea of education.

Tracing the potential impact on state poli-cies of federal E-Rate funding also illus-trates this principle of rich interdependenceamong interventions in the educational sys-tem. Support from the federal governmentreduces the amount of funding a state mustspend on financial support for districts' pur-chasing of technology and telecommunica-tions products. Both these added funds andthe additional equipment and telecommuni-cations services provided by the E-Ratehave beneficial second-order impacts onother types of state education policies, asillustrated by the concept map in Figure 2.

StateTechnologyPurchasing

Guidelines andInfrastructure

Standards

'41111616Financial

State

al

Jit,i

Collection and

StatewideData Support and I

Purchasing

FinancialFederal

Subsidized Virtua

.).,A Ir

State-

A

Analysis /4k Discounjts Learning

InstructionalSubsidized

State-

V Environments

Assistance 14 811:4 ..iiiiiiirResearch,

Evaluation, andDissemination

StateResources forProfessional

Development

State-Subsidized

DevelopjmentTechnology

IP'

TeacherProfessional

Development

Figure 2: Concept Map of Financial Support and Purchasing Discounts

4 7

StudentLearningOutcomes

Teacher'sPedagogy

(Ji

As an illustration, Chapter Two of this report(reflecting interviews with teachers, princi-pals and administrators) documents many ofthese second-order effects of E-Rate expen-ditures on technology infrastructure. Theseinclude both how the curriculum and themedia used to deliver and assess learninghave been altered and how opportunities forprofessional development have increased.Distant experts and archives complementteachers' knowledge. With a reliable, multi-year source of funding, strategic visionsguide annual investments in computers andtelecommunications. As these innovationspermeate statewide from exemplaryschools, this evolution will provide a founda-tion for new types of state policies like thosein Figure 2.

Increased federal financial support and pur-chasing discounts for telecommunicationscan free up state funding allocated for thatpurpose to be used instead to enhanceother areas of educational technology policy,such as instructional assistance, technologydevelopment, data collection and analysis, astate electronic network, virtual learningenvironments, professional development,and research, development and evaluation.These in turn can have positive effects onteaching and learning and other initiatives inthe state plan for educational improvement.

Furthermore, the state can implement anelectronic network with higher bandwidthand more advanced services, because ofthe sophisticated technology purchasingand infrastructure development districts canaccomplish using federal E-Rate funding.Overall, the E-Rate can have positive policyeffects well beyond its direct impact ontelecommunications equipment and servicesin districts. Of course, this assumes thatstates use the funds the E-Rate frees inother ways that enhance the implementationof educational technology, rather than reallo-cating those resources into the generalfund, as some states unfortunately aredoing.

As these concept map analyses demon-strate, the framework's lists of categories,essential questions and indicators are effec-tive in highlighting the types of policies avail-able to states, but they don't convey theextent to which effective educational innova-tion requires the simultaneous implementa-tion of mutually reinforcing, co-alignedpolicies across various categories.

A concept map that conveyed the completeand complex web of interrelationships thatunderlie a systems model of educationalimprovement, such as that presupposed bythe framework, would be too complicated toserve as a useful analytic tool. However, anelectronic version of the framework couldincorporate into each indicator its interrela-tionships with policy options in other cate-gories, allowing decision-makers to imple-ment mutually reinforcing, co-aligned poli-cies with maximum leverage for educationalimprovement.

Beyond incorporating the interrelationshipsdescribed above, future stages in the evolu-tion of the framework could involve:

including a glossary

implementing a question-based formatfor using the assessment (such as theComputer Systems Policy ProjectReadiness Guide atwww.cspp.org/projects/readiness)

including policy examples from variousstates that provide specific illustrations,benchmarks and quantitative levels forindicators

validating the overall model throughapplication to several states exemplify-ing a wide range of policy settings.

Other ideas for further development arewelcome. An online forum for providingfeedback about this framework is availableat ww.neirtec.org/statepolicy.

48

CONICLUSEIONThe types of improvements specified abovewould enhance the effectiveness andutility of the State Policy Framework forAssessing Educational Technology Imple-mentation. Even when this framework isfully developed, however, the individualvariability among states is so large thatdecision-makers should not use this type ofanalytic strategy to identify a "best" modelfor technology policies. Feedback by statepolicymakers on the framework indicatedthat, while successful practices in otherstates often offered insights for adaptingthose policies, each state has its ownunique politics, culture and local implemen-tation challenges. This is why the frameworkis best used as a menu of possible policyactions, a map of interrelationships amongpolicies and a means of self-assessment,rather than a comparative measure ofstates' conformance to a single constella-tion of policy choices.

Chris Dede is the Timothy E. Wirth Professor ofLearning Technologies at Harvard's GraduateSchool of Education.

4 9

APPIEHOON A: STATE POLICY IFF-AMEWORKFOR ASSESSING EDUCATIONAL TECHNOLOGYIMPLEMENTATION

State Policy Framework ©2001 by Chris Dede. Permission granted to copy without revisionfor educational use. To request permission for other uses, contact Chris Dede(Chris_Dede@Harvard.edu).

This framework describes ways that state policy can enhance educational technology usageto improve student learning and standards-based educational reform. The framework pres-ents a process for categorizing and charting the evolution of state policy for learning tech-nologies in the overall context of strategic planning to improve education. Its content drawson ideas in the policy papers commissioned for the February 2001 CCSSO State EducationalTechnology Conference, as well as on items in the American Institutes for Research OnlineSurvey of State Technology Coordinators (www.ainorg/essi/look/ isetview .html). Itsformat is based on assessments using a framework of indicators and essential questions,such as En Gauge (www.ncrel.org/engauge/) and the New England Association ofSchools and Colleges Standards for High School Accreditation and Restructuring (www.lab.brown.edu/neasc/).

This framework is intended primarily as a means of self-assessment for state department ofeducation staff. The categories provide a menu for potential state policy actions, as well as acommon template for comparative policy discussions among education staffs. The indicatorsdepict an evolutionary path for the progression of state policy, while allowing for variationamong states depending on individual circumstances and political philosophy. The essentialquestions are a way for staff to communicate the rationale for policies to decision-makerssuch as governors and legislators.

To refine the framework, I am interested in receiving feedback on the following questions:

What types of state policies for educational technology should be added to, or deletedfrom, these categories?

Should any categories of policies be combined or disaggregated?

How can the essential questions and the indicators of policy evolution for each categorybe improved?

How are these categories of state policies interrelated?

Beyond incorporating the results of feedback, future stages in the evolution of this assess-ment tool may include:

developing a visual model of educational innovation through technology, to complementthe category format by showing interrelationships among policy activities

ES including policy examples from various states that provide specific illustrations, bench-marks and quantitative levels for indicators

including a glossary

implementing a question-based format for using the assessment (such as the ComputerSystems Policy Project Readiness Guide online at www.cspp.org/projects/readiness)

g validating the overall model through application to several states exemplifying a widerange of policy settings.

I welcome other ideas for further development.

Please let me know your reactions by joining the online dialogue about this framework atwww.neirtec.org/statepolicy. Thanks!

VD. CATEGORIESOMPLEMENTATION

F STATE P. LOCIES FOR EDUCATBONAL TECHNOLOGY

A. State Technology Standards and Assessments for StudentsB. State Technology Standards, Assessments, Professional Development and

Assistance for Teachers, Teacher Educators, and AdministratorsC. Statewide Subsidized Electronic Network Linking Districts and Other Stakeholders

for Information Exchange, Collaboration and Distance EducationD. Statewide Program Providing Data or Administrative Systems to Districts (e.g., fiscal

databases, student assessment results)E. State Guidelines for Technology-Related Facilities Design, Equipment, Software,

Connectivity and Infrastructure; Statewide Consortium Purchasing Programs(Discounts for Large-Scale Orders) and Funding Support for Technology Acquisition

F. State-Sponsored Research and Evaluation of Educational Technology Initiatives;Development of Educational Technology Devices, Applications and Approaches;Dissemination and Adaptation of Educational Technology

G. State Strategic Plan for Educational Improvement, Including Technology; StateFunding for Educational Technology Plans and Initiatives

A. STATE TECHNOLOGYSTANDARDS AND ASSESSMENTS FOR STUDENTS

Essential Questions

How does the state ensure quality in student learning, assessment and usage of technology?Does the state have a well-articulated set of policies and procedures that support theongoing improvement of student learning?

Do these policies and procedures specifically address the integrated usage of technolo-gy to enhance teaching, learning and assessment?

Do these policies and procedures specifically address the diverse needs of the variedpopulation of learners for whom schools are responsible?

Are technology tools and services used as a vehicle for ongoing improvement of studentlearning and assessment?

Indicators

State technology standards and assessments created for students at all grade levels.

State technology standards for students are validated against national models such asInternational Society for Technology in Education standards.

State technology standards for students include provisions for learners with specialneeds and varied linguistic, cultural, ethnic and socioeconomic backgrounds, includingeducational resources designed for universal usability.

State provides incentives to develop virtual high schools and similar distance learningopportunities for students who have difficulty obtaining access to classroom settings.

State technology standards for students are integrated into state content standards forstudents.

State assessments of students' progress in meeting technology standards are integrat-ed into and aligned with state assessments of students' progress in meeting contentstandards.

State assessment strategies exemplify effectivitur of technology for assessment. 1 cm

B. STATE TECHNOLOGY STANDARDS, ASSESSMENTS, PROFESSIONALDEVELOPMENT AND ASSISTANCE FOR TEACHERS, TEACHEREDUCATORS AND ADMINISTRATORS

Essential Questions

How does the state ensure quality in teaching and administration?

Does the state have a well-articulated set of policies and procedures that support theongoing improvement of teaching and administration?

Do these policies and procedures specifically address the integrated usage of technologyto enhance teaching, learning and assessment, as well as to improve school management?

Are technology tools and services used as a vehicle for ongoing improvement ofteaching and administration, as well as for assessing compliance with these standards?

Is state financial assistance provided to enable districts to meet these standards and usetechnology as a means of professional development?

Does the state fund programs and organizations whose mission is to aid educators inachieving these standards?

Indicators

State technology standards are set for all teachers and teacher educators.

State technology standards for teachers and teacher educators center on technologyintegration.

State technology standards for teachers and teacher educators are validated againstnational models such as ISTE standards for teachers and NCATE standards for teachereducators.

State technology standards for teachers are integrated into other state teacher standardsand aligned with student technology and content standards.

State assessments of teachers' and teacher educators' progress in technologystandards, based on integration into content areas, exemplify effective use oftechnology for assessment.

State assistance is provided for developing district and teacher education programprofessional development plans in technology usage, in students safe and acceptableuse of technology, and in copyright and intellectual property issues.

State financial incentives for professional development include administrators and state-level leaders.

State financial incentives for professional development emphasize technology integrationand student assessment.

State financial incentives for professional development emphasize districts with unusualchallenges, such as urban and rural settings and impoverished communities.

State support for professional development includes aid for exemplary technology usagein professional development (e.g., virtual communities-of-practice and distant mentoring),and increased time for educators' planning.

State supports technical training programs for school and district technology coordina-tors, teacher education program faculty, and instructors of content courses for teachers.

52

o State-provided regional technology centers aid all educators (including instructors ofcontent courses for teachers), including providing quality reviews and adoption guidelinesfor digital resources.

C. STATEINODE SUBSODOZED ELECTRONIC IETWORIK LONKONGMST OCTS AND OThEr STAKEHOLDE S FOR VJFOIRMATOOEXCO-OANGE, COLLOlRATDOr AND DOST NCE EDUCATION

Essential Questions

Has the state developed a capacity for information exchange, resource sharing and collabo-rative action among multiple stakeholders at all levels?

o Do educators have access to high-bandwidth networking services?

o Do state subsidies for these services give priority to groups who otherwise would likelynot be connected?

Are policies and programs in place that support collaborative partnerships amongdistricts, higher education, state government agencies and for-profit and nonprofit organ-izations?

Indicators

O All districts and schools are connected via a state-subsidized electronic network.

E State financial incentives for participating in the network emphasize districts with unusualchallenges, such as urban and rural settings and impoverished communities.

o High-speed connections to the state network are installed at all schools

o State creates incentives for districts to receive discounts in implementing and maintain-ing the network.

o State provides financial incentives and logistical support for districts collaboratingelectronically with other districts, higher education, industry, and other social serviceproviders.

D. STATEINO E PROGRAM PROVIDING DATA OR ADMINISTRATIVESYSTEMS TO DOSTROCTS (e.g., fiscal1 databases, student assessmentresuOits)

Essential Questions

Does the state provide districts with aggregated, synthesized information on which to baseeducational policies and decisions?

O Does the state encourage districts to collect data in ways that facilitate aggregation andanalysis?

o Does the state provide financial incentives for districts to participate in data-baseddecision making?

O Does the state analyze comparative data from schools to assess alternative educationalinterventions and disseminate the results to districts?

53

-7

zC-

Indicators

State provides incentives for districts using common administrative systems.

State financial incentives for participating in data collection and analysis emphasizedistricts with unusual challenges, such as urban and rural settings and impoverishedcommunities.

Educational statistics are collected from all districts are aggregated and analyzed by thestate.

Data mining of statistical records is conducted by the state, extensively disseminated,and used for policy decisions.

E. STATE GUIDELINES FOR TECHNOLOGY-RELATED FACILITIES DESIGN,EQUIPMENT, SOFTWARE, CONNECTIVITY AND INFRASTRUCTURE;STATEWIDE CONSORTIUM PURCHASING PROGRAMS (DISCOUNTS FORLARGE-SCALE ORDERS) AND FUNDING SUPPORT FOR TECHNOLOGYACQUISITION

Essential Questions

Does the state aid districts in designing and purchasing their technology infrastructure?

Does the state provide guidelines for purchasing equipment, software and connectivityand for designing or renovating facilities?

Does the state negotiate volume-purchasing discounts from suppliers and vendors?

Does the state provide financing for technology infrastructure, and does this support givepriority to districts that face economic challenges?

Indicators

State technology purchasing guidelines stress evolutionary, strategic approaches indeveloping infrastructures for new and existing facilities and for installations of equip-ment, software and connectivity.

State technology infrastructure standards are integrated with districts' and teachereducation programs' educational plans.

Based on advances in technology, technology purchasing guidelines and infrastructurestandards for facilities, equipment, software and connectivity are regularly updated.

State has established a program for hardware, software and online services purchasing,with discounts for large-scale orders.

Extensive state financial support is offered for hardware, software and online servicespurchasing.

State financial incentives for technology and infrastructure development emphasizedistricts with unusual challenges, such as urban and rural settings and impoverishedcommunities.

F. STATE-SPONSORED RESEARCO-O AND EVALUATOON OF EDUCATUONALTECHNOLOGY ONOTOATOVES; DEVELOPMENT OF EDUCATOONALTECHNOLOGY DEVOCES, APPLOCATUONS AND APPROACHES;DOSSEMONATOON AND ADAPTATOON OF EDUCATOONAL TECHNOLOGY

Essential Questions

How does the state provide support for the effective usage and evolution of educationaltechnology resources?

Does the state assess the effectiveness of its technology investments through conduct-ing research and evaluation studies?

Does the state sponsor the development of technologies customized to its needs?

o Does the state provide means by which districts can adapt innovations successfulelsewhere?

Indicators

o Systematic state-sponsored research is based on analyses from statewide database;policy decisions are based on research syntheses.

State provides incentives for districts to apply as designated testbeds for innovation,evaluation.

State systematically evaluates technology initiatives, including collection of evaluationsfrom other sources and usage of analyses from statewide database; outcomes informpolicy decisions.

State provides vendors with guidelines on desired devices, applications and assistivetechnologies.

State offers vendors incentives for developing desired devices and applications as wellas for involving local teachers in adapting standards-based software.

State sponsors educational technology development, including distance education, aswell as participation in consortia for this purpose.

State systematically disseminates information on transfer and adaptation of innovationsvia technology service centers and state electronic network.

State research, development and dissemination initiatives emphasize technologicalinnovations that could improve equal educational opportunity.

Z23

G. STATE STRATEGIC PLAN FOR EDUCATIONAL IMPROVEMENT,INCLUDING TECHNOLOGY; STATE FUNDING FOR EDUCATIONALTECHNOLOGY PLANS AND INITIATIVES

Essential Questions

Does the state make strategic investments in improving education through technology?Does the state have a strategic process for educational improvement to which the statetechnology plan is aligned?

Does the state provide systematic long-term funding for achieving the objectives of itstechnology plan?

Does the state leverage its educational technology investments with related funding fromother sources?

Indicators

State plan for educational technology incorporates insights and lessons learned fromother states, federal efforts.

State plan for educational technology is aligned with overall state strategic plan for educa-tional improvement.

State plan for educational technology emphasizes innovations that could improve equaleducational opportunity.

State systematically funds capital investments in educational technology infrastructure,including depreciation.

State systematically supports long-term funding for hardware, applications, connectivityand professional development.

State systematically supports long-term funding for adequate staffing in technology usage(based on guidelines such as those in the North Carolina Technology Plan atwww.tps.dpi.state.nc.us/tech plan2000/techplan2000.htm1 #Personnel).

State provides assistance in applying for funded educational technology initiatives.

State provides matching funds to foster participation in federal and philanthropic educa-tional technology initiatives.

State policy complements and supplements federal and philanthropic funding, research,and equity initiatives.

State financial incentives and assistance emphasize districts with unusual challenges,such as urban and rural settings and impoverished communities

56

A f EN LIY1 EDUCATDOI AL TECHNOLOGYEVALU TION RESOURCES

1. PU LEICATOON/TOOL:Professional Development: Learning from the Best

ORGANIZATION:NCREL

URL:www.ncrel.org/pd/toolkit.htm

PURPOSE:Help districts and schools plan and evaluate professional development.

KEY FEATURESGuidelines for implementing a successful, large-scale professionaldevelopment (PD) initiative, with elements culled from winners of the NationalAwards for Model Professional Development.

STRUCTURE:Three steps (designing professional development; implementing professionaldevelopment; evaluating and improving professional development), each withaccompanying suggestions and tools.

IMPLEMENTATION:A manual is available for downloading or use online and lays out the steps tosuccessful PD, with accompanying forms, checklists, etc. to help teachers andadministrators organize the PD process.

2. PU LOCAT1ON/TOOL:enGauge

ORGANIZATION:NCREL and the Metiri Group

URL:www.ncrel.org/engauge

PURPOSE:Help districts and schools plan and evaluate the system-wide use ofeducational technology.

KEY FEATURESGuidelines for implementing and evaluating large-scale technology integration.Covers pedagogy, professional development, and evaluation and assessment.Provides further resources.

STRUCTURE:Outline of six essential conditions of effective use of technology: vision,effective practice, equity, leadership, access and educator proficiency.Offers suggestions and examples for implementing the essential conditionsand assessment tools for evaluating initiatives against the enGauge model.

IMPLEMENTATION.Project leaders register with the enGauge site and can then access onlinesurveys that help them assess the progress of their tech integration initiatives.

5ZZ

3. PUBLICATION/TOOL:Profiler

ORGANIZATION:High Plains RTEC

URL:http://profiler.scrtec.org/

PURPOSE:Improve teacher/student collaboration to improve technology skills.

KEY FEATURES:Assessment tools around a variety of technology skill areas, from facility withindividual programs to the understanding of proper pedagogy. The programhelps groups of users identify individuals who can be mentors around a givenskill, thus encouraging collaboration within the group.

STRUCTURE:Thirty-four online surveys designed by both corporate and educationalorganizations track individual progress around a variety of technology andtechnology-teaching skills. Individuals use surveys and Profiler informs them ofother people in their group (class, school, district) who are expert in the givenskill area, encouraging group members to teach one another. The site also offerstutorials in all of the surveyed skills.

IMPLEMENTATION:Surveys are available online or in print. Implementing the collaboration elementis up to the user group.

4. PUBLICATION/TOOL:Technology Project Evaluation Instrument

ORGANIZATION:SEIR*TEC and the North Carolina Department of Public Instruction

URL:www.seirtec.org/eval.html

PURPOSE:Created to assess Technology Literacy Challenge Grant projects but alsoapplicable to assessing district technology programs.

KEY FEATURES:Tool to help school system leaders: (a) reflect on the progress of an ongoingtechnology integration project, (b) think about what needs to be done in order tomeet project goals, and (c) consider strategies for maximizing project impact.The instrument also offers tools for collecting and reporting comparableinformation across the projects.

STRUCTURE:A survey for project managers to assess the progress of their ed tech integrationprojects.

IMPLEMENTATION:The tool is available online as an aid to reflection and self-evaluation.

5. PUBLICATIO /TOOL:STAR Chart

ORGANIZATION:The CEO Forum on Education and Technology

URL:www.ceoforum.org/starchart.cfm

PURPOSE:Provides benchmarks against which K-12 schools and colleges can assess andtrack their progress in technology integration.

KEY FEATURES:Identifies and defines four school profiles ranging from the early tech school (aschool with little or no technology) to the target tech school (a school is a modelfor the integration and innovative use of education technology).

STRUCTURE:Questionnaire addresses three critical questions: a.) Is your school usingtechnology effectively to ensure the best possible teaching and learning? b.)What is your school's current education technology profile? C.) On what areasshould your school focus to improve its level of technology integration? TheChart is intended to help project managers set benchmarks, apply for grants,determine funding priorities, and create individualized assessment tools.

IMPLEMENTATIONUsers complete an online, multiple-choice questionnaire that provides instantfeedback on how their school is doing in its technology integration process.

6. PUBLICATION/TOOL:Learning with Technology Profile Tool

ORGANIZATION:NCRTEC

URL:www.ncrtec.org/capacity/profile/profwww.htm

PURPOSE:Helps teachers rethink both their general teaching practices and their approachto technology use.

59Z77

KEY FEATURE:Survey tool to help educators compare their current instructional practices witha set of indicators for engaged learning and high-performance technology usage.

STRUCTURE:Teachers fill out a multiple-choice survey to create a profile of their teachingpractices, which is then graphed against a low-to-high scale in two major areas,engaged learning and high-performance technology.

IMPLEMENTATION:This is a tool for individuals to complete online.

7. PUBLICATION/TOOL:Professional Competency Continuum and the PCC Assessment Tool

ORGANIZATION-The Milken Family Foundation

.URL:

www.mff.org/publications/publications.taf?page=158

PURPOSE:The assessment tool helps educators assess their or their school's status alonga "professional competency continuum" (PCC). The PCC represents research-and classroom-tested approaches to developing the skills in teachers andadministrators necessary for effective integration of technology in learning.

KEY FEATURES:Survey covers five key areas: administrative competency; classroom andinstructional management; core technology skills; curriculum, learning andassessment; and professional practice.

STRUCTURE:Users can complete either a general or a detailed assessment. The tool scoreseach of 4 to 6 specific indicator areas within the five major areas. Taking thedetailed assessment provides the user with access to customized advice andother Milken Foundation PD resources.

IMPLEMENTATION:Survey is aimed at individual teachers or administrators.

60

BENTONFOUNDATION

BOARD OF DIRECTORS

Charles Benton

Chairman

Worth Bruntjen

Treasurer

Henry Rivera

General Counsel

Shelley Benton

Elizabeth Daley

Terry Goddard

Harold Richman

Terry Tinson Saario

Jorge Reina Schement

Andrea L. Taylor

TRUSTEES

Charles Benton

Marjorie C. Benton

Leonard Schrager

STAFF

Andrea L. Taylor

President

Karen Menichelli

Executive Vice President

The Commonwealth Building

1625 K Street, NW, 11"' floor

Washington, DC 20006

Phone: 202.638.5770

Fax: 202.638.5771

Email: benton@benton.org

www.benton.org

8 February 2002

Dear Colleague:

We are pleased to share our latest publication, Great Expectations:Leveraging America's Investment in Educational Technology and the companionvolume, The Evaluation Toolkit: A Work in Progress.

This work is phase two of a collaboration between the Center for Childrenand Technology at the Education Development Center and the BentonFoundation. The effort examined the impact of the E-Rate and educationaltechnology in schools and developed new tools to assist teachers, administratorsand policymakers. The findings and recommendations in our report werecompiled by experts in the field of educational technology and highlight the needfor expanded teacher training and new curriculum design.

In February 2000, the Benton Foundation and the Center for Children andTechnology released one of the first studies of the federal program begun in 1997,The E-Rate in America: A Tale of Four Cities. The Joyce Foundation, a Chicago-based philanthropy committed to school reform and educational technology,generously supported both projects. Recently, the Joyce Foundation announcedsupport for a third phase of the E-Rate series to investigate the important issue ofsustainability in the educational technology arena.

The Benton Foundation, now in its 20 year, continues its focus oncommunications in the public interest with a strong commitment to acceleratingdigital opportunity for all people. Additional information about our program andvision for the future is available online at www.benton.org.

Thank you for your interest in our work.

Sincerely,

Andrea L. Taylor

Enclosures (2)

61

"BEST COPY AVAILABLE

-7111111

BENTONFOUNDATION

www.benton.org

www.edc.org

ISBN 1-930615-02-7

,

U.S. Department of EducationOffice of Educatonal Research and Improvement (OERI)

National Library of Education (NLE)Educational Resources Information Center (ERIC)

REPRODUCTION RELEASE(Specific Document)

NOTICE

REPRODUCTION BASIS

SI

Elluetticad Restates Mamie tete

This document is. covered by a signed "Reproduction Release(Blanket) form (on file within the ERIC system), encompassing all

or classes of documents from its source organization and, therefore,does not require a "Specific Document" Release form.

This document is Federally-funded, or carries its own permission toreproduce, or is otherwise in the public domain and, therefore, may

be reproduced by ERIC without a signed Reproduction Release form

(either "Specific Document" or "Blanket").

EFF-089 (9/97)