Table of Contents - Harvard University · 2015-10-27 ·...

Table of Contents

Foreword by Andreas Schleicher....................................................................................................xxviii

Foreword by Chris Dede.................................................................................................................... xxx

Preface.............................................................................................................................................xxxiii

Acknowledgment..........................................................................................................................xxxviii

Volume ISection 1

Defining Real-World Skills in Technology-Rich Environments

Chapter 1TwentyFirstCenturySkillsvs.DisciplinaryStudies?........................................................................... 1

Lars Vavik, Stord/Haugesund University College, NorwayGavriel Salomon, Hiafa University, Israel

Chapter 2DigitalCompetence:ANetofLiteracies.............................................................................................. 13

Edith Avni, Toward Digital Ethics Initiative, IsraelAbraham Rotem, Toward Digital Ethics Initiative, Israel

Chapter 3TheApplicationofTransdisciplinaryTheoryandPracticetoSTEMEducation................................. 42

Susan Malone Back, Texas Tech University, USAHeather Greenhalgh-Spencer, Texas Tech University, USAKellilynn M. Frias, Texas Tech University, USA

Chapter 4TheSOARStrategiesforOnlineAcademicResearch:HelpingMiddleSchoolStudentsMeetNewStandards............................................................................................................................................... 68

Carolyn Harper Knox, University of Oregon, USALynne Anderson-Inman, University of Oregon, USAFatima Terrazas-Arellanes, University of Oregon, USAEmily Deanne Walden, University of Oregon, USABridget Hildreth, University of Oregon, USA

Chapter 5TheValueofMetacognitionandReflectivityinComputer-BasedLearningEnvironments.............. 105

Sammy Elzarka, University of La Verne, USAValerie Beltran, University of La Verne, USAJessica Decker, University of La Verne, USAMark Matzaganian, University of La Verne, USANancy T. Walker, University of La Verne, USA

Chapter 6AFrameworkforDefiningandEvaluatingTechnologyIntegrationintheInstructionofReal-WorldSkills........................................................................................................................................ 137

J. Christine Harmes, Assessment Consultant, USAJames L. Welsh, University of South Florida, USARoy J. Winkelman, University of South Florida, USA

Chapter 7EquippingAdvancedPracticeNurseswithReal-WorldSkills........................................................... 163

Patricia Eckardt, Stony Brook University, USABrenda Janotha, Stony Brook University, USAMarie Ann Marino, Stony Brook University, USADavid P. Erlanger, Stony Brook University, USADolores Cannella, Stony Brook University, USA

Section 2Technology Tools for Learning and Assessing Real-World Skills

Chapter 8SimulationsforSupportingandAssessingScienceLiteracy............................................................. 191

Edys S. Quellmalz, WestEd, USAMatt D. Silberglitt, WestEd, USABarbara C. Buckley, WestEd, USAMark T. Loveland, WestEd, USADaniel G. Brenner, WestEd, USA

Chapter 9UsingtheCollegiateLearningAssessmenttoAddresstheCollege-to-CareerSpace........................ 230

Doris Zahner, CAE, USAZachary Kornhauser, CAE, USARoger W. Benjamin, CAE, USARaffaela Wolf, CAE, USAJeffrey T. Steedle, Pearson, USA

Chapter 10Rich-MediaInteractiveSimulations:LessonsLearned...................................................................... 261

Suzanne Tsacoumis, HumRRO, USA

Chapter 11AnApproachtoDesign-BasedImplementationResearchtoInformDevelopmentofEdSphere®:ABriefHistoryabouttheEvolutionofOnePersonalizedLearningPlatform................................... 284

Carl W. Swartz, MetaMetrics, USA & University of North Carolina, USASean T. Hanlon, MetaMetrics, USAE. Lee Childress, Corinth School District, USAA. Jackson Stenner, MetaMetrics, USA & University of North Carolina, USA

Chapter 12ComputerAgentTechnologiesinCollaborativeAssessments........................................................... 319

Yigal Rosen, Harvard University, USAMaryam Mosharraf, Pearson, USA

Chapter 13AToughNuttoCrack:MeasuringCollaborativeProblemSolving................................................... 344

Lei Liu, Educational Testing Service, USAJiangang Hao, Educational Testing Service, USAAlina A. von Davier, Educational Testing Service, USAPatrick Kyllonen, Educational Testing Service, USADiego Zapata-Rivera, Educational Testing Service, USA

Chapter 14Animalia:CollaborativeScienceProblemSolvingLearningandAssessment.................................. 360

Sara Bakken, Pearson Education, USAJohn Bielinski, Pearson Education, USACheryl K. Johnson, Pearson Education, USAYigal Rosen, Harvard University, USA

Chapter 15UsingTechnologytoAssessReal-WorldProfessionalSkills:ACaseStudy..................................... 385

Belinda Brunner, Pearson, UKKirk A. Becker, Pearson, USANoel Tagoe, Chartered Institute of Management Accountants, UK

Volume II

Chapter 16AssessmentintheModernAge:ChallengesandSolutions................................................................ 415

Mahmoud Emira, City & Guilds of London Institute, UKPatrick Craven, City & Guilds of London Institute, UKSharon Frazer, City & Guilds of London Institute, UKZeeshan Rahman, City & Guilds of London Institute, UK

Chapter 17Technology-AssistedLearningforStudentswithModerateandSevereDevelopmentalDisabilities.......................................................................................................................................... 445

Diane M. Browder, University of North Carolina at Charlotte, USAAlicia Saunders, University of North Carolina at Charlotte, USAJenny Root, University of North Carolina at Charlotte, USA

Chapter 18MitigationofTestBiasinInternational,Cross-NationalAssessmentsofHigher-OrderThinkingSkills................................................................................................................................................... 472

Raffaela Wolf, CAE, USADoris Zahner, CAE, USA

Chapter 19Evidence-CenteredConceptMapinComputer-BasedAssessmentofCriticalThinking.................. 502

Yigal Rosen, Harvard University, USAMaryam Mosharraf, Pearson, USA

Chapter 20“VisittoaSmallPlanet”:AchievementsandAttitudesofHighSchoolStudentstowardsLearningonFacebook–ACaseStudy.............................................................................................................. 528

Rikki Rimor, Kibbutzim College of Education Technology and the Arts, IsraelPerla Arie, Kibbutzim College of Education Technology and the Arts, Israel

Chapter 21Cross-BorderCollaborativeLearningintheProfessionalDevelopmentofTeachers:CaseStudy–OnlineCoursefortheProfessionalDevelopmentofTeachersinaDigitalAge................................. 558

Rafi Davidson, Kaye Academic College of Education, IsraelAmnon Glassner, Kaye Academic College of Education, Israel

Section 3Automated Item Generation and Automated Scoring Techniques for Assessment and

Feedback

Chapter 22UsingAutomatedProcedurestoGenerateTestItemsThatMeasureJuniorHighScienceAchievement....................................................................................................................................... 590

Mark Gierl, University of Alberta, CanadaSyed F. Latifi, University of Alberta, CanadaHollis Lai, University of Alberta, CanadaDonna Matovinovic, CTB/McGraw-Hill, USAKeith A. Boughton, CTB/McGraw-Hill, USA

Chapter 23AutomatedScoringinAssessmentSystems....................................................................................... 611

Michael B. Bunch, Measurement Incorporated, USADavid Vaughn, Measurement Incorporated, USAShayne Miel, LightSide Labs, USA

Chapter 24AutomatedScoringofMulticomponentTasks................................................................................... 627

William Lorié, Questar Assessment, Inc., USA

Chapter 25AdvancesinAutomatedScoringofWritingforPerformanceAssessment........................................ 658

Peter W. Foltz, Pearson, USA & University of Colorado – Boulder, USA

Chapter 26UsingAutomatedFeedbacktoImproveWritingQuality:OpportunitiesandChallenges.................. 678

Joshua Wilson, University of Delaware, USAGilbert N. Andrada, Connecticut State Department of Education, USA

Section 4Analysis, Interpretation, and Use of Learning and Assessment Data from Technology

Rich Environments

Chapter 27AssessingProblemSolvinginTechnology-RichEnvironments:WhatCanWeLearnfromOnlineStrategyIndicators?............................................................................................................................ 705

Jean-Francois Rouet, University of Poitiers, FranceZsofia Vörös, University of Poitiers, FranceMatthias von Davier, Educational Testing Service, USA

Chapter 28AnalyzingProcessDatafromTechnology-RichTasks....................................................................... 724

Lisa Keller, University of Massachusetts – Amherst, USAApril L. Zenisky, University of Massachusetts – Amherst, USAXi Wang, University of Massachusetts – Amherst, USA

Chapter 29AnalyzingProcessDatafromProblem-SolvingItemswithN-Grams:InsightsfromaComputer-BasedLarge-ScaleAssessment........................................................................................................... 749

Qiwei He, Educational Testing Service, USAMatthias von Davier, Educational Testing Service, USA

Chapter 30AssessmentofTaskPersistence.......................................................................................................... 777

Kristen E. DiCerbo, Pearson, USA

Chapter 31AssessingEngagementduringtheOnlineAssessmentofReal-WorldSkills..................................... 804

J. Christine Harmes, Assessment Consultant, USASteven L. Wise, Northwest Evaluation Association, USA

Compilation of References............................................................................................................ xxxix

About the Contributors................................................................................................................. cxxix

Index................................................................................................................................................. cxlvi

Foreword

Weneedtothinkharderabouthowweprepareyoungpeoplefortomorrow’sworld.Inthepast,educa-tionwasaboutteachingstudentssomething.Now,it’saboutmakingsurethatstudentsdevelopareliablecompassandthenavigationskillstofindtheirownwaythroughanuncertain,volatile,andambiguousworld.Now,schoolsneedtopreparestudentsforaworldinwhichmostpeoplewillneedtocollaboratewithpeopleofdiverseculturaloriginsandappreciatedifferentideas,perspectivesandvalues;aworldinwhichpeopleneedtodecidehowtotrustandcollaborateacrosssuchdifferences;andaworldinwhichtheirliveswillbeaffectedbyissuesthattranscendnationalboundaries.Technologyhasbecomethekeytobridgespaceandtimeinallofthis.

Thesedays,wenolongerknowexactlyhowthingswillunfold.Weareoftensurprisedandneedtolearnfromtheextraordinary,andsometimeswemakemistakesalongtheway.Anditwilloftenbethemistakesandfailures,whenproperlyunderstood,thatcreatethecontextforlearningandgrowth.Agenerationago,teacherscouldexpectthatwhattheytaughtwouldlasttheirstudentsalifetime.Today,schoolsneedtopreparestudentsformorerapideconomicandsocialchangethaneverbefore,forjobsthathavenotyetbeencreated,tousetechnologiesthathavenotyetbeeninvented,andtosolvesocialproblemsthatwedon’tyetknowwillarise.

Howdowefostermotivated,engagedlearnerswhoarepreparedtoconquertheunforeseenchal-lengesoftomorrow,nottomentionthoseoftoday?Thedilemmaforeducatorsisthatroutinecognitiveskills—theskillsthatareeasiesttoteachandeasiesttotest—arealsotheskillsthatareeasiesttodigitize,automate,andoutsource.Thereisnoquestionthatstate-of-the-artknowledgeandskillsinadisciplinewillalwaysremainimportant.Innovativeorcreativepeoplegenerallyhavespecializedskillsinafieldofknowledgeorapractice.Andasmuchas‘learningtolearn’skillsareimportant,wealwayslearnbylearningsomething.However,educationalsuccessisnolongeraboutreproducingcontentknowledge,butaboutextrapolatingfromwhatweknowandapplyingthatknowledgeinnovelsituations.Putsim-ply,theworldnolongerrewardspeopleforwhattheyknow—Googleknowseverything—butforwhattheycandowithwhattheyknow.Becausethatisthemaindifferentiatortoday,educationtodayneedstobemuchmoreaboutwaysofthinking,involvingcreativity,criticalthinking,problem-solving,anddecision-making;aboutwaysofworking,includingcommunicationandcollaboration;abouttoolsforworking,includingthecapacitytorecognizeandexploitthepotentialofnewtechnologies;and,lastbutnotleast,aboutthesocialandemotionalskillsthathelpusliveandworktogether.

Conventionally,ourapproachtoproblemswastobreakthemdownintomanageablebitsandpiecesandthentoteachstudentsthetechniquestosolvethem.Buttodaywecreatevaluebysynthesizingthedisparatebits.Thisisaboutcuriosity,open-mindedness,andmakingconnectionsbetweenideasthatpre-

xxviii

Foreword

viouslyseemedunrelated,whichrequiresbeingfamiliarwithandreceptivetoknowledgeinotherfieldsthanourown.Ifwespendourwholelifeinasiloofasinglediscipline,wewillnotgaintheimaginativeskillstoconnectthedotswherethenextinventionwillcomefrom.

Equallyimportant,themorecontentknowledgewecansearchandaccess,themoreimportantbecomesthecapacitytomakesenseofthiscontent—thecapacityofindividualstoquestionorseektoimprovetheacceptedknowledgeandpracticesoftheirtime.Inthepast,youcouldtellstudentstolookintoanencyclopediawhentheyneededsomeinformation,andyoucouldtellthemthattheycouldgenerallyrelyonwhattheyfoundtobetrue.Today,literacyisaboutmanagingnon-linearinformationstructures,buildingyourownmentalrepresentationofinformationasyoufindyourownwaythroughhypertextontheInternet,anddealingwithambiguity—interpretingandresolvingconflictingpiecesofinformationthatwefindsomewhereontheWeb.

Perhapsmostimportantly,intoday’sschools,studentstypicallylearnindividuallyandattheendoftheschoolyear,wecertifytheirindividualachievements.Butthemoreinterdependenttheworldbe-comes,themoreweneedgreatcollaboratorsandorchestrators.Innovationtodayisrarelytheproductofindividualsworkinginisolationbutanoutcomeofhowwemobilize,share,andlinkknowledge.Intheflatworld,everythingthatisourproprietaryknowledgetodaywillbeacommodityavailabletoeveryoneelsetomorrow.Expresseddifferently,schoolsneedtodriveashiftfromaworldwhereknowledgeisstackedupsomewhere,depreciatingrapidlyinvalue,towardsaworldinwhichtheenrichingpowerofcommunicationandcollaborativeflowsisincreasing.Andtheywillneedtohelpthenextgenerationtobetterreconcileresilience(managinginanimbalancedworld)withgreatersustainability(puttingtheworldbackintobalance).

Thisisatoughagenda.Whatiscertainisthatitwillnevermaterialiseunlessweareabletoclearlyconceptualiseandmeasurethose21stcenturyknowledgeareasandskills.Withoutrigorousconcep-tualisation,wewillnotbeabletobuildmeaningfulcurriculaandpedagogiesaroundtheseknowledgeareasandskills.And,attheendoftheday,whatisassessediswhatgetstaught.Thisvolumemakesamajorstepinadvancingthisfrontier.Itexaminesarangeofskillsthatareimportant;itlooksatinnova-tivemeasurementmethodstomaketheseskillsamenabletoquantitativeassessmentinwaysthattheybecomeactivatorsofstudents’ownlearning,anditlooksathowwecanlearntodrinkfromthefirehoseofincreasingdatastreamsthatarisefromnewassessmentmodes.

Andreas Schleicher Organisation for Economic Co-Operation and Development (OECD), France

xxix

Foreword

InitslandmarkreportEducation for Life and Work in the 21st Century,theNationalResearchCouncil(2012)described“deeperlearning”asaninstructionalapproachimportantinpreparingstudentswithsophisticatedcognitive,intrapersonal,andinterpersonalskills.Theapproachesrecommendedbyadvo-catesofdeeperlearningarenotnew,andhistoricallytheseinstructionalstrategieshavebeendescribedunderavarietyofterms.Untilnow,however,theyhavebeenrarelypracticedwithintheschools(Dede,2014),resultinginthesadsituationthatstudentswhoexcelinschoolmaystruggleintherealworld.Andstudentswhostruggleinschoolarelikelytosinkintherealworld.Various“deeperlearning”ap-proachesaredescribedbelow.

• Case-basedlearninghelpsstudentsmasterabstractprinciplesandskillsthroughtheanalysisofreal-worldsituations;

• Multiple, varied representations of concepts provide different ways of explaining complicatedthings,showinghowthosedepictionsarealternativeformsofthesameunderlyingideas;

• Collaborativelearningenablesateamtocombineitsknowledgeandskillsinmakingsenseofacomplexphenomenon;

• Apprenticeshipsinvolveworkingwithamentorwhohasaspecificreal-worldroleand,overtime,enablesmasteryoftheirknowledgeandskills;

• Self-directed,life-wide,open-endedlearningisbasedonstudents’passionsandisconnectedtostudents’identitiesinwaysthatfosteracademicengagement,self-efficacy,andtenacity;

• Learning for transferemphasizes that themeasureofmastery isapplication in life rather thansimplyintheclassroom;

• Interdisciplinarystudieshelpstudentsseehowdifferingfieldscancomplementeachother,offer-ingaricherperspectiveontheworldthananysingledisciplinecanprovide;

• Personalizedlearningensuresthatstudentsreceiveinstructionandsupportsthataretailoredtotheirneedsandresponsivetotheirinterests(U.S.DepartmentofEducation,2010;Wolf,2010;Rose&Gravel,2010);

• Connectedlearningencouragesstudentstoconfrontchallengesandpursueopportunitiesthatexistoutsideoftheirclassroomsandcampuses(Itoetal.,2013);and

• Diagnosticassessmentsareembedded into learningandare formative for further learningandinstruction(Dede,2012).

Theseentailverydifferentteachingstrategiesthanthefamiliar,lecture-basedformsofinstructioncharacteristicofindustrial-eraschooling,withitsone-size-fits-allprocessingofstudents.Ratherthanrequiringrotememorizationandindividualmasteryofprescribedmaterial,theyinvolvein-depth,dif-

xxx

Foreword

ferentiatedcontent;authenticdiagnosticassessmentembeddedininstruction;activeformsoflearning,often collaborative; and learning about academic subjects linked to personal passions and infusedthroughoutlife.

Thechaptersinthisbookdemonstratethatnewtoolsandmediacanbeveryhelpfultomanyteacherswhowouldotherwisestruggletoprovidethesekindsofinstructionfordeeperlearning(Dede,2014).Byanalogy,imaginethatyouwishtovisitafriend20milesaway.Youcouldwalk(andsomepeoplewouldprefertodoso),butitwouldbemucheasiertouseabicycle,anditwouldbefareasierstilltouseacar.Inshort,teacherswhowishtopreparetheirstudentsfortherealworld,aswellasforfurtheracademics,don’thavetouseeducationaltechnology;theymayprefertowalk.Realistically,however,many,ifnotmost,teacherswillbehard-pressedtogetfromindustrial-styleinstructiontodeeperlearningwithoutthevehiclesofdigitaltools,media,andexperiences.

InanextensivereviewoftheliteratureontechnologyandteachingfortheforthcomingAmericanEducationalResearchAssociation(AERA)HandbookofResearchonTeaching(5thEdition),BarryFishmanandI(inpress)notetheimportantdistinctionbetweenusingtechnologytodoconventionalthingsbetter andusing technology todobetter things (Roschelleet al.,2000).While theremaybevalueindoingsometypesofconventionalinstructionbetter(i.e.,moreefficientlyandeffectively),therealvalueintechnologyforteachingliesinrethinkingtheenterpriseofschoolinginwaysthatunlockpowerfullearningopportunitiesandmakebetteruseoftheresourcespresentinthe21st-centuryworld.

Inourreview,weconsiderhowandunderwhatconditionstechnologycanbeproductivelyemployedbyteacherstomoreeffectivelypreparestudentsforthechallengespresentedbyarapidlyevolvingworld.Wearguethattechnologyasacatalystiseffectiveonlywhenusedtoenablelearningwithrichercontent,morepowerfulpedagogy,morevalidassessments,andlinksbetweenin-andout-of-classroomlearning.Theexaminedthefollowingtechnologiesindepth:

• Collaborationtools,includingWeb2.0technologiesandtoolsthatsupportknowledgebuilding;• Onlineandhybrideducationalenvironments,whichareincreasinglybeingusedtobroadenaccess

toeducationbutalsohavethepotentialtoshiftthewayweconceiveofteachingandlearning;• Toolsthatsupportlearnersasmakersandcreators,whichhavetheirdeeprootsinhelpingstudents

learntobecomeprogrammersofcomputers(andnotjustusersofthem);• Immersivemediathatcreatevirtualworldstosituatelearningoraugmenttherealworldwithan

overlayofcomputationalinformation;and• Gamesandsimulationsthataredesignedtoenhancestudentmotivationandlearning.

Thisbookprovidesexamplesoftheseandotherpowerfultechnologiestoaidthistypeofinstruction.Ifusedinconcert,thesedeeper-learningtechnologiescanhelppreparestudentsforlifeandworkinthe21stcentury,mirroringintheclassroomsomepowerfulmethodsofknowinganddoingthatpervadetherestofsociety.Further,theycanbeusedtocreateapractical,cost-effectivedivisionoflabor,onethatempowersteacherstoperformcomplexinstructionaltasks.Inaddition,thesemediacanaddressthelearningstrengthsandpreferencesofstudentsgrowingupinthisdigitalage,includingbridgingformalinstructionandinformallearning.And,finally,thesetechnologiescanprovidepowerfulmechanismsforteacherlearning;bywhicheducatorsdeepentheirprofessionalknowledgeandskillsinwaysthatmirrorthetypesoflearningenvironmentsthroughwhichtheywillguidetheirstudents.

xxxi

Foreword

Atatimeinhistorywhencivilizationfacescrisesthatweneedthefullcapacityofpeopleacrosstheworldtoresolve,thisvolumeprovidesanexemplarysuiteofpracticalwaystomoveforwardwithcur-ricula,instruction,andassessmentsthataretrulyorientedto21st-centurylifeandwork.

Chris Dede Harvard University, USA

REFERENCES

Dede,C.(2012).Interweaving assessments into immersive authentic simulations: Design strategies for diagnostic and instructional insights(CommissionedWhitePaperfortheETSInvitationalResearchSymposiumonTechnologyEnhancedAssessments).Princeton,NJ:EducationalTestingService.

Dede,C.(2014).The role of technology in deeper learning.NewYork,NY:JobsfortheFuture.

Fishman,B.,&Dede,C.(inpress).Teachingandtechnology:Newtoolsfornewtimes.InD.Gitomer&C.Bell(Eds.),Handbook of research on teaching(5thed.).NewYork,NY:Springer.

Ito,M.,Gutiérrez,K.,Livingstone,S.,Penuel,B.,Rhodes,J.,&Salen,K.…Watkins,S.C.(2013).Connectedlearning:Anagendaforresearchanddesign.Irvine,CA:DigitalMediaandLearningRe-searchHub.

NationalResearchCouncil.(2012).Education for life and work: Developing transferable knowledge and skills in the 21st century.Washington,DC:TheNationalAcademiesPress.Retrievedfromhttp://www.nap.edu/catalog.php?record_id=13398

Roschelle,J.M.,Pea,R.D.,Hoadley,C.M.,Gordin,D.N.,&Means,B.M.(2000).Changinghowandwhatchildrenlearninschoolwithcomputer-basedtechnologies.The Future of Children: Children and Computer Technology,10(2),76–101.doi:10.2307/1602690PMID:11255710

Rose,D.H.,&Gravel,J.W.(2010).Universaldesignforlearning.InE.Baker,P.Peterson,&B.McGaw(Eds.),International Encyclopedia of Education(3rded.).Oxford,UK:Elsevier.doi:10.1016/B978-0-08-044894-7.00719-3

U.S.DepartmentofEducation.(2010).Transforming American education: Learning powered by technol-ogy (National Educational Technology Plan 2010).Washington,DC:OfficeofEducationalTechnology,U.S.DepartmentofEducation.

Wolf, M. A. (2010, November). Innovate to educate: System [re]design for personalized learning.Washington,DC:SoftwareandInformationIndustryAssociation.

xxxii

http://www.nap.edu/catalog.php?record_id=13398

http://www.nap.edu/catalog.php?record_id=13398

http://dx.doi.org/10.2307/1602690

http://www.ncbi.nlm.nih.gov/pubmed/11255710

http://dx.doi.org/10.1016/B978-0-08-044894-7.00719-3

http://dx.doi.org/10.1016/B978-0-08-044894-7.00719-3

Preface

Changesintheworldeconomy,specificallytowardinformationindustries,havechangedtheskillsetdemandofmanyjobs(OrganizationforEconomicDevelopment[OECD],2012a).Informationiscre-ated,acquired,transmitted,andused—ratherthansimplylearned—byindividuals,enterprises,organi-zations,andcommunitiestopromoteeconomicandsocialdevelopment.Majoremployersandpolicymakersare increasinglyasking teachersandeducators tohelpstudentsdevelopso-calledreal-worldskills(Gallup,2013).Whilelearningbasicnumeracyandliteracyskillsstilliscrucialtosuccessinthejobmarket,developingreal-worldskillsalsoisessentialtosuccessinthejobmarketandworldwideeconomicdevelopment.

Real-world skills, or “21st century skills,” include critical thinking, collaborative problem solv-ing,creativity,andglobalcompetency.Theseskillsthatfacilitatemasteryandapplicationofscience,mathematics,languagearts,andotherschoolsubjectswillgrowinimportanceoverthecomingdecade(NationalResearchCouncil,2012;OECD,2012a,2012b).Awiderangeofinitiativesandprogramsineducationpromotelearningandassessmentofreal-worldskills.Theseinclude,forexample,theCom-monCoreStateStandards(NationalGovernorsAssociationCenterforBestPracticesandCouncilofChiefStateSchoolOfficers,2010a,2010b),NextGenerationScienceStandards(NationalResearchCouncil,2013),CommonEuropeanFrameworkofReference(CouncilofEurope,2011),Partnershipfor21stCenturySkills(Partnershipfor21stCenturySkills,2009),EducationforLifeandWork(NationalResearchCouncil,2012),andassessmentframeworksintheProgrammeforInternationalStudentAs-sessment(PISA)(OECD,2013).

Becauseoftheimportanceofpromotingtheseskills,wehaveembarkedonajourneytocreateaHandbook of Research on Technology Tools for Real-World Skill Development.Becauseconceptionsandeducationalapplicationsofreal-worldskillsareevolvingrapidly,wehavewelcomedawiderangeofskillsintheHandbook.Thefollowingfourstrandsofskillsarerepresentedinthechapters:Think-ing skillsrefertohigher-ordercognitionanddispositionssuchascriticalthinking,complexproblemsolving,metacognition,andlearningtolearn.Social skillsrefertoattitudesandbehaviorsthatenablesuccessfulcommunicationandcollaboration.Global skillsrefertoattitudesandbehaviorsthatempha-sizetheindividual’srolein,andawarenessof,thelocalaswellastheglobalandmulticulturalenviron-ment.Digital skillsemphasizeinformationanddigitalliteraciesneededinthetechnology-richworldinwhichwelive.Similarly,thechaptersinthisHandbookdescribearangeoftechnologytoolstosupportteaching,learning,assessmentforlearning(e.g.,Stiggins,2005;Wiliam,2011),feedbackforlearning(e.g.,Hattie,&Timperley,2007;Shute,2008),andscoringofstudentresponses.Forexample,section1includeschaptersoncurriculaandframeworksforteachingreal-worldskills;thechaptersinsection2describespecifictechnologytoolsforteaching,learning,andassessingreal-worldskills;thechaptersin

xxxiii

Preface

section3describeautomatedscoringtoolsforassessmentandlearning;andsection4containschaptersontechniquesforanalyzingdatafromtechnology-basedperformanceassessments.Helpingstudentslearnreal-worldskills—thatis,tointernalizethemandusethemflexiblyacrossarangeofchallengesandcontextsintheireverydayandworklives—isasignificanteducationalchallenge.Real-worldskillscannotbetaughtinasinglecourseorinasingleyearofschooling.Andassessingreal-worldskillstoprovidefeedbacktoguidedevelopmentofthoseskillscannotbeaccomplishedusingconventional,large-scaleassessmentandscorereportingmethodsalone.Thetechnologytoolsdescribedhererepresenttherangeofcurrentanddevelopingcapabilitiesoftechnologytoolstosupportteaching,learning,assess-ment,andfeedbackforlearning.

Astechnology-richenvironmentsforteaching,learning,assessment,andfeedbackarebeingintegratedintoeducationalprocesses,thereismuchtobelearnedabouthowtoleverageadvancesintechnology,learningsciences,andassessmenttodevelopreal-worldskillsforthe21stcentury.Researchfindingsonwhatworksbestarejustemerging,possiblyduetothestrongmulti-disciplinaryapproachesrequiredtoextractthegreatestvalue.ThisHandbookisintendedtoserveasafirstbodyofresearchintheexpand-ingareaoftechnologytoolsforteaching,learning,assessment,andfeedbackonreal-worldskillsthateducatorscanturntointhecomingyearsasareference.Ouraimistobringtogethertopresearcherstosummarizeconceptsandfindings.TheHandbookcontainscontributionsofleadingresearchersinlearningscience,educationalpsychology,psychometrics,andeducationaltechnology.Assumingthatmanyreaderswillhavelittlegroundinginthosetopics,eachchapteroutlinestheoryandbasicconceptsandconnectsthemtotechnologytoolsforreal-worldskilldevelopment.WeseethisasoneofthemostcrucialcontributionsoftheHandbook,seekingtoestablishstrongtheoreticalprinciplesthatcaninformeducationalresearchandpracticeandfutureresearchanddevelopment.TheHandbookalsoprovidesbriefoverviewsineachtopicsectionformoreknowledgeablereaders.TheHandbookisorganizedintofoursections.

SECTION 1: DEFINING REAL-WORLD SKILLS IN TECHNOLOGY-RICH ENVIRONMENTS

ThesevenchaptersinSection1exploreconceptualizationofreal-worldskillsandtheroleoftechnol-ogy.Thesectionincludeschaptersoncurriculaandframeworksforteachingreal-worldskills.Toaidreadersinselectingspecificchapterstostudy,welistthetechnologytoolsdescribedinthesechapters.

Chapter 1:Aprincipledapproachfordevelopingdigitalcompetency.Chapter 2:Amodelforteachingdigitalcompetency.Chapter 3:Amodel formeasuringproblemsolvingskills inscience, technology,engineering,and

mathematics(STEM).Chapter 4:AmodelforteachingInternetresearchskills.Chapter 5:AnothermodelforteachingInternetresearchskills.Chapter 6:AmatrixforevaluatingtechnologyintegrationinK-12instructionalsettings,andteacher-

relatedprofessionaldevelopment.Chapter 7:Anonlineteam-basedlearningmodelinnursingeducation.

xxxiv

Preface

SECTION 2: TECHNOLOGY TOOLS FOR LEARNING AND ASSESSING REAL-WORLD SKILLS

Chapters8through21dealwiththecoretopicoftechnologytoolsandawiderangeofapplicationsaimedatlearningandassessingofreal-worldskills.Thetechnologytoolsdescribedinthesechaptersincludethefollowing.

Chapter 8:Technology-richsimulationsforlearningandassessingscienceskills.Chapter 9:TheCollegiateLearningAssessment,atest toevaluatethecriticalthinkingandwritten

communicationskillsofcollegestudents.Chapter 10:Guidance,basedonlessonslearnedfromdevelopingrich-mediasimulations,forassess-

mentfororganizationstaffpromotionanddevelopment.Chapter 11:Apersonalizedlearningplatformfordevelopingearlyreading.Chapter 12:Computeragenttechnologyforassessingcollaborativeproblemsolvingskills.Chapter 13:Amodelforassessingcognitiveandsocialskillsthroughonlinecollaboration.Chapter 14:Anapproachfortechnology-richlearningandformativeassessmentofcollaborativeprob-

lemsolvingskills.Chapter 15:Aframeworkforprincipledthinkingaboutaconstructmapassessmentofahigher-order

thinkingskills.Chapter 16:Computer-basedandcomputer-assistedapproachesforassessmentofknowledgeandskills.Chapter 17:Technology toolsfor learningforstudentswithmoderateandseveredevelopmentand

intellectualdisabilities.Chapter 18:Strategiesformitigatingbiasforacomputer-administeredperformance-basedassessment

ofhigher-orderskills.Chapter 19:Anevidence-centeredconceptmapforacriticalthinkingassessment.Chapter 20:Facebookasasocialnetworkforlearning.Chapter 21:Aframeworkforteachers’professionaldevelopmentinthedigitalage.

SECTION 3: AUTOMATED ITEM GENERATION AND AUTOMATED SCORING TECHNIQUES FOR ASSESSMENT AND FEEDBACK

ThefivechaptersinSection3addressarangeoftechnologiesforautomatedscoring,automateditemgeneration,andlearnerfeedback.Thetechnologytoolsdescribedinthesechaptersincludethefollowing.

Chapter 22:Proceduresforautomatedgenerationofscienceitems.Chapter 23:Automatedscoringapproachesfordevelopmentofwritingproficiency.Chapter 24:Aprincipledframeworkfordesigningautomatedscoringofmulticomponentassessment

tasks.Chapter 25:Automatedscoringasthebasisforfeedbacktosupportimprovementofwritingskills.Chapter 26:Automatedfeedbacktoimprovewritingquality.

xxxv

Preface

SECTION 4: ANALYSES OF PROCESS DATA IN TECHNOLOGY-RICH PERFORMANCE TASKS

Chapters27through31dealwithanalysis,interpretation,anduseoflearningandassessmentdataintechnologyenvironments.Thetechnologytoolsdescribedinthesechaptersincludethefollowing.

Chapter 27:Analysisofsolutionpathsinatechnology-richproblemsolvingassessment.Chapter 28:Analysisofsolutionpathsintechnology-richcriticalthinkingassessment.Chapter 29:Useofachi-squarefeaturesselectionalgorithm(i.e.,sequentialpatternmining)andN-

gramsrepresentationmodeltoanalyzeprocessdataintechnology-richproblemsolvingtasks.Chapter 30:Analyticmethodstoinduceapersistencemeasurefromgameplayclickstreamdataanda

designpatterntoguidefuturedevelopmentofpersistencemeasuresindigitalenvironments.Chapter 31:AnItemEngagementIndex(IEI)andStudentEngagementIndex(SEI)forassessingen-

gagementduringtheonlineassessmentofreal-worldskills.

Ourgoalincollectingandorganizingtheseexcellentchaptersistobeginaprocessofcrystalizingwhatourfieldhasaccomplishedtodateandwhatitknows,collectively,abouttechnologytoolsandhowthosetoolscanbeusedtosupportandenhanceteachingandlearningofreal-worldskills.Knowingwhatweknowshouldhelpusidentifywhatweneedtoknow.Anditshouldguidefurtherdevelopmentofpracticalapplicationsandempiricalresearchontheefficacyofusingtechnologytoolsforteaching,learning,assessing,andprovidingfeedbackaslearnersworktodeveloptheskillstheyneedfortoday’shigh-tech,higher-orderknowledgeandskillsworld.WehopethisHandbookwillserveasatooltoen-couragecollaborationsamongresearchers,educators,policymakers,employers,andthegeneralpublictopromotelearning,assessment,andpersonalizedfeedbacktechnologies.BycompilingtherichresearchandknowledgeinthisHandbook,wehopetosparkinnovationineducation.

TheHandbookisarecommendedreadingsourcetothefollowingaudiences:Educators:Thisbookwillshareessentialinsightsforpolicymakers,principals,curriculumexperts,

andteacherswhoareinterestedinbetterunderstandingthepracticalchallengesandopportunitiesinintroducingnewtechnology-richprogramsaimedtopromote learning,assessment,andfeedbackonreal-worldskills.

Researchers:Thisbookwillprovideavaluablespringboardtoresearchersinpsychology,educa-tion,assessment,andcomputersciencetoengagewiththeconceptoftechnology-richassessmentandlearningofhigher-orderthinkingskillsandworkonnewresearchdirections.Thiswillbeaidedbytheemphasisofkeygapsinexistingresearchandprovidingdetailsonwhatareasneedmorecarefulresearchandempiricalvalidation.

General audienceswithinterestinupcomingtrendsinlearning,assessment,andfeedback:Thisbookwillcoverarangeoftopicsrelatedtoreal-worldskillsandvalueofreal-worldskillsinnext-generationeducation.

xxxvi

Preface

REFERENCES

CouncilofEurope.(2011).Common European framework of references for languages: Learning, teach-ing, assessment.Strasburg:Author.

Gallup.(2013).21st century skills and the workplace: A 2013 Microsoft-Pearson Foundation study on 21st century skills and the workplace.Washington,DC:Author.

Hattie,J.,&Timperley,H.(2007).Thepoweroffeedback.Review of Educational Research,77(1),81–112.doi:10.3102/003465430298487

NationalGovernorsAssociationCenterforBestPracticesandCouncilforChiefStateSchoolOfficers.(2010a).Common core state standards for mathematics.Washington,DC:Author.

NationalGovernorsAssociationCenterforBestPracticesandCouncilforChiefStateSchoolOfficers.(2010b).Common core state standards for English language arts and literacy in history/social, science, and technical subjects.Washington,DC:Author.

NationalResearchCouncil.(2012).Education for life and work: Developing transferable knowledge and skills in the 21st century.Washington,DC:TheNationalAcademiesPress.

NationalResearchCouncil.(2013).Next generation science standards: For states, by states.Washington,DC:TheNationalAcademiesPress.

OrganizationforEconomicDevelopment(OECD). (2012a).Better skills, better jobs, better lives: A strategic approach to skills policies.OECDPublishing.

OrganizationforEconomicDevelopment(OECD).(2012b).Education at a glance 2012: OECD indica-tors.OECDPublishing.

OrganizationforEconomicDevelopment(OECD).(2013).PISA 2015 collaborative problem solving framework.OECDPublishing.

Partnershipfor21stCenturySkills.(2009).P21 framework definitions.Washington,DC:Author.

Shute,V.J.(2008).Focusonformativefeedback.Review of Educational Research,78(1),153–189.doi:10.3102/0034654307313795

Stiggins,R.J.(2005).FromformativeassessmenttoassessmentFORlearning:Apathtosuccessinstandards-basedschools.Phi Delta Kappan,87(4),324–328.doi:10.1177/003172170508700414

Wiliam,D.(2011).Embedded formative assessment.Bloomington,IN:SolutionTreePress.

xxxvii

http://dx.doi.org/10.3102/003465430298487

http://dx.doi.org/10.3102/0034654307313795

http://dx.doi.org/10.1177/003172170508700414

Acknowledgment

Inthewritingandeditingthisbookwehavetothanktheconceptualvisionarieswhopushedourthinkingandcontributedgreatlytotheformationoftheideaspresentedhere.AtPearson,Dr.KimberlyO’Malley,athoughtfulleaderwhoprovidedexecutivesupportduringallthestagesofourworkontheHandbook.Ourcolleagues,Dr.PeterFoltzandDr.KatieMcClartywhohavebeenkeyindevelopingtheseideasaspartof21stCenturySkillsproject.Prof.AndreasSchleicher,OECDandProf.ChrisDede,HarvardUniversitysharedtheirinvaluableinsightsanddirectionsforfurtherresearchinForewordsectionoftheHandbook.Andthenthereistheoutstandinggroupofauthorsfromawiderangeoforganizationsandgeographieswhocontributedtheirchapterstothisvolume.Alongtheway,theauthorsgraciouslyservedaseachother’sreviewersaswepasseddraftsaround,nurturingeachother’schaptersandaddingnewperspectives.WethankyouallformakingourownworkontheHandbookagreatpleasure.

xxxviii

Editorial Advisory BoardChrisDede,Harvard University, USAKimberlyO’Malley,Pearson, USAAndreasSchleicher,Organisation for Economic Co-Operation and Development (OECD), France

About the Contributors

Yigal RosenisaSeniorResearchScientistinTheViceProvostforAdvancesinLearning(VPAL)atHarvardUniversity.HeleadstheHarvardVPALResearchTeamthatenablesfacultytocreateandimplementresearch-enabledonlinelearningexperiencesforresidentialandonlineuse,suchasHarvardXMassivelyOpenOnlineCourses.PriortojoiningHarvardhewasaSeniorResearchScientistatPearson,leadingresearchanddevelopmentofinnovativelearningandassessmenttechnologiesforreal-lifeskilldevelopment.YigalwasaHeadofAssessment&EvaluationatTimeToKnow.HewasamemberofOECD’sPISA2015AssessmentFrameworkExpertGroupandconsultedtoMicrosoft,Intel,ETS,Min-istryofEducationinIsraelandtheNationalAuthorityforMeasurement&Evaluation.YigalpreviouslyheldacademicappointmentsattheUniversityofHaifa,theOpenUniversityofIsraelandBen-GurionUniversity.In2009hereceivedtheUniversityofHaifaFacultyofEducation’sOutstandingLecturerAward.HeobtainedhisPh.D.degreefromtheUniversityofHaifa.Yigalwasapost-doctoralfellowatHarvardUniversityGraduateSchoolofEducationandatTelAvivUniversitySchoolofEducation.Hebeganhiscareerasacomputerscience,mathandscienceteacheringrades5through12.Yigalco-editedthebookduringhisworkasaSeniorResearchScientistatPearson.

Steve FerraraisVicePresident,PerformanceAssessment,andleadstheCenterforNextGenerationLearningandAssessmentinPearson’sResearchandInnovationNetwork.Thecenterdesigns,develops,validates,andconductsresearchonperformanceassessmentforsummativeandformativeusestosup-portlearning.PriortojoiningPearson,StevewasaPrincipalResearchScientistatCTB/McGraw-Hill,ManagingResearchDirectorintheassessmentprogramatAmericanInstitutesforResearch,andStateAssessmentDirectorinMaryland,andaHeadStartandhighschoolspecialeducationteacher.SteveconductspsychometricresearchanddesignslargescaleandformativeassessmentsforK-12educationalachievement.Heservesontechnicaladvisorypanelsforassessmentprogramsandresearchprojects.HewasEditorofEducationalMeasurement:IssuesandPracticeandservesonjournaleditorialadvisoryboards.Stevewasco-recipient,AERACognitionandAssessmentSpecialInterestGroup2014awardforOutstandingContributiontoPracticeinCognitionandAssessment;andco-recipient,AERADivisionD2006awardforSignificantContributionstoEducationalMeasurementandResearchMethodology.

Maryam MosharrafhasbeenworkingatPearsonsince2009.Shehasmanagedanumberofna-tionalandglobalassessmentproductlines.InhermostrecentrolesheworksasaResearchtoPracticeManager.Inthisroleshebringsinnovation,researchandnewtechnologyintoeducationalsolutionsforteachersandstudents.Maryamhasconductedresearchondeveloping innovative21stcenturyskillsassessmenttaskswhichhavebeenadministeredinTurkey,SouthAfrica,Singapore,UK,USandothercountries.SomeofherpreviousexperienceincludesworkingatProcterandGamble.Priortomoving

cxxix


totheStatestoattendbusinessschool,MosharraflivedinIranandworkedattheBritishEmbassyinTehranasaninterpreter.Sheholdsamaster’sdegreeinBusinessAdministrationfromtheUniversityofTexasinAustinandabachelor’sdegreeinTranslationfromAzadUniversity.

***

Lynne Anderson-InmanisaprofessorofeducationattheUniversityofOregonandpastDirectoroftheuniversity’sCenterforAdvancedTechnologyinEducation.Dr.Anderson-Inmanisanationallyandinternationallyrecognizedexpertontheuseoftechnologytoimprovereading,writing,andstudy-ing,withspecialemphasisonstrategiesforusingtechnologytoenhancetheacademicsuccessofstrug-glinglearners.Shepioneeredtheconceptof“computer-supportedstudying,”conductingresearchonstrategy-basedinstructiontopromoteeffectivereadingandlearningonline.Recentworkhasfocusedoninvestigating“supportedetext”asameanstoimprovecontent-areareadingandstudying.Dr.Anderson-Inmanisawidelypublishedauthoroneffectiveusesoftechnologyandafrequentspeakeratnationalandinternationalconferences.

Gilbert Andrada,PhD.,StateEducationConsultant,hasbeenwiththeConnecticutStateDepart-mentofEducationfor21years.InadditiontohavingbeentheprogrammanagerfortheConnecticutBenchmark Assessment System (CBAS), his duties involve psychometric and statistical analyses,appliedresearchprojects,programevaluations,andlarge-scalestudentassessment.HeholdsaCon-necticutschooladministrator’slicense(092).GilisanactiveresearcherwithcolleaguesfromtheNeagSchoolofEducationattheUniversityofConnecticut.Heisafrequentadjunctfacultymemberteachingcoursesinstatistics,measurementandresearchmethods.GilholdsaBAinpsychologyfromCaliforniaStateUniversity,Stanislaus;anMSinPsychologyfromtheSchoolofPsychologicalSciencesatPurdueUniversity;andaPhDinEducationalPsychologyfromtheMeasurement,EvaluationandAssessmentprogramattheUniversityofConnecticut,NeagSchoolofEducation.

Perla Arie,M.Ed.HonorsDegreeinEducationalTechnologyfromKibbutzimCollegeofEducation,TechnologyandtheArts.EFLandEnglish-HebrewTranslationteacheratKugelHighSchool(Holon,Israel).Mainfieldsofinterest:Languageteaching(English,FrenchandSpanish),Innovativeteachingmethods,ComputerAssistedLearning,Web2.0toolsandFacebookuseforeducationalpurposes.

Edith Avni,Ph.D.inEducation,HaifaUniversity,Israel.ChargeofICTinEducation,MinistryofEducation,HaifaDistrict,Israel.Lecturerinprofessionaldevelopmentcoursesforeducationalleadershipandteachers.EstablishingwithDrA.Rotem“TowardDigitalEthicsInitiative”,Israel,containsoriginalarticlesrelatedtotechnology,pedagogicalandethicalaspectsinlearningandteaching.

Susan Malone Back,PhD,MBA,isaResearchScientistintheInstituteforMeasurement,Meth-odology,AnalysisandPolicyoftheCollegeofEducationatTexasTechUniversityandDirectoroftheuniversitySmallBusiness InnovationResearch/SmallBusinessTechnologyTransfer (SBIR/STTR)ResourceCenter.Shehasextensiveexperiencemanaging federal andprivate foundationgrantsandworkingwithpublic/privatepartnerships.Herareasof researcharecommunityengagement,STEMeducation,andnewfrontiersinbusinesseducation.Dr.BackreceivedherM.Ed.andPhDinEducationalPsychologyfromTempleUniversityandherMBAfromtheUniversityofDenver.

cxxx


Sara Bakken’seducationcareerbeganin2001asahighschoolscienceteacher.Afterleavingtheclassroom,sheworkedasaninstructionaldesignerandadvocateforonlinelearninginboththeK-12andhighereducationenvironments.ShehasworkedextensivelywiththeTexasVirtualSchoolNetwork(TxVSN),andservedascommitteeco-chairforthe2011publicationoftheInternationalAssociationforK-12OnlineLearning(iNACOL)NationalStandardsforQualityOnlineTeaching.Shehaspresentedontheeffectiveincorporationof21stcenturyskillsintothevirtualclassroomatseveralnationalconferencesincludingtheTexasDistanceLearningAssociation(TxDLA)andiNACOL’sVirtualSchoolSympo-sium.ShenowdevelopsnewandinnovativeonlineassessmentsatPearson,whereshewonthePearsonInnovativeAssessmentIdeacompetitionin2012.SheholdsaMasterofSciencedegreeininstructionaldesignfromtheUniversityofHouston-ClearLake.

Kirk A. BeckerisaseniorresearchscientistwithPearsonandconsultingeditorfortheJournalofComputerAdaptiveTesting.Hehasoverfifteenyears’experienceconductingresearchandperformingoperationalworkinmeasurementandpsychometrics.Dr.Becker’scurrentresearchfocusesonefficientmethodsforevaluatingandcodinglargeitempoolsusingnaturallanguageprocessing,especiallywithinthecontextofadaptivetesting,LOFT,andautomatedtestassembly.Thisresearchonnaturallanguageprocessingiscurrentlybeingextendedtoautomateditemgeneration,itemmodeling,andtheevaluationofitemquality.AdditionallyKirkhascontinuedtoresearchmethodsfordeveloping,evaluating,andscoringinnovativetestitemsandperformancetasks,withcurrentresearchlookingattheapplicationofpolytomousscoringmodelsoninformationfunctionsandadaptiveadministration.Dr.BeckerhaspresentedresearchonapplicationsofnaturallanguageprocessingattheannualconferencesoftheAmeri-canEducationalResearchAssociation,theAssociationofTestPublishers,andtheNationalCouncilonMeasurementinEducation.KirkreceivedhisPh.D.inEducationalPsychologyfromtheUniversityofIllinoisatChicago.

Valerie Beltran,Ed.D.,isanAssociateProfessorofEducationattheUniversityofLaVerne.IntheK-12field,shehasbeenateacher,aresourcespecialist,andaprincipal.Attheuniversitylevel,sheteachescoursesinboththeteachercredentialandMaster’sprograms.ShechairstheMaster’sofEduca-tionSpecialEmphasisprogram.Herresearchinterestsincludewritinginstruction,bilingualeducation,onlineteaching,andstudentengagement.

Roger BenjaminhasbeenPresidentofCAE(CouncilforAidtoEducation)since2005.HewasaresearchscientistatRANDfrom1990to2005(directorofRANDEducation,1994-1999).PrevioustohisappointmenttoRAND,hewasprofessorofpoliticalscienceattheUniversityofMinnesota,1966to1983andassociatedeanandexecutiveofficer,CollegeofLiberalArts,UniversityofMinnesota,1980to1983,vicechancellorforacademicaffairsandprovostattheUniversityofPittsburgh,1983to1986,andvicepresidentforacademicaffairsandprovost,UniversityofMinnesota,1986to1988,andprofessorofpoliticalscience,1988to1990.Heistheauthororco-authorofnumerousbooks,monographsandarticlesinpoliticaleconomyandpublicpolicy,includinggovernance,strategicplanning,andeduca-tionalassessment.HislatestbookisTheNewLimitsofEducationPolicy:AvoidingATragedyoftheCommons.London:EdwardElgar,2012.HedirectsCAE,aneducationalassessmenttestingorganiza-tion,implementingperformanceassessmentsandrelated21stcenturyassessmentsthroughouttheK-16educationsystemintheUnitedStatesandbeyond.

cxxxi


John Bielinskihasworkedineducationalassessmentasadeveloperandresearchersince1995.HeearnedaPhDinquantitativemethodsthroughtheEducationalPsychologydepartmentattheUniversityofMinnesota.Hisresearchincludesgenderdifferencesinmathachievement,testaccommodationvali-dation,DIF,andgrowthmodeling.SincejoiningPearsonin2002,Johnhasdirectedthedevelopmentofseveralclinicalandeducationalassessment includingtheKeyMath-3DiagnosticAssessmentandtheBruininks-OseretskyTestofMotorProficiency,2ndEdition,andaimsweb.Heiscurrentlyleadingdevelopmentonnextgenerationcurriculum-basedmeasurementandformativeassessment.Inadditiontohisresearchanddevelopmentwork,Johnvolunteersasamiddleschoolmathcoach.

Keith A. BoughtonisaSeniorResearchScientistatCTB/McGraw-HillEducation.Hisareasofspe-cializationincludeautomaticitemgeneration,adaptivetesting,mixture/latentclassmodels,hierarchicalIRTmodels,andmultidimensionalIRTmodels.

Daniel G. Brenner, a Senior Research Associate at WestEd’s Science, Technology, Engineer-ing,andMathematics(STEM)program,isCo-PrincipalInvestigatoronmultipleprojectsdevelopingsimulation-basedassessmentsinvarioussciencecontentareas:theNSF-fundedHumanBodySystemand SimScientists Assessment System: Physical Science Links projects as well as the IES-FundedSimScientistsAssessmentSystem:LifeSciencesandSimScientistsModelProgressions(GeneticsandEvolution)providingscientificcontentexpertise,managingtheassessmentprogramming,andoverseeingtheSimScientistslearningmanagementsystemthatservesover10,000studentsannually.Dr.BrennerisalsoPrincipalInvestigatorontheInstituteofEducationSciences-fundedVoyagetoGalapagosprojecttodevelopadifferentiatedassistancemodelthatisbasedonknowledgeaboutwhatkindsofassistancearemosteffectiveforstudentsofdifferingabilitiesastheyworkinaninquiry-drivenlearningenviron-mentforevolutionsciencecontent.Brenner’sresearchinterestsfocusononlineeducationandassess-ment,andontheapplicationofnewtechnologiestotheseareas,andonapplicationoftheartstoSTEMresearch.Hehasdoneextensiveresearchandprojectmanagementinacademicandindustrialresearchintheareasofbioorganicchemistryandbiopharmaceuticals.PriortojoiningWestEd,Brennerworkedasaninstructionaldesignerheadingtheproductionofover500in-depthonlinemediarichcoursesfortheAcademyofArtUniversity.Dr.BrennerreceivedanABinChemistryandVisual&EnvironmentalStudiesfromHarvardCollege,andaPhDinChemistryfromHarvardUniversity.

Diane BrowderistheLakeandEdwardP.SnyderDistinguishedProfessorofSpecialEducationattheUniversityofNorthCarolinaatCharlotte.Dr.Browderhasovertwodecadesofresearchandwritingonassessmentandinstructionforstudentswithseveredevelopmentaldisabilities.Dr.Browder’sresearchhasbeenrecognizedthroughmultipleawards.Shereceivedthe2009DistinguishedResearcherAwardinSpecialEducationfromtheAmericanEducationResearchAssociationandwasthe2009FirstCitizensBankScholarattheUniversityofNorthCarolinaatCharlotte.In2011Dr.BrowderwasrecognizedbythestateofNorthCarolinawiththeO.MaxGardnerAwardforresearchthathasmadeacontributiontohumanity.

Barbara Buckley, a Senior Research Associate at WestEd’s Science, Technology, Engineering,&Mathematics(STEM)programisPrincipleInvestigatoroftheSimScientistsHumanBodySystems(HBS)projectfundedbyNSF(DRK12),andco-PIoftwocurrentSimScientistsprojectsinthelifesci-encesfundedbyIES.Asaproponentofmodel-basedlearning,Dr.Buckleyguidestheevidence-centered

cxxxii


designprocessusedbytheSimScientiststeamtocreatesimulation-basedinstructionalandassessmentsuitesthatincorporatemodel-basedlearning,complexsystemsthinking,andNextGenerationScienceStandards.PriortojoiningWestEd,Dr.BuckleywasaresearcherattheConcordConsortium,playingacentralroleintheModelingAcrosstheCurriculumProject(IERI#0115699).AtTERC,shecontributedtotheGLOBEandGlobalLabprojects.

Michael B. BunchisSeniorVice-PresidentofMeasurementIncorporated,atestdevelopmentandscoringcompanyservingthelarge-scaleassessmentandprofessionallicensure/certificationfields.Heoverseesaresearchanddevelopmentunitofover80assessmentprofessionalsandservesasacompanydirector.Dr.Bunchearnedadoctorateinpsychologywithaspecialtyinmeasurementandhumandif-ferencesfromtheUniversityofGeorgiain1976.PriortojoiningMIin1982,heservedasresearchpsychologistattheAmericanCollegeTestingProgram(ACT)andseniorprofessionalatNTSResearchCorporation.SincejoiningMI,hehasbeenactivenotonlyintheprofessionbutintheDurhamcom-munityaswell,servingonandchairingboardsoflocaleducationandhumanserviceorganizations.HeispastchairmanoftheGraduateEducationAdvancementBoardoftheUniversityofGeorgia.

Dolores CannellaearnedherPhDinSocialandHealthPsychology.SheistheDirectorofBehavioralSciencesandtheActingChairintheDepartmentofGeneralDentistryatStonyBrookUniversity,SchoolofDentalMedicine.Herteachingandresearchinterestsfocusonthedevelopmentofpatient-centeredinterprofessionalhealthcareteamstoimprovehealthoutcomes.

E. Lee ChildresshasbeensuperintendentoftheCorinthSchoolDistrictfor13years.AnativeofClarksdale,Mississippi,ChildressisagraduateofDeltaStateUniversity.HeservedasasocialstudiesteacherintheClarksdalePublicSchoolsbeforebecomingProgramManagerfortheSchoolExecutiveManagementInstitutefortheMississippiDepartmentofEducation,whichprovidedtrainingforover2,000Mississippischooladministrators.ChildresswasalsoDirectorofSchoolImprovementfortheDeSotoCountySchoolDistrictbeforeservingfornineyearsasAssistantSuperintendentfortheCorinthSchoolDistrict.Dr.ChildresshasbeenactiveinseveraleducationalorganizationsandiscurrentlyservingasaTrusteeforMississippiPublicEmployeesRetirementSystemrepresentingpublicschoolandcom-munitycollegeeducators;servesasChairmanoftheMississippiCommissiononSchoolAccreditation,President,oftheProgramforResearchandEvaluationinPublicSchools;andservesontheAdvisoryBoardfortheMississippiSchoolforMathandScience.

Rafi DavidsonreceivedhisPH.DinBiochemistryfromtheWeizmannInstituteinRechovot,Israel.Heisaseniorlecturerinthedepartmentofscience&technologyinKayeCollegeofeducation,where,for20years,heteachescoursesforB.Ed.andM.ED.inthesubjectsof“ICTineducation”,“informationliteracyandcriticalthinking”and“teachingandlearninginaninformationage”.In1999hefoundedtheCenterforEducationalTechnologyInitiativeswhichsupports,advisesandprovidespedagogical-technologicalassistancetolecturersandstudentatKayeCollege.Forthelast4yearsheisresponsiblefor thedevelopmentand implementationofprogramsforprofessionaldevelopmentof lecturersandpedagogicalsupervisorsinthecollegesforeducationinIsrael.TheseprogramsarepartofthenationalprogramfortheadaptationoftheIsraelieducationsystemtothe21stcentury.HisareasofresearchanddevelopmentincludeintegrationofICTinteachingandlearning,socialandemotionalaspectsofonlinelearningandthefactorsthatinfluencethedigitaldivide.

cxxxiii


Jessica Decker isanAssistantProfessorofEducationattheUniversityofLaVerne.Previously,shehasworkedasanelementaryschoolteacher.Herareasofresearchincludeimprovingwritingskills,technology,andmentoring.

Kristen DiCerbo isaPrincipalResearchScientistintheCenterforDigitalData,Analytics,andAdaptiveLearningatPearson.Herresearchprogramcentersonhowtouseevidencefromlearneractivityingamesandsimulationstounderstandwhatlearnersknowandcando.Dr.DiCerbohasalsoengagedwithteacherstounderstandhowtobestcommunicateinformationaboutstudentperformancetoinforminstructionaldecisions.PriortojoiningPearson,KristenprovidedresearchsupporttotheNetworkingAcademiesatCiscoandworkedasacertifiedschoolpsychologist.KristenreceivedherMaster’sDegreeandPh.D.inEducationalPsychologyatArizonaStateUniversity.ShereceivedaB.A.inPsychologyandSociologyatHamiltonCollege.

Patricia EckardtisaneducatorwithaPhDinQuantitativeMethodsforResearch,EducationalPsy-chology.Shehastaughtresearchandstatisticstonursesandhealthprofessionalsacrossundergraduateandgraduatestudies.Shehasbeenaregisteredprofessionalnurseforoverthirtyyears.

Sammy ElzarkaistheDirectoroftheCenterfortheAdvancementofFacultyExcellenceattheUni-versityofLaVerneinLaVerne,California.Hehasworkedineducationfornearly20yearsincludingrolesasteacher,curriculumdeveloper,assessmentanalyst,andaccreditationliaison.Hiscurrentfocusisontheeffectiveuseofnewtechnologiesforteachingandlearning.

David P. Erlangerisaneducatorandresearchdirector.Hisinterestsincludehowtoapplyeducationaltheorytoensurestudentslearnbestinbothtraditionalandnon-traditionalsettings.Hehasafocusonidentifyingmethodsneededtomakedistancelearningeffective,especiallyforresearchstudents.

Peter FoltzisaVicePresidentforresearchanddevelopmentandworkstobringinnovativetechnolo-giestolearningandassessment.Heisoneoftheoriginaldevelopersofautomatedscoringtechnologiesandholdsapatentonmethods forscoringofwriting.Dr.Foltz’s researchhas focusedon languagecomprehension,21stCenturyskillslearningandassessment,andusesofmachinelearningandnaturallanguageprocessingineducationaltechnology.Themethodshehaspioneeredimprovestudentachieve-ment,expandstudentaccess,andmakelearningmaterialsmoreaffordable.HehasledtheframeworkdevelopmentforanewassessmentofcollaborativeproblemsolvingfortheOrganisationofEconomicCooperationandDevelopment’sProgrammeforInternationalStudentAssessment(PISA)test.AformerprofessorofpsychologyatNewMexicoStateUniversity,hehasauthoredmorethan100journalarticles,bookchapters,conferencepapers,andotherpublications.HepreviouslyworkedatBellCommunica-tionsResearchandtheLearningResearchandDevelopmentCenterattheUniversityofPittsburgh.Dr.Foltzholdsdoctorateandmaster’sdegreesinCognitivePsychologyfromtheUniversityofColorado,Boulder,andabachelor’sdegreefromLehighUniversity.

Kellilynn M. FriasisanAssistantProfessorintheMarketingDepartmentatTexasTechUniversity.Her researchagenda focusesonunderstanding the technology, firm, andenvironmental factors thatimpactafirm’sdecisionof“wheretoparticipateinthevaluechain”or“whattosell”.Thisresearchhasimplicationsforproductdevelopmentandinnovation,designofinter-firmties,andintellectualpropertyenforcement.cxxxiv


Mark Gierl isProfessorofEducationalPsychologyandtheDirectorof theCentreforResearchinAppliedMeasurementandEvaluation(CRAME)attheUniversityofAlberta.Hisspecializationiseducationalandpsychologicaltesting,withanemphasisontheapplicationofcognitiveprinciplestoassessmentpractices.ProfessorGierl’scurrentresearchisfocusedonautomaticitemgenerationandautomatedessayscoring.HisresearchisfundedbytheMedicalCouncilofCanada,Elsevier,ACTInc.,andtheSocialSciencesandHumanitiesResearchCouncilofCanada.Dr.GierlholdstheTierICanadaResearchChairinEducationalMeasurement.

Amnon GlassnerwastrainedasamathteacherandthencompletedhisM.A.andPhDinthecenterofcognition,instructionandcomputersattheHebrewUniversityinJerusalem.HeparticipatedinsomeresearchesaboutlearningandargumentationintheframeofKishurimGroupintheleadershipofProf.BaruchSchwarz.Hiscurrenteducationalinterestistoleadsomenewprogressiveprogramsofteachertraining,suchasthosewhousePBL(projectorproductbasedlearning)asmaindirectionforlearningandinstruction.HeservesastheheadofEducationDepartment,theheadofInformalStudiesandapedagogicalguideinKayeAcademicCollegeofEducationinBeer-Sheva.Hiscurrentresearchesdi-rectionsincludeinfusionofcreativeandcriticalthinkingduringlearningofanycontent,learningwithPBL,learningbysuccesses,andmoderationofdialogicallearningdiscussion.

Heather Greenhalgh-Spencer isanAssistantProfessorofCurriculumand InstructionatTexasTechUniversity.Herresearchfocusesondiversityandequityissuesinonlineeducation.Thisresearchhasimplicationsforthewaysweteachinonlinespaces,andimplicationsforcreatinggreateraccessandengagementineducationforunderrepresentedstudents.

Sean T. Hanloncreatesstudent-andteacher-centerededucationaltechnologyapplicationsthatblendassessmentandinstruction.HereceivedhisundergraduatedegreeincomputersciencefromTheGeorgeWashingtonUniversity(GW),hismaster’sdegree incomputerscienceandhisPh.D. ineducationalpsychologyfromtheUniversityofNorthCarolinaatChapelHill.DuringhissenioryearatGW,SeanreceivedtheGaryBardEntrepreneurawardforBlaise,anonlineessay-writingsystemwithgesture-basedscoring.ThisapplicationwasdeployedintheGWWritingCenterandusedinthecompulsoryfreshmencompositioncourse.SincejoiningMetaMetrics,Seanhas ledthedesign,development,deployment,and enhancement of multiple educational technologies including MyWritingWeb, MyReadingWeb,MyMathWeb,LearningOasis,EdSphere,ModuleCreator,andGuidedLiteracy.Seansupplementshistechnologicalexpertisewiththeinstructionalandassessmentprinciplesnecessarytocreateresearch-basedtechnologiesthatimpacttheeverydaylivesofstudentsandteachers.

Jiangang HaoisaResearchScientistintheCenterforAdvancedPsychometricsatETS.PriortojoininginETS,hehasbeenworkingonmodelingandminingTerabyte-scaledatainphysicsatFermiNationalAcceleratorLaboratoryaftergettinghisPhDinphysicsandMAinStatisticsbothfromtheUniversityofMichigan.Hehasbeenworkingextensivelyondatamining&machinelearning,statisticalmodeling&inference,datastandardizationanddatamodelschemadesignforbigdata.Hepublishedover40papersinpeer-reviewedjournalswithover2400totalcitationsandhisworkhasbeenreported

cxxxv


byWired,MIT Technology Reviewandetc.Jiangang’scurrentresearchcentersoncollaborativeproblemsolving,gameandsimulationbasedassessment,educationaldatamining,logfilestructuringandanalysis.Heisalsoapplyingnaturallanguageprocessingtechniquestoanalyzetheconversationsincollaborationtodevelopintelligent(adaptive)facilitationmechanism.

J. Christine Harmesisaconsultantintheareasofassessment,measurement,andtechnologyin-tegration.SheholdsaPh.D.withdualconcentrationsinEducationalMeasurementandResearchandInstructionalTechnology.Dr.Harmes’researchincomputer-basedtestinghasmostrecentlyfocusedonthedesign,cognitiveprocessing,memorability,andusabilityofinnovativeitems.HerrecentprojectsregardingtechnologyintegrationinK-12settingshaveincludeddevelopingtoolsforteacherprofessionaldevelopment with and use of technology, and developing technology-related assessments, observa-tiontools,andsurveys.Shehastaughtcoursesinmixedmethodsresearch,scholarlycommunication,performanceassessment,computer-basedtesting,measurementforteachers,andinstructionaldesign.

Qiwei HeisanAssociateResearchScientistinETS’sResearch&DevelopmentDivision,GlobalAssessmentCenter.Herbackgroundandexpertiseisinpsychometrics,datamining,textminingandnaturallanguageprocessingwithafocusonbuildingapplicationsforeducationalandpsychologicaltechnology.Shecurrentlyfocusesonexplorationresearchofprocessdatainlargescaleassessments.Previously,Dr.Hehasconductedresearchintheareasoftext-basedonlineassessmentforpsychiatricandpsychologicalassessmentintheUniversityofTwente,theNetherlands.ShealsowontheBestDis-sertationPrizeofAbbasFundinthefieldofpsychologicalassessmentsintheNetherlands.

Bridget Hildreth,M.F.A.,isaResearchAssistantattheCenterforAdvancedTechnologyinEduca-tionattheUniversityofOregon.ShewasasecondaryteacherandELAInterventionSpecialistforovertenyears,andshetaughtwriting,remedialELA,andeLiteracyatthecommunitycollegelevel.Herre-searchwithfederallyfundedprojects(SSOAR,SAIL,SOAR,andMeTRC)primarilyinvolvesimprovingeducationoutcomesformiddleschoolstudentswithlearningdisabilitiesthroughtheuseoftechnology.

Cheryl Johnsonhasworkedinprofessional,clinical,andeducationassessmentdevelopmentsince1998.Shehaswrittencontentandmanagedteamstosuccessfullyexecutemulti-yearassessmentresearchprojectsthatincludetheKaufmanAssessmentBatteryforChildren,2ndEdition,theKaufmanTestofEducationalAchievement,2ndand3rdEditions,SensoryProfile,2ndEdition,andaimsweb.Shecur-rentlyleadsresearchonearlyliteracy,andeffectiveprogressmonitoringtowardachievementgoalsforschool-agedchildren,asResearchDirectoratPearson.Inadditiontoherresearchwork,sheisanadvocateforMontessorieducation,approachestohelpmaximizecognitiveandphysicalperformanceinadults,andnewtechnologyandexperiencesthatfosteraperson’snaturalcuriosity.

Carolyn KnoxhasaPh.D.inDigitalLearningwithanemphasisonstudentswithspecialneedsandEnglishLearners.SheisAssistantDirectorofCATEattheUniversityofOregon.Dr.KnoxisPIoftheSteppinguptoSOAR:StrategiesforOnlineAcademicReadingproject,developingboththeproject’sstrategiesanditsvideo-basedinstructionalwebsite.ShewasaseminalpartofthedevelopmentofCATE’soriginal“computer-basedstudystrategies”forusingtechnologytoenhancetheacademicsuccessofstrugglinglearners.Sheisanationallyrecognizedexpertontheuseofdigitalnote-takingsystemsinthe

cxxxvi


classroomforstudentswithdisabilities.Forthelast25years,Dr.Knoxhasbeenconductingresearchanddevelopmentprojectsthatcreateandpromoteinnovativeusesoftechnologyforlearning,innovativeinstructionalmaterials,andevidence-basedstrategiesdesignedtohelpstrugglinglearnersusetechnol-ogyforeffectivelearningandacademicsuccess.

Zach KornhauserisanAssociateMeasurementScientistattheCouncilforAidtoEducation.PriortojoiningCAEZachworkedasaSeniorResearchAssistantfortheCommunityCollegeResearchCenter.Hisresearchinterestsincludeexaminingqualityofeducationalprogrammingincollegesanduniversi-ties,understandingfactorsthataffectstudentmotivationonlow-stakesassessments,studyingstudents’useoftechnologyincollegeclassrooms,andinvestigatingfactorsthatexplainstudentperformanceincollegeoverandabovestudentability.ZachreceivedaMaster’sinAppliedDevelopmentalPsychologyfromFordhamUniversityin2011,andiscurrentlyfinishinghisdoctorateatFordham.

Patrick KyllonenisSeniorResearchDirectoroftheCenterforAcademicandWorkforceReadinessandSuccessatEducationalTestingService(ETS)inPrinceton,N.J.Centerscientistsconductinnovativeresearchon(a)highereducationassessment,(b)workforcereadiness;(c)internationallargescaleassess-ment(e.g.,ProgramforInternationalStudentAssessment;PISA);and(d)21stcenturyskillsassessment,suchascreativity,collaborativeproblemsolving,andsituationalinterviews.Dr.KyllonenreceivedhisB.A.fromSt.John’sUniversityandPh.D.fromStanfordUniversityandisauthorofGeneratingItemsforCognitiveTests(withS.Irvine,2001);LearningandIndividualDifferences(withP.L.Ackerman&R.D.Roberts,1999);ExtendingIntelligence:EnhancementandNewConstructs(withR.RobertsandL.Stankov,2008),andInnovativeAssessmentofCollaboration(withA.vonDavierandM.Zhu,forthcoming).HeisafellowoftheAmericanPsychologicalAssociationandtheAmericanEducationalResearchAssociation,recipientofTheTechnicalCooperationProgramAchievementAwardforthe“de-sign,development,andevaluationoftheTrait-SelfDescription(TSD)PersonalityInventory,”andwasacoauthoroftheNationalAcademyofSciences2012report,EducationforLifeandWork:DevelopingTransferableKnowledgeandSkillsinthe21stCentury.

Hollis LaiisAssistantProfessorofDentistryandtheDirectorofAssessmentforUndergraduateMedi-calEducationprogramattheUniversityofAlberta.Hisspecializationiseducationalandpsychologicaltesting,withanemphasisonassessmentdesignsinmedicaleducation,curriculummapping,educationaldatamining,anditemgeneration.

Syed F. LatifiisadoctoralcandidateinEducationalPsychologyattheUniversityofAlbertaandagraduateresearcherintheCenterforResearchinAppliedMeasurementandEvaluation(CRAME)atthesameuniversity.HeholdsaM.S.inComputerSciencefromLUMS,andanM.Ed.inMeasurement,EvaluationandCognitionfromUniversityofAlberta.Hehaspublishednumerouspapersandarticlesinreputedpeer-reviewedconferencesandjournals.Hiscurrentresearchinterestsincludeautomaticitemgeneration,automatedessayscoring,evolutionaryprogrammingandapplicationofswarmintelligenceinassessmentsystems.

Lei LiuisaresearchscientistfromtheCognitiveandLearningSciencesCenteratEducationalTest-ingService.Sheisleadingmultipleprojectsfocusingonthedesignofinnovativeandtechnology-richscienceassessmentsthatarecompetency-basedandNGSSaligned.Herresearchhasdrawnheavilyon

cxxxvii


cognitiveandsocioconstructivistlearningtheories.Herresearchinterestisontheroleoftechnologyinlearningandassessing.Shehasdevelopedsimulation-basedlearningenvironmentandassessments,learningprogressionbasedassessments,conversation-basedassessments,andcollaborativeproblemsolvingassessments.Shehaspublishednumerous journalarticlesandbookchapters in the fieldoflearningaboutcomplexsystems,computer-supportedcollaborativelearning,andlearningsciences.SheearnedherPhDfromRutgersUniversityandherdissertationwonthebestDoctoralResearchAwardbytheNationalAssociationofResearchinScienceTeaching.

William Lorié,Ph.D.,isDirectorofEducationResearchatQuestarAssessment,Inc.AgraduateofNotreDame(B.A,Philosophy,B.S.Physics)andStanford(M.S.Statistics,Ph.D.Education),Dr.LoriéhasworkedatCTB/McGraw-Hill,ETS,theWorldBank,andMetrica,anassessmentcompanyheco-founded.Dr.LoriéhasleadR&Dforlarge-scaletestingprogramsandhasadvisedonassessment-relatedcapacitybuildingforstatesandministriesofeducation.HisfocusatQuestarisonR&Dinnovation.

Mark Loveland,aSeniorResearchAssociateatWestEd’sScience,Technology,Engineering,&Mathematics(STEM)programisCo-PrincipalInvestigatoroftheIESfundedSimScientistsAssessmentSystemandSimScientistsModelProgressionsprojects,andtheNSFfundedSimScientistsAssessments:PhysicalScienceLinksproject.Dr.Lovelandhasover20yearsofexperienceworking in scientificresearchandSTEMeducation.PriortojoiningWestEd,Dr.LovelandcoordinatedresearcheffortstotransformmiddleschoolmathematicsandscienceteachingandlearninginSanFranciscopublicschools.Hehasconductedcancerresearchandtaughtbiology,chemistry,andenvironmentalscienceinformalandinformalenvironments.

Marie Ann MarinoistheAssociateDeanforAcademicAffairsandStrategicPartnershipsattheStonyBrookUniversitySchoolofNursing.Dr.MarinoreceivedaBachelorofSciencefromMolloyCollege,aMasterofScienceinNursingfromStonyBrookUniversityandaDoctorofEducationatTeachersCollege,ColumbiaUniversity.ShecompletedaPost-DoctoralFellowshipinFamilyViolenceResearchatTheChildren’sHospital/HarvardUniversitySchoolofMedicine.Dr.Marinohasmaintainedaclinicalpracticeasapediatricnursepractitioner/forensicmedicalexaminerconductingmedicalevalu-ationsofchildvictimsofabuseandviolence.Dr.Marinoisanactivememberofseveralprofessionalorganizations,includingtheInternationalAssociationofForensicNurses.Sheisakeydriveronseveralacademic-practicepartnershipsaimedatinnovatingcaremanagementandimprovingtransitionalcare,aswellasdevelopmentofcompetenceininterprofessionalcollaborativepractice.

Donna MatovinovicisVicePresidentofTestDevelopmentatACT.Donnahasover25yearsofexperienceprovidingleadershipandstrategicdirectionforthedesignanddevelopmentofassessmentproductsandservicesacrosstheKthroughcareercontinuum.Herfocusittoensuretightalignmentbetweencontentdevelopmentandpsychometricprinciplesaswellaswithproduct,program,andcus-tomerrequirements.

Mark MatzaganianisAssociateProfessorofEducationattheUniversityofLaVerne.HeteachesintheareasofAssessmentandResearch.Hisscholarlyinterestsincludetheroleoftechnologyineduca-tion,onlinelearning,andstudentmotivation.

cxxxviii


Shayne MieliscurrentlytheVicePresidentofSoftwareDevelopmentatLightSideLabs,aned-techstartupfocusedonprovidingautomatedwritingsupporttoK-12studentsaroundtheworldthroughtheuseoftheirautomatedscoringandfeedbacktechnology.HisroleatLightSideisprimarilyresearchanddevelopmentoftheautomatedassessmentandfeedbacksoftware.PriortohisexperienceatLightSideLabs,ShaynewastheSr.DirectorofAITechnologyatMeasurementIncorporated,whereheledagroupofresearchersanddevelopersinthecreationanddevelopmentofmoderndayPEG.

Edys QuellmalzisDirectorofTechnology-EnhancedAssessmentandLearningSystemsinWestEd’sMath,ScienceandTechnologyprogram.SheleadsSimScientistsprojectsfundedbyNSFandtheU.S.DepartmentofEducationrelatedtosimulation-basedsciencecurriculaandassessmentsforformativeandsummativeusesthatcanserveascomponentsofbalancedstatescienceassessmentsystems.Proj-ectsincludeCalipersII:UsingSimulationstoAssessComplexScienceLearning(NSF);Foundationsof21stCenturyScienceAssessment (NSF);ModelProgressions (IES);SimScientistsAssessments:PhysicalScienceLinks(NSF);SimScientistsCrosscuttingConcepts:Progressions inEarthSystems(NSF);SimScientistsAssessmentSystem(IES);SimScientists:InteractiveSimulation-basedLearningEnvironments(IES);MutilevelAssessmentsofScienceStandards(IES);TransformativeAssessmentsforScience(CarnegieCorporation);andIntegratingScienceSimulationsintoBalancedStateScienceAssessmentSystem(OESE).Sheco-directedthedevelopmentoftheframeworkandspecificationsforthe2014TechnologicalLiteracy(TEL)NationalAssessmentofEducationalProgressandservedontheSteeringCommitteeforthe2011NAEPWritingFramework.Shehasconsultedfornumerousstate,national,internationalassessmentprograms.ShewasAssociateDirectoroftheCenterforTechnologyandLearningatSRIInternationalandtheDirectorofAssessmentResearchandDesign.SheservedonthefacultyattheStanfordSchoolofEducationandasresearchfacultyintheUCLAGraduateSchoolofEducation.

Rikki Rimor,Ph.D.,SeniorLecturer,TechnologyinEducationGraduateProgram,DepartmentofEducationatKibbutzimCollegeofEducation,TechnologyandtheArts,Israel.Pastexperience:developerandinstructorofonlinegraduatecourse“FromSearchforInformationtoConstructionofKnowledgeinTechnology-BasedEnvironments”attheOpenUniversityofIsrael.ThiscoursewonExcellencePrizeintheOpenUniversityOnlineCoursesContest.ShewastheHeadoftheCenterforPedagogy&InnovativeTechnologies(PITEC)attheTeachersCollegeofTechnology,Tel-Aviv.ShewasalsoHeadoftheUnitforResearchandEvaluationattheDepartmentofEducationintheMunicipalityofJerusalem.Mostofherpublicationsdealwithwhyandhowtopromotecollaborativeknowledgeconstructioninonlinelearningenvironments,teachingandassessingproblemsolvinginanonlinecollaborativeenvironment,complexityofsocial interactionsincollaborativeonlinelearningenvironmentaswellasanalysisofcognitiveandmetacognitiveaspectsoflearningonFacebook.

Jenny Root,M.Ed.istheSnyderFellowatUNCCharlottewheresheisadoctoralstudentinSpe-cialEducation.SheisagraduateresearchassistantforTheSolutionsProject,anIESgrant,developingamathematicsproblem-solvingcurriculumforstudentswithseveredisabilities.Jennyhasoversevenyearsofexperienceworkingwithchildrenandadolescentswithdisabilitiesinschoolsettings.Jenny’sresearchfocusesongivingstudentswithautismandintellectualdisabilityaccesstothegeneralcur-

cxxxix


riculumthroughtechnology,systematicinstruction,andpositivebehaviorsupports.Futuredirectionsfor her research include systematic instruction to increase independence within academic tasks forstudentswithdevelopmentaldisabilitiesandusingtechnologytoovercomebarrierstocriticalthinkingandproblemsolvingskills.

Abraham Rotem,Ph.DinPhysicsandElectricalEngineering,KSUUSAandBen-GurionUni-versity,Israel(1983);FounderSENTOPLtd(1990);EstablishingTeachersDevelopmentcenterwithBrancoWeissInstitute,Israel(1994).EstablishmentofthefirstsocialeducationalnetworkinIsrael,toschoolsandstudent(1998).WrotetheBook“Schoolturnsonline”withDrY.Peled(2008).Lecturerinprofessionaldevelopmentcoursesforschooladministratorsandteachers.EstablishingwithDrE.Avni“TowardDigitalEthicsInitiative”,Israel,containsoriginalarticlesrelatedtotechnology,pedagogicalandethical aspects in learningand teaching;Academicadvisor ICTprogram in Israel (MinistryofEducation)forthelastdecade.

Jean-François RouetisaSeniorresearchscientistwiththeFrenchCentreNationaldelaRechercheScientifique(NationalCenterforScientificResearch).Hewastrainedasadevelopmentalandlanguagepsychologistandhehaspublishedextensivelyonthecognitiveunderpinningsofreadingliteracyandskilledusesofinformationtechnology(e.g.,“Reading:Fromwordstomultipledocuments”,coeditedwithBrittandGoldman,Routledge,2013).Jean-FrançoisRouethasbeeninvolvedsince2006asanexpertintheOECD’sPISAandPIAACsurveysofteenageandadultliteracy,respectively.HehasservedasthedirectoroftheCenterforResearchonCognitionandLearningattheUniversityofPoitiers,France,from2004to2011.AfterhisPhD(1991),Jean-FrançoisRouetspentseveralyearsasapost-doctoralfellowattheLearningResearchandDevelopmentCenter(UniversityofPittsburgh,USA).Hehasde-velopednumerouscollaborationswithscholarsfromvariouscountriesincludingArgentina,Germany,Japan,Norway,SpainandtheUSA.HeisontheboardofseveralmajoracademicjournalsandaformerassociateeditorofLearningandInstruction.

Gavriel Salomon(PhDatStanford,1968)isprofessorofeducationalpsychologyattheHaifaUniver-sityinIsraelwhereheservedasdeanoftheFacultyofEducation.AseditorofEducationalPsychologist,hehaspublishedfourbooksandabouta150research,theory,andmethodologyjournalarticles.HeistherecipientoftheIsraelAwardforscientificachievementsandholdsanhonorarydoctoratefromtheLeuvenUniversityinBelgium.Hisfieldsaretheuseandabuseofcomputersineducationandresearchonpeaceeducationinregionsofconflict.

Alicia Saunders,Ph.D.,istheProjectCoordinatorandResearchAssociatefortheSolutionsProject,anIESgrant,developingamathematicalwordproblemsolvingcurriculumforstudentswithseveredisabilitiesattheUniversityofNorthCarolinaatCharlotte.Shehasconductedresearchintheareasofgeneralcurriculumaccess,specificallyinscience,mathematics,andEnglishlanguagearts,aswellasintechnology,includingvideomodelingandcomputer-assistedinstruction.Additionally,shehaspub-lishedmultiplepeerreviewedjournalarticlesandbookchapters,developedandconductedprofessionaldevelopment webinars on aligning instruction to alternate achievement standards and the CommonCoreStateStandards,andisaco-authorofEarlyNumeracy,amathematicscurriculumforstudentswithseveredisabilities.Dr.Saundersreceivedherbachelor’sdegreeinpsychologyfromtheUniversity

cxl


ofNorthCarolinaatChapelHill,andbothhermaster’sanddoctoraldegreesinspecialeducationfromtheUniversityofNorthCarolinaatCharlotte.ShetaughtstudentswithseveredisabilitiesandautisminNorthCarolinapriortoobtainingherdoctorate.ShealsoworkedasaresearchassociateonanIESfundedgrantinvestigatingmathematicsandscienceinstructionforstudentswithseveredisabilities,andasaresearchassociateforoneofthetwonationalconsortiacalledtodevelopanalternateassessmentandcurricularmaterialsbasedonalternateachievementstandardsalignedtotheCommonCoreStateStandardsforstudentswithsignificantcognitivedisabilities.

Matt Silberglitt,aSeniorResearchAssociateatWestEd’sScience,Technology,Engineering,&Mathematics(STEM)programisCo-PrincipalInvestigatoroftheSimScientistsAssessments:PhysicalScienceLinksandSimScientistsCrosscuttingConcepts:ProgressionsinEarthScienceprojectsfundedbyNSFandtheSimScientistsAssessmentSystemsprojectfundedbyIES.Asaformerscienceteacherandexperiencedassessmentdeveloper,Mr.Silberglittprovidescontentexpertiseinphysicalscienceandexpertise in standards for educational assessments.Prior tohis currentposition,Mr.SilberglittmanageddevelopmentofscienceassessmentsforWestEdinseveralstates.PriortojoiningWestEd,Mr.SilberglittoversawdevelopmentofscienceassessmentsattheMinnesotaDepartmentofEducationanddevelopedassessmentsatDataRecognitionCorporation.Mr.Silberglittstartedhiscareerasahighschoolscienceteacher.

Jeffrey T. SteedleisaResearchScientistatPearson,whereheprovidespsychometricsupportforstatetestingprogramsandconductsresearchwiththeCenterforNextGenerationLearningandAssess-ment.Hisresearchinterestsincludestudentmotivation,comparativejudgment,itemdifficultymodeling,andscienceassessment.HeearnedaPh.D.inEducationalPsychologyandaM.S.inStatisticsfromStanfordUniversityin2008.

A. Jackson Stennerisaco-founderofMetaMetricsandisitsChiefExecutiveOfficerandChairmanoftheBoard.Withcolleagues,hedevelopedTheLexileFrameworkforReading,TheQuantileFrame-workforMathematics,andtheLexileFrameworkforWriting.Dr.Stennerhaspublishedmorethan60papers,monographsandbooks,primarilyonstatisticalandmeasurementmethodology.Currently,heisaResearchProfessorintheDepartmentofAppliedDevelopmentalandSpecialEducationProgramintheSchoolofEducationattheUniversityofNorthCarolina-ChapelHill.Dr.StennerreceivedhisPh.D.ineducationalpsychology,withanemphasisinmeasurement,researchdesignandevaluationmethodology,fromDukeUniversity,anddualundergraduatedegreesinpsychologyandeducationfromtheUniversityofMissouri–SaintLouis.HehastaughtgraduateseminarsatDukeUniversityandtheUniversityofNorthCarolinaatChapelHillandguestlecturedatseveraldozenuniversities.

Carl W. Swartzconductsresearchonmodelsandtechnologiesforblendinglearningandassessmentanduseoftechnologyasacomponentofeducationalprogramstoenhancepersonalizedlearningforstudentswithlearning,attention,andlanguagedifferences.HeiscurrentlyaResearchProfessorintheDepartmentofAppliedDevelopmentalandSpecialEducationintheSchoolofEducationattheUniversityofNorthCarolina-ChapelHillwherehewasaResearchScientistintheSchoolofMedicineandClini-calAssistantProfessorintheSchoolofEducationatTheUniversityofNorthCarolinaatChapelHill.Currently,HereceivedhisundergraduatedegreeineducationfromIndianaUniversityandhismaster’s

cxli


anddoctoratedegreesineducationfromTheUniversityofNorthCarolinaatChapelHill.Dr.SwartztaughtearlyadolescentswithsevereemotionaldisturbancesandbehaviordisordersatamiddleschoolinGreenville,SouthCarolina(1983-1986).

Noel Tagoe is theExecutiveDirectorofEducationatCIMA,wherehewaspreviouslyHeadofResearchandDevelopment.PriortojoiningCIMANoelhadheldaccounting,strategyandfinancialconsultingpositionswithBPandKMPGinGhana.HehasalsotaughtatvariousuniversitiesintheUKincludingManchesterUniversityandOxfordUniversity.Noel’scurrentinterestsareindesigningprofes-sionalaccountingcurriculathataddressbusinessneedsandensuringthattheyareassessedrigorouslyandrobustlyusingtechnologywherepossible.

Fatima E. Terrazas-ArellanesisPIofProjectESCOLAR,anOSEP-fundedprojecttocreateonlinemodulesforteachingsciencetomiddleschoolstudentswithlearningdisabilities.Dr.Terrazas-ArellaneswasateacherfromMexico,coordinatingresearchprojectsinareasofSpanishliteracyinstruction,secondlanguageacquisition,andeTextsupports.Dr.Terrazas-Arellanes’areasofexpertiseincludedesigningandconductingresearchtolearnhowtechnologyaddressestheeducationalneedsofchildrenandyouths,particularlystudentswithlearningdisabilitiesandEnglishLanguageLearners.Dr.Terrazas-ArellanesreceivedherBachelorofArtsinPsychologyfromtheUniversidadAutonomadeSinaloain2000,aMaster’sofScienceinSpecialEducationfromtheUniversityofOregonin2008,aswellasaDoctorofPhilosophyfromtheUniversityofOregon’sSchoolofPsychologyin2009.

Suzanne Tsacoumishasbuiltherprofessionalcareerconductingandmanagingprojectsassociatedwiththeresearch,development,andimplementationofpersonnelassessmentsandhumancapitalsys-tems,ofteninlitigiousenvironments.Herexpertiserevolvesaroundjobanalysis,selection,promotion,leadershipassessment,andperformanceevaluation.Currently,sheisspearheadinginnovativeworkinthedevelopmentofvalid,onlinesimulations,suchasvirtualroleplaysandinteractivein-baskets,foruseinbothpromotionsystemsandself-assessmentprocesses.Sheoftenconsultsonarangeofpolicyandimplementationissues.Inadditiontohertechnicalwork,SuzanneisacorporateofficerandHumRRO’svicepresidentofbusinessdevelopment.Inthisrole,sheisresponsibleforprovidingleadershipanddi-rectiontothebusinessdevelopmentandmarketingteam.SuzannehasbeenanactivevolunteerfortheSocietyofIndustrial-OrganizationalPsychology(SIOP)andsheisafellowofSIOPandtheAmericanPsychologicalAssociation(APA).SuzannehasservedasanadjunctprofessoratbothTheGeorgeWash-ingtonUniversityandGeorgeMasonUniversity.SheearnedherPh.D.fromtheUniversityofGeorgia,specializinginIndustrial-OrganizationalPsychologyandherB.A.fromBucknellUniversity.

David Vaughnisasoftwareengineerwithovertenyearsofexperienceinthefieldofartificialintel-ligence(AI)andmachinelearning.AsaSeniorSoftwareDeveloperhedesigns,develops,andenhancesautomatedscoringtechnologies.Mr.VaughnwastheprimaryarchitectofMI’swinningsubmissionstotheAutomatedStudentAssessmentPrize(ASAP)contest,anautomatedscoringcompetitionsponsoredbytheHewlettFoundation.HesubsequentlyenhancedtheAIscoringsystemusedintheASAPcom-petitiontoimproveitsaccuracy,speed,efficiency,andscalabilityforuseontheSmarterBalancedPilotTestautomatedscoringproject.Previously,Mr.Vaughndevelopedandimplementedmachinelearningalgorithmsinavarietyofcontextsincludingemailclassification,genomeresearch,andhuman/computervoicedialoguesystems.HereceivedhisB.S.inComputerSciencefromHarvardUniversity.

cxlii


Lars VavikisaprofessorininteractiveeducationaltechnologyatStord/HaugesundUniversityCollege.Hehasbeenworkingwithdesignofdigitalmediasince1980andhasbeengivenEuropeanAcademicSoftwareAwardfordevelopingofmodelingandsimulationtools.Hisworkthelast15yearshasbeendedicatedtodevelopingamasterprograminICTandlearningandtheleadershipofseveralresearchprogramas“EducationonCurriculumandTechnology”(2008-2012)and“Learninginthe21stcentury”(2012-2016)supportedbyTheResearchCouncilofNorway.

Alina von DavierisaSeniorResearchDirectorandleaderoftheCenterforAdvancedPsychometricsatETS.ShealsoisanAdjunctProfessoratFordhamUniversity.AtETS,vonDavierisresponsiblefordevelopingateamofexpertsandapsychometricresearchagendainsupportofnextgenerationofas-sessments.Computationalpsychometrics,whichincludemachinelearninganddataminingtechniques,Bayesianinferencemethods,stochasticprocessesandpsychometricmodelsarethemainsetoftoolsemployedinhercurrentwork.Shealsoworkswithpsychometricmodelsappliedtoeducationaltesting:testscoreequatingmethods,itemresponsetheorymodels,adaptivetesting.Shepublishedseveralbooks,authoredoredited;shealsopublishednumerouspapersinpeerreviewedjournals.

Matthias von DavierisaSeniorResearchDirectorinETS’sResearch&DevelopmentDivision,managingtheinternationallarge-scaleassessmentmethodologygroupintheETScenterforglobalas-sessment.AtETS,Dr.vonDaviermanagesagroupofresearchersconcernedwithmethodologicalques-tionsarisinginlarge-scaleinternationalcomparativestudiesineducation.Heiscurrentlyeditor-in-chiefoftheBritishJournalofMathematicalandStatisticalPsychologyandco-editorofthejournal“LargeScaleAssessmentsinEducation”,whichisjointlypublishedwiththeInternationalAssociationfortheEvaluationofEducationalAchievement(IEA)andETSthroughtheIEA-ETSResearchInstitute(IERI).HiscurrentworkatETSinvolvesthepsychometricmethodologiesusedinanalyzingcognitiveskillsdataandbackgrounddatafromlarge-scaleeducationalsurveys,suchastheOrganizationforEconomicCo-operationandDevelopment’supcomingPIAACandtheongoingPISA,aswellasIEA’sTIMSSandPIRLS.HisworkatETSalsoinvolvesthedevelopmentofsoftwareformultidimensionalmodelsforitemresponsedata,andtheimprovementofmodelsandestimationmethodsfortheanalysisofdatafromlargescaleeducationalsurveyassessments.PriortojoiningETS,Dr.vonDavierledaresearchgrouponcomputerassistedsciencelearning,wasco-directorofthe“Computerasatoolforlearning”sectionattheInstituteforScienceEducation(IPN)inKiel,Germany,andwasanassociatememberofthePsychometrics&MethodologyDepartmentofIPN.

Zsofia VorosiscurrentlyresearchassociateintheCenterforResearchonCognitionandLearning,CNRSandUniversityofPoitiers,France.Herresearchinvestigatescognitivedimensionsthatmayun-derlieelectronicreading,learningandproblemsolving.Previously,sheworkedasaresearchconsultantfortheEducationalTestingService.SheexploredProgrammefortheInternationalAssessmentofAdultCompetencies(PIAAC)processingdatatoidentifycognitive/metacognitiveskillsandsolutionstrategieslinkedtosuccessinvariouselectronicproblemsolvingtasks.SheearnedherPh.DinpsychologyfromtheUniversityofPoitiers,France(2009).

Emily Walden,M.A.,isaResearchAssistantattheCenterforAdvancedTechnologyinEducationattheUniversityofOregoninEugene,Oregon.Shecompletedabachelor’sdegreeinEnglishandpsy-chologyattheUniversityofOregonin2010andamaster’sdegreeindevelopmentalpsychologyatthe

cxliii


UniversityofOregonin2011.Herresearchwithfederallyfundedprojects(ESTRELLAS,COPELLS,SSOAR,andESCOLAR)primarilyinvolvesimprovingeducationoutcomes,especiallyinscience,formiddleschoolstudentswithlearningdisabilitiesandEnglishlearnersthroughtheuseoftechnology.

Nancy T. WalkeriscurrentlyaProfessorofEducationattheUniversityofLaverne.Forthepastfifteenyears,shehasoverseentheteachereducationliteracyfacultyandworkedwithadjunctfacultyinonlinelearning.Herresearchhasfocusedoncontentarealiteracy,disciplinaryliteracy,visualliteracy,andonlinelearninginhighereducation.Previously,shetaughtLanguageArtsatthemiddleschoollevelforeightyearsandmentorednewteachersineffectiveliteracyinstruction.Dr.Walkerisapublishedauthorontheuseofmultipletextsintheclassroomandisafrequentspeakeratnationalandinternationalconferences.

James WelshistheDirectoroftheFloridaCenterforInstructionalTechnologyandholdsaPh.D.inLiteracyStudiesfromtheUniversityofSouthFloridaCollegeofEducation.Aformerelementaryschoolteacher,JamesistheprogramcoordinatorfortechnologyintegrationsupportintheUSFCollegeofEducationandtheprojectleaderfortheTechnologyIntegrationMatrixK-12evaluationtools.JamesconductsresearchwiththeContemporaryLiteraciesCollaborativeatUSF.Hisresearchinterestsincludeevaluationofeducationaltechnology,criticalmedialiteracy,studentcreationofmultimediatexts,andtheroleofgenreinstudentcomposition.

Joshua WilsonisanAssistantProfessorintheSchoolofEducationattheUniversityofDelaware.HeearnedhisPh.D.inSpecialEducationfromtheUniversityofConnecticut in2014.Hisresearchfocusesonmethodsofassessingandinstructingstrugglingwriters,andontheapplicationofautomatedessayevaluation(AEE)technologyinResponse toIntervention(RtI)contexts.Hecurrently teachescoursesonelementaryspecialeducationmethods.PriortoearninghisPh.D.,Dr.Wilsonwasaspecialeducationteacherforsixyears.

Roy WinkelmanhasspenttwodecadesinK-12educationandtwodecadesinhighereducation.FormerlydirectoroftheFloridaCenterforInstructionalTechnology,henowworksonspecialprojectsfortheCenterandcreatesdigitalresourcesforK-12studentsandteachers.

Steven WiseisaSeniorResearchFellowatNorthwestEvaluationAssociation.Dr.Wisehaspub-lishedextensivelyduringthepastthreedecadesinappliedmeasurement,withparticularemphasesincomputer-basedtestingandthepsychologyoftesttaking.Inaddition,hesitsontheeditorialboardofseveralacademicjournalsandprovidespsychometricconsultationtoavarietyoforganizations.Inrecentyears,Dr.Wise’sresearchhasfocusedprimarilyonmethodsforeffectivelydealingwiththemeasure-mentproblemsposedbylowexamineeeffortonachievementtests.

Raffaela WolfcompletedherPhDinResearchMethodologyfromtheUniversityofPittsburghinDecember2013.Dr.RaffaelaWolfjoinedCAEinJanuary2014asaMeasurementScientist.Inthisrole,Dr.RaffaelaWolfisinvolvedinresearchprojectsthatpertaintovalidityandreliabilitystudiesoftheCollegiateLearningAssessment-Plus.Otherresponsibilitiesincluderunningpsychometricanalysesfornationalandinternationalclients.Researchinterestsincludeequating,scaling,andlinking;itemresponsetheory;structuralequationmodeling;and,cross-nationalassessmentsinhighereducation.

cxliv


Doris ZahnerjoinedCAEinMarch2011astheDirectorofTestDevelopmentandaMeasurementScientist.HerresponsibilitiesincludeoverseeingallitemdevelopmentforCAEaswellasconductingresearchstudiespertainingtotheCLA+andotherCAEassessmentinstruments.Priortoherpositionwith theCAE,shewasaProgramDirectorwithaprofessional licensureandcertification firmandmanagedavarietyofclients,includingamedicalcertificationboard,aninformationsystemsauditingcertificationprogram,variousnursingspecialties,andplasticsengineers.Shehasexperiencemanaginglarge-scaleitemdevelopmentprojectsandisknowledgeableaboutallaspectsofassessmentdevelopmentincludingtaskanalyses,testspecifications,itemwriting,examconstruction,scorereportingandequating,andstandardsetting.SheholdsaPh.D.inCognitivePsychologyandaM.S.inAppliedStatisticsfromTeachersCollege,ColumbiaUniversity.Herpost-doctoralworkattheStevensInstituteofTechnologywasontheuseofdiagramsininformationsystemsdesignandherownresearchinterestspertaintotheuseofdiagramsinprobabilityandmathematicsproblemsolving.InadditiontoherresponsibilitiesatCAE,Dr.ZahnerisanadjunctassistantprofessoratBarnardCollegewheresheteachesacourseonstatisticstoundergraduatestudentsinthesocialsciences.

Diego Zapata-Rivera,PhD,isaseniorresearchscientistintheCognitiveScienceResearchGroupatEducationalTestingService.Hisresearchfocusesoninnovationsinscorereportingandtechnology-enhancedassessment,includingworkonassessment-basedlearningenvironmentsandgame-basedas-sessments.HisresearchinterestsalsoincludeEvidence-CenteredDesign,Bayesianstudentmodeling,openstudentmodels,conversation-basedassessments,virtualcommunities,authoringtoolsandprogramevaluation.Hehasbeenacommitteememberandorganizerofinternationalconferencesandworkshopsinhisresearchareas.HeisamemberoftheBoardofSpecialReviewersoftheUserModelingandUser-AdaptedInteractionjournalandanAssociateEditoroftheIEEETransactionsonLearningTechnolo-giesJournal.Mostrecently,Dr.Zapata-RiverahasbeeninvitedtocontributehisexpertisetoprojectssponsoredbytheNationalResearchCouncil,theNationalScienceFoundationandNASA.

cxlv

Table of Contents - Harvard University · 2015-10-27 ·...

Documents

Transcript of Table of Contents - Harvard University · 2015-10-27 ·...