The Marzano Teacher Evaluation - slcs.us
Transcript of The Marzano Teacher Evaluation - slcs.us
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
1
The Marzano Teacher Evaluation Model
Learning Sciences International 1400 Centrepark Boulevard, Suite 1000 West Palm Beach, FL 33401 717.845.6300
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
2
Table of Contents
1.THERESEARCHBASEFORTHEMARZANOTEACHEREVALUATIONMODEL 3
EXPERIMENTAL/CONTROLSTUDIES 4CORRELATIONALSTUDIES 5TECHNOLOGYSTUDIES 5SUMMARYOFRESEARCHBASE 6REFERENCES 6
2.ABOUTROBERTMARZANOANDLEARNINGSCIENCESINTERNATIONAL 7
3.EVIDENCEOFRELIABILITY,VALIDITY,ANDEFFICACYOFTHEMARZANOTEACHEREVALUATIONMODEL 9
RECENTRESEARCHVALIDATINGTHEMARZANOTEACHEREVALUATIONMODEL 9MARZANOOBSERVATIONCORRELATIONSWITHFLORIDAVAM 102013‐14PINELLASPILOTFINDINGS 12
4.OVERVIEWOFTHEMARZANOTEACHEREVALUATIONMODELANDRUBRICS 15
THERESEARCH‐BASEDMODEL:FOURDOMAINSDIRECTLYTIEDTOSTUDENTACHIEVEMENT 15THEMARZANOTEACHEREVALUATIONMODELLEARNINGMAP 17
5.PROCESSFORCLASSROOMOBSERVATIONS 17
SCALESANDEVIDENCESFORTHEMARZANOTEACHEREVALUATIONMODEL. 21
6.TRAININGPLANFOREVALUATORSANDOBSERVERS 21
7.THEMARZANOCENTERNONCLASSROOMINSTRUCTIONALPERSONNELEVALUATIONMODEL 21
8.APPENDIX 21
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
3
1. The Research Base for the Marzano Teacher Evaluation Model
(For an in-depth examination of the research base, please see: Marzano, Toth, Schooling, “Examining the Role of Teacher Evaluation in Student Achievement: Contemporary Research Base for the Marzano Causal Teacher Evaluation Model,” 2012. http://www.marzanocenter.com/teacher-evaluation/mc-whitepaper/ and Basilio and Toth, “The Research Base for the Marzano Teacher Evaluation Model and Correlations to State VAM,” 2016. http://www.marzanocenter.com/MCTeacherEval_VAM%2020160328.pdf) TheMarzanoEvaluationModelisbasedonanumberofprevious,relatedworksthat
include:WhatWorksinSchools(Marzano,2003),ClassroomInstructionthatWorks
(Marzano,Pickering,&Pollock,2001),ClassroomManagementthatWorks(Marzano,
Pickering,&Marzano,2003),ClassroomAssessmentandGradingthatWork
(Marzano,2006),TheArtandScienceofTeaching(Marzano,2007),Effective
Supervision:SupportingtheArtandScienceofTeaching(Marzano,Frontier,&
Livingston,2011).Eachoftheseworkswasgeneratedfromasynthesisofthe
researchandtheory.Thusthemodelcanbeconsideredanaggregationofthe
researchonthoseelementsthathavetraditionallybeenshowntocorrelatewith
studentacademicachievement.Themodelincludesfourdomains:
Domain1:ClassroomStrategiesandBehaviors
Domain2:PreparingandPlanning
Domain3:ReflectingonTeaching
Domain4:CollegialityandProfessionalism
Thefourdomainsinclude60elements:41inDomain1,8elementsinDomain2,5
elementsinDomain3and6elementsinDomain4.Foradetaileddiscussionofthese
elementsseeEffectiveSupervision:SupportingtheArtandScienceofTeaching
(Marzano,Frontier,&Livingston,2011).
Eachoftheworks(citedabove)fromwhichthemodelwasdevelopedreport
substantialresearchontheelementstheyaddress.Forexample,TheArtandScience
ofTeachingincludesover25tablesreportingtheresearchonthevariouselements
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
4
ofDomain1.Thesetablesreportthefindingsfrommeta‐analyticstudiesandthe
averageeffectsizescomputedinthesestudies.Inall,over5,000studies(i.e.,effect
sizes)arecoveredinthetablesrepresentingresearchoverthelastfivedecades.The
samecanbesaidfortheothertitleslistedabove.Thus,onecansaythatthemodel
wasinitiallybasedonthousandsofstudiesthatspanmultipledecadesand
thesestudieswerechronicledandcataloguedinbooksthathavebeenwidely
disseminatedintheUnitedStates.Specifically,over2,000,000copiesofthebooks
citedabovehavebeenpurchasedanddisseminatedtoK‐12educatorsacrossthe
UnitedStates.
Experimental/Control Studies
Perhapsoneofthemoreuniqueaspectsoftheresearchonthismodelisthatithasa
growingnumberofexperimental/controlstudiesthathavebeenconductedby
practicingteachersontheeffectivesofspecificstrategiesintheirclassrooms.Thisis
unusualinthesensethatthesestudiesaredesignedtoestablishadirectcausallink
betweenelementsofthemodelandstudentachievement.Studiesthatuse
correlationanalysistechniques(seenextsection)canestablishalinkbetween
elementsofamodelandstudentachievement;however,causalitycannotbeeasily
inferred.Otherevaluationmodelscurrentlyusedthroughoutthecountryonlyhave
correlationaldataregardingtherelationshipbetweentheirelementsandstudent
achievement.
Todateover300experimental/controlstudieshavebeenconducted.Thosestudies
involvedover14,000students,300teachers,across38schoolsin14districts.The
averageeffectsizeforstrategiesaddressedinthestudieswas.42withsomestudies
reportingeffectsizesof2.00andhigher.Anaverageeffectsizeof.42isassociated
witha16percentilepointgaininstudentachievement.Stateddifferently:onthe
average,whenteachersusetheclassroomstrategiesandbehaviorsintheMarzano
EvaluationModel,theirtypicalstudentachievementincreasedby16percentile
points.However,greatgains(i.e.,thoseassociatedwithaneffectsizeof2.00)canbe
realizedifspecificstrategiesareuseinspecificways.
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
5
Correlational Studies
Asmentionedabove,correlationalstudiesarethemostcommonapproachto
examiningthevalidityofanevaluationmodel.Suchstudieshavebeen,andcontinue
tobeconducted,onvariouselementsoftheMarzanoEvaluationModel.For
example,onesuchstudywasrecentlyconductedinthestateofOklahomaasapart
oftheirexaminationofelementsthatarerelatedtostudentachievementinK‐12
schools(seeWhatWorksinOklahomaSchools:PhaseIReportandWhatWorksin
OklahomaSchool:PhaseIIReport,byMarzanoResearchLaboratory,2010and2011
respectively).Thosestudiesinvolved59schools,117teachersandover13,000K‐12
students.Collectively,thosereportsindicatepositiverelationshipswithvarious
elementsoftheMarzanoEvaluationModelacrossthedomains.Specificemphasis
wasplacedonDomain1particularlyinthePhaseIIreport.Usingstatemathematics
andreadingtestdata,96%ofthe82correlations(i.e.,41correlationsfor
mathematicsand41forreading)werefoundtobepositivewithsomeashighas.40
andgreater.A.40correlationtranslatestoaneffectsize(i.e.,standardizedmean
difference)of.87whichisassociatedwitha31percentilepointgaininstudent
achievement.Thesestudiesalsoaggregateddataacrosstheninedesignquestionsin
Domain1.Allcorrelationswerepositiveforthisaggregateddata.Sevenofthose
correlationsrangedfrom.33to.40.Thesecorrelationstranslateintoeffectsizesof
.70andhigher.Highcorrelationssuchasthesewerealsoreportedforthetotal
numberofDomain1strategiesteachersusedinaschool.Specificallythenumberof
Domain1strategiesteachersusedinschoolhada.35correlationwithreaching
proficiencyanda.26correlationwithmathematicsproficiency.
Technology Studies
Anotheruniqueaspectoftheresearchconductedonthemodelisthatitseffects
havebeenexaminedinthecontextoftechnology.Forexample,atwoyearstudywas
conductedtodetermine(inpart)therelationshipbetweenselectedelementsfrom
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
6
Domain1andtheeffectivenessofinteractivewhiteboardsinenhancingstudent
achievement(seeFinalReport:ASecondYearEvaluationStudyofPromethean
ActivClassroombyHaysteadandMarzano,2010).Inall,131experimental/control
studieswereconductedacrossthespectrumofgradelevels.Selectedelementsof
Domain1werecorrelatedwiththeeffectsizesforuseoftheinteractivewhite
boards.AllcorrelationsforDomain1elementswerepositivewithsomeashighas
.70.Thisimpliesthattheeffectivenessoftheinteractivewhiteboardsasusedin
these131studieswasgreatlyenhancedbytheuseofDomain1strategies.
Summary of Research Base
Insummary,theMarzanoEvaluationModelwasdesignedusingliterallythousands
ofstudiesconductedoverthepastfiveormoredecadesandpublishedinbooksthat
havebeenwidelyusedbyK‐12educators.Inaddition,experimental/controlstudies
havebeenconductedthatestablishamoredirectcausallinkageswithenhanced
studentachievementthatcanbemadewithothertypesofdataanalysis.Correlation
studies(themoretypicalapproachtoexaminingtheviabilityofamodel)havealso
beenconductedindicatingpositivecorrelationsbetweentheelementsofthemodel
andstudentmathematicsandreadingachievement.Finally,themodelhasbeen
studiedastoitseffectsontheuseoftechnology(i.e.,interactivewhiteboards)and
foundittobehighlycorrelatedwiththeeffectivenessofthattechnology.
References
Haystead,M.W.&Marzano,R.J.(2010)FinalReport:ASecondYearEvaluation
StudyofPrometheanActivClassroom.Englewood,CO:MarzanoResearch
Laboratory(marzanoresearch.com)
Haystead,M.W.&Marzano,R.J.(2010).Meta‐AnalyticSynthesisofStudies
ConductedatMarzanoResearchLaboratoryoninstructionalStrategies.Englewood,
CO:MarzanoResearchLaboratory(marzanoresearch.com)
Marzano,R.J.(2003).Whatworksinschools.Alexandria,VA:ASCD
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
7
Marzano,R.J.(2006).Classroomassessmentandgradingthatwork.Alexandria,VA:
ASCD.
Marzano,R.J.(2007).Theartandscienceofteaching.Alexandria,VA:ASCD
Marzano,R.J.,Frontier,T.,&Livingston,D.(2011).Effectivesupervision:Supporting
theartandscienceofteaching.AlexandriaVA:ASCD
Marzano,R.J.,Pickering,D.J.,&Pollock,J.E.(2001).Classroominstructionthat
works.Alexandria,VA:ASCD.
Marzano,R.J.,Marzano,J.S.,&Pickering,D.J.(2003).Classroommanagementthat
works.Alexandria,VA:ASCD
MarzanoResearchLaboratory.(2010)WhatWorksinOklahomaSchools:PhaseI
Report.Englewood,CO:MarzanoResearchLaboratory(marzanoresearch.com)
MarzanoResearchLaboratory.(2011)WhatWorksinOklahomaSchools:PhaseII
Report.Englewood,CO:MarzanoResearchLaboratory(marzanoresearch.com)
2. About Robert Marzano and Learning Sciences International
RobertJ.Marzano,PhD,isanationallyrecognizedresearcherineducation,
speaker,trainer,andauthorofmorethan30booksand150articlesontopicssuch
asinstruction,assessment,writingandimplementingstandards,cognition,effective
leadership,andschoolintervention.HisbooksincludeDistrictLeadershipThat
Works,SchoolLeadershipthatWorks,MakingStandardsUsefulintheClassroom,The
ArtandScienceofTeaching,andEffectiveSupervision.
Hispracticaltranslationsofthemostcurrentresearchandtheoryintoclassroom
strategiesareinternationallyknownandwidelypracticedbybothteachersand
administrators.Hereceivedabachelor’sdegreefromIonaCollegeinNewYork,a
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
8
master’sdegreefromSeattleUniversity,andadoctoratefromtheUniversityof
Washington.HeisalsoExecutiveDirectoroftheLearningSciencesMarzanoCenter
locatedinWestPalmBeach,Florida,andofMarzanoResearchinColorado.
Dr.Marzanobelievesthatgreatteachersmakegreatstudents:HisMarzanoTeacher
EvaluationModelhasbeenadoptedbyschooldistrictsinall50statesbecauseit
doesn’tjustmeasureteacherability,ithelpsteachersgetbetter,improvingtheir
instructionovertime.Dr.MarzanohaspartneredwithLearningSciences
InternationaltodevelopandimplementtheMarzanoTeacherEvaluationModel,the
SchoolLeaderandDistrictLeaderEvaluationModels,andtheNon‐Classroom
InstructionalPersonnelEvaluationmodel,fourcomplimentaryevaluationsystems
thatmaybeusedwiththeiObservationtechnologyplatform.
Foundedin2002,LearningSciencesInternationalpartnerswithschoolsand
districtstodevelopcustomsolutionsforschoolimprovementandprofessional
development.WithRobertMarzano,LearningSciencesco‐developedtheMarzano
EvaluationModelsandwasselectedasthestatewidetechnicalassistanceprovider
forteacherevaluationimplementationthroughoutthestateofFlorida.Learning
ScienceswasselectedbytheMichiganDepartmentofEducation’sSchoolReform
OfficetoprovidemonitoringandtechnicalassistancetoPrioritySchools.Learning
Sciencesoffersinnovativetechnology,dataanalysis,research,consultation,andthe
toolsandtrainingtohelpschoolsmeettheirchallengesandreachtheirgreatest
potentialintoday’shigh‐stakeseducationalenvironment.Forfurtherinformation,
visitwww.LearningSciences.com.
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
9
3. Evidence of reliability, validity, and efficacy of the Marzano
Teacher Evaluation Model
Recent Research Validating the Marzano Teacher Evaluation Model
TworecentstudiesaddresswhethertheMarzanoTeacherEvaluationModelisa
validatedframework.Thefirst,(BasileoandToth,InProgress1),investigates
whethertheobservationdatafromtheMarzanoTeacherEvaluationModel
correlateswithteachervalue‐addedmeasures(VAMs)acrossthestateofFlorida.
Thesecondstudy,whichwasfeaturedinaUSDepartmentofEducationreportin
2015,directlytestedwhetheraprofessionaldevelopmentprogrambasedonthe
MarzanoTeacherEvaluationModelincreasedstudentachievementinapilotin
PinellasCountyPublicSchools,Florida(seeBasileo,Toth,&Kennedy,2015).Both
studiessupportandvalidatetheMarzanoTeacherEvaluationModelinFlorida.
Whenevaluatingthevalidityofobservationprotocols,studiestypicallyassessthe
correlationsbetweenteacherobservationscoresandtheirvalue‐addedscores.
Smalltomoderatecorrelationspermitresearcherstoclaimthattheframeworkis
validated(Kane,Taylor,Tyler,&Wooten,2010).(SeeEndnoteiforanoverviewof
currentresearchonthemagnitudeandrangeofcorrelationcoecientsbetween
observationdataandVAMestimates).
AcorrelationbetweentwovariablesdoesnotnecessarilymeanthatXcausesY;it
merelyprovidesevidencethatthereisarelationshipbetweenthetwo.Thus,
validitystudiesthatinvestigatewhethertheframeworkincreasesstudent
achievementshouldincludeeitherexperimentalorquasi‐experimentaldesigns,to
demonstratethattheframeworkincreasesstudentachievement.
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
10
Marzano Observation Correlations With Florida VAM
BasileoandToth1investigatedthemagnitudeofcorrelationsusingtwoyearsofdata
includingallteachersinthestateofFloridawheredistrictswereimplementingthe
MarzanoTeacherEvaluationModelandusingtheiObservationtechnologyplatform
tocollectobservationdata.Teachers’averageobservationsscoreswerematchedto
stateVAMstoassessvaliditycoeffcientsfortheframework.Thestudyincludedtwo
yearsofdatafromthe2012‐13and2013‐14schoolyears.Additionally,each
teacher’saveragescoreforeachelementwascorrelatedtothestatereadingVAM,
mathVAM,andalgebraVAMtoinvestigatewhethercertainelementsintheMarzano
EvaluationModelhadlargercorrelationstostudentachievementthanothers.
Forthe2012‐13results,therewereatotalof62,742teacherswhohadan
observationscore.Researcherswereabletomatch13,236(21%)ofthoseteachers
toareadingVAMand/ormathVAM.Thematchingprocesswasquiteintensive
becausewithinstateles,observationscorescouldbematchedonlybyteachername.
Table1showsthecorrelationsbetweentheaverageteacherobservationscoreand
thereadingVAMormathVAM.Asnotedbelow,bothcorrelationsweresmalland
statisticallysignificant(p<.01)withthecoefficientsranginginsizefrom.13to.14.
Table1.2012‐13MarzanoObservationCorrelationsandFloridaVAMScores
Avg.Obs.Score ReadVAM MathVAM
Avg.Obs.
Score1.00 .132** .145**
N 62,742 8,511 6,001
1Thisstudyisinprogressandwillbepublishedafterthe2014‐15stateVAMscoresarereleasedandanalyzed.Checkhttp://www.learningsciences.com/resources/formoreinformation.
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
11
Additionally,theaveragescoreforeachelementinthemodelwascorrelatedtothe
readingandmathstateVAM.Thirty‐eight,or92%,oftheelementswere
significantlycorrelatedwiththereadingVAM(n=5,021).Significantcoefficients
weresmallandrangedfrom.05to.13.Thirty‐six,or87%,oftheelementswere
significantlycorrelatedwiththemathVAM(n=3,515).Significantcoefficientswere
smallandrangedfrom.06to.13.
Forthe2013‐14results,therewereatotalof58,527teacherswhohadan
observationscore.Researcherswereabletomatch15,452teachers(26%)toVAM
data.Inthe2013‐14schoolyear,studentswerealsotestedinalgebra.Table2
showsthecorrelationsbetweentheaverageteacherobservationscoreandthe
reading,math,oralgebraVAM.Correlationsweresmallandsta‐tisticallysignicant
withthecoefficientsrangingfrom.14to.21.
Additionally,theaveragescoreforeachelementinthemodelwascorrelatedtothe
reading,math,andalgebraVAM.Forty,or98%,oftheelementsinthemodel
weresignificantlycorrelatedwiththereadingVAM(n=6,720).Significant
coefficientsweresmallandrangedfrom.05to.13.Thirty‐eight,or93%,ofthe
elementsweresignificantlycorrelatedwiththemathVAM(n=4,464).
Significantcoefficientsweresmallandrangedfrom.06to.17.Lastly,29,or71%,of
theelementsinthemodelweresignificantlycorrelatedwiththealgebraVAM(n=
642).Significantcoefficientsweresmallandrangedfrom‐.02to.27.
Thisin‐progressstudyisoneofthelargestvalidationstudiesonanobservation
framework.Thestudyhasfoundthatacrosstwoyearsofdata,theMarzano
TeacherEvaluationModelhadsignificantandsmallcorrelationswithteacher
stateVAMs.Moreover,whilethereweresmallvariationsinthecorrelationscoe
cientsbyelement,eachelementalmostalwayshadasignificantcorrelationwith
teachervalue‐addedscores.Takenasawhole,thesefindingssupportthemodel
asavalid,reliable,andaccuratesystemtomeasureteacherproficiency.
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
12
Educatorscanrelyonthemodeltoaccuratelydetermineteachereffectiveness.
2013‐14 Pinellas Pilot Findings
Inthespring2012‐2013schoolyear,PinellasCountySchools(PCS)receivedFlorida
DepartmentofEducationapprovalforaresearchprojecttodevelopateachere
ectivenesssystemthatwouldhelpteachersgrowprofessionally.Thenewsystem
wouldrevitalizetheevaluationsystem,diagnosingteacherpedagogicalstrengths
andareasforgrowth,providingtargetedsupportforindividualprofessionalskill
development,andoeringafoundationinresearch‐basedclassroomstrategiesto
improveteacherpractice.Theprojectedoutcomeofthepilotwastoincrease
studentachievementasteachersimprovedtheirpedagogythroughimmersionin,
andpracticewith,theMarzanoTeacherEvaluationModel.
Oneinnovationofthepilotwastoemployshort‐durationstudentgrowthmetrics
forteacherevaluation.Incontrasttoevaluationmeasuresthatscoredteacher
practicelongafterstudentshadlefttheclassroom(ineffect,generatingscoreswhen
itwastoolateforteacherstomakeadjustments),theideawastoimproveteacher
practicewithinasingleyearwhilestudentswerestillintheclassroom.Thepilot
includedtheuseofmultiplemetrics:teacherself‐assessment,principalobservation
scores,studentperceptionsurveys,andashort‐durationvalue‐added
Table2.2013‐14MarzanoObservationCorrelationsandFloridaVAMscores
Avg.Obs.
ScoreReadVAM MathVAM AlgebraVAM
Avg.Obs.Score 1.00 .132** .145** .205**
N 62,742 8,511 6,001 1,217
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
13
measure(VAM)basedattheunitlevel.Thepilothadtwoadditional,overarching
aims:first,tocreatethediagnosticmeasuresofteachereffectiveness,andsecond,to
documentandempiricallytestwhethertheprofessionaldevelopmentandcoaching
receivedbyteachersandleadersthroughouttheyearontheMTEMincreased
studentachievementbytheendoftheyear.
Toassessprogrameffects,aprocessandoutcomeevaluationwasconductedto
investigatewhethertheprogramhadtheintendedeffectsofincreasingstudent
achievement.Intotal,fivetreatmentschoolsandfivestatisticallymatchedcontrol
schoolswereincludedinthestudy.Onlythetreatmentschoolsreceivedthetraining,
coaching,anddiagnosticmeasuresofeffectiveness.
TwosetsoffindingsfromthisstudyarerelevanttothevalidityoftheMarzano
TeacherEvaluationModel.Thefirstfindingpertainstothemagnitudesofthe
correlationcoefficientswithVAMs.Whilethesamplesizeismuchsmallerthanthe
statelevelstudy,themagnitudesofthecorrelationsaremuchhigherwhenthe
modelisimplementedwithfidelity.Table3showscorrelationcoefficientsbetween
observationscoresandseveraldierentVAMsinPinellascounty.Significant
coeffcientsrangedfromsmalltolarge(.14to.53)withthelargestcorrelationforthe
three‐yearaggregatedmathVAMat.53.
Theoutcomeevaluationusedseveraldifferentmethodstoassessprogrameffects,
includingindependentsamplet‐tests,ordinaryleastsquaresregression,and
hierarchicallinearmodeling.Outofthe26assessmentsthathadacontrolgroup
match,21showedpositiveandsignificantgrowthforstudentsattreatmentschools
(p<.10).Consequently,favorableandsignificantresultswereshownfortreatment
studentsin81%ofadministeredassessments.Moreover,fixedeffectsmodels
showedsimilarresults:Studentswhoattendedtreatmentschoolshadsignificantly
increasedgrowthscores(.37to.39standarddeviationsaboveprediction)
comparedtostudentsatcontrolschools,whichaccountedforbothindividualand
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
14
schoolcharacteristics(Basileo,Toth,&Kennedy,2015).
“Studentswhoattendedtreatmentschoolshadsignificantlyincreasedgrowth
scores(.37to.39standarddeviationsaboveprediction)comparedtostudents
atcontrolschools,whichaccountedforbothindividualandschool
characteristics.”
ThePinellaspilotgainednationalattentionfromtheResearchSupportNetworkand
USDepartmentofEducationfortheseinnovativeeffortstoreformteacher
evaluation.
Overall,bothstudiesoutlinedhereprovideamplesupportthattheMarzanoTeacher
EvaluationModelhasbeenvalidatedinthestateofFlorida.Specifically,thefirst
study,oneofthelargestvalidationstudiesconductedonanobservationframework,
foundsmalltomoderatecorrelationswithteacherVAMsdemonstratingthat
educatorscanrelyonthemodeltoaccuratelydetermineteachereffectiveness.The
secondstudyfoundevidencethatstudentachievementsignificantlyincreased
wherethemodelwascoupledwithleadershipcoachingandimplementedwith
fidelity.
Table3.2013‐14ValidityCoefficientsinPinellasCounty
Unit
VAMS1
Unit
VAMS2
Year1
State
VAM
Read
Year1
State
VAM
Math
Year1
State
VAM
Combine
d
Year2
State
VAM
Read
Year2
State
VAM
Math
Year
State
VAM
Comb
d
Obs.
ScoreS2.104 .135* .168 .444** .239* .221 .460** .287*
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
15
N 127 249 61 40 75 64 41 75
4. Overview of the Marzano Teacher Evaluation Model and
Rubrics
The Research‐Based Model: Four Domains Directly Tied to Student Achievement
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
16
Domain1,whichcontains41ofthe60elements,focusesonpedagogical
strategiesthathaveadirectlinkwithstudentachievement.Domain1addresses
whatteachersdointheclassroom:Itreflectstheintricacyofwhathappensduring
anygivenlessonandthenaturalflowofactivities.Themodelisbasedonthe
premisethatlessonsareconstructedwithmultiplepartsandthateachpartofa
lessonhasdistinctcharacteristics,routines,andprocesses.Amodelbuilttosupport
teachersastheydeveloptheirskillsmustnecessarilyreflectthecomplexityoftheir
work.Butnotallofthe41elementsneedtobe,orshouldbe,observedinasingle
lesson.Domain1breaksdownteachinginto“thinslices”forricherdiagnosticand
feedbackpurposes.
Domain2focusesonplanningandpreparingforunitsofinstructionandlessons
withinunits.BecausetheseelementsaredirectlyrelatedtoDomain1,thebettera
teacherprepares,themoreeffectivearehisorherinstructionalchoices.
Domain3addressesdeliberatepractice.Itencouragesteacherself‐reflectionin
theareasofevaluatingpersonalperformanceanddevelopingandimplementinga
professionalgrowthplan.Whenteachersreceivespecificandfocusedfeedback
usingacommonlanguageofinstruction,theyincreasetheirexpertiseand
subsequently,studentperformance.
Domain4isthebackdropfortheotherdomainsandencouragesasupportive
culture.Itaddressescollegialityandprofessionalism,emphasizingopportunities
toobserveanddiscussstrategies. Thisdomainsupportsteacherparticipationin
lessonstudy,instructionalrounds,teacher‐ledprofessionaldevelopment,and
professionallearningcommunitiesinwhichteacherscollaborativelyexamine
evidenceofstudentlearningandtheimpactthatspecificinstructionalstrategies
haveonlearning.
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
17
The Marzano Teacher Evaluation Model Learning Map
5. Process for Classroom Observations
(Note:ThechartsbelowsummarizeLSIrecommendationsforimplementation.Pleaseseeadditionaldistrictattachments)
Protocolsforcollectingevidence,conductingevaluationconferences,developing
learning,anddevelopingperformanceimprovementplans.
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
18
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
19
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
20
TheMarzanoTeacherEvaluationModel:Michigan
ScalesfortheMarzanoTeacherEvaluationModelcopyrightRobertJ.Marzano,2013.ReportcopyrightLearningSciencesMarzanoCenter,2013.AllRightsReserved.TheMarzanoTeacherEvaluationdomains,elements,scalesandevidencesmay
onlybedigitizediniObservation.
21
Scales and Evidences for the Marzano Teacher Evaluation Model. Thefullscalesandevidencesforthemodelmaybedownloadedhere:http://www.marzanocenter.com/files/Marzano_AST_Domain1234_20130107.pdf
6. Training Plan for Evaluators and Observers
(Pleaseseedistrictattachment)
7. The Marzano Center Non Classroom Instructional Personnel
Evaluation Model
Afullreportonthemodelmaybedownloadedhere.http://www.marzanocenter.com/non‐classroom‐instructional‐support‐personnel/
8. Appendix
Paper:MarzanoTeacherEvaluationModel:PreparedfortheMichiganDepartmentof
Education.http://www.marzanocenter.com/files/MI_White_Papers_20131120.pdf
Forfurtherinformation,contactusatLearningSciences.com.