Developmental Evalua.on Toolkit - Collaboration for Impact · Developmental Evalua.on Toolkit...
15
Developmental Evalua.on Toolkit Prepared as part of the developmental evalua2on of the Victorian Children & Youth Area Partnerships commissioned by the Victorian Vulnerable Children’s Reform Unit in 2016 and undertaken by the Developmental Evalua2on Consor2um
Transcript of Developmental Evalua.on Toolkit - Collaboration for Impact · Developmental Evalua.on Toolkit...
Developmental Evalua.on Toolkit
Preparedaspartofthedevelopmentalevalua2onoftheVictorianChildren&YouthAreaPartnershipscommissionedbytheVictorianVulnerableChildren’sReformUnitin2016andundertakenbythe
DevelopmentalEvalua2onConsor2um
The Developmental Evalua.on Consor.um: TheDevelopmentalEvalua.onConsor.umisacollabora2onofconsultantswithspecificinterestandspecialistexper2seandexperienceintheuseofCollec2veImpactapproachestoaddresscomplexsocialissuesandthedevelopmentalevalua2onofsuchini2a2ves.
TheConsor2umisledbyReginaHillfromReginaHillEffec2veConsul2ngPtyLtdandincludesJennyRileyfromNaviga2ngOutcomes,KerryGrahamfromCollabora2onforImpact,MarkCabajfromHeretoThereConsul2ngandMaXhewMagainfromSketchVideos.Abriefoutlineofthecreden2alsofeachoftheseorganisa2onsfollowsbelow.
ReginaHillEffec.veConsul.ngPtyLtd(Effec.veConsul.ng):Establishedin2003,Effec2veConsul2ngisaspecialiststrategyandevalua2onconsultancythatworkswithgovernment,notforprofitandphilanthropicorganisa2ons.Itprovidesprac2cal,accessiblesolu2onsandadvicetohelpclientstodesign,implement,evaluateandfundeffec2vesocialchangeini2a2ves,policyandprogramac2vity.Effec2veConsul2ngDirector,ReginaHill,hasaspecificinterestincommunityandplacebasedapproachestoaddresscomplexsocialissuesandhassupportedanumberofcommuni2estoco-design,implementandevaluatecollabora2veandCollec2veImpactbasedini2a2ves.
Naviga.ngOutcomes:Establishedin2016Naviga2ngOutcomesisalsoaspecialistevalua2onconsultancy.Naviga2ngOutcomesDirector,JennyRiley,hasover17yearsexperienceinthesocialsector.Shehasworkedinstateandfederalgovernmentdepartmentsandforinterna2onalNGOsincludingOxfamAustralia,PlanInterna2onalandwasrecentlyUnitedWayAustralia’sHeadofCommunityImpact.InherlastroleatMurdochChildren’sResearchIns2tute,Jenny
2
The Developmental Evalua.on Consor.um: Naviga.ngOutcomes(con.nued)furtherenhancedherskillsindevelopingandimplemen2ngevalua2onframeworksforarangeofclientsfromthegovernment,non-profitandphilanthropicsectors.Withastrongfocusonfeedbackandcon2nuouslearning,Jennyhasdesignednumerousmeasurementsystemsfororganisa2onsandcollabora2ons.Collabora.onforImpact:Collabora2onforImpactisAustralia’sleadingorganisa2onforlearninghowtorespondtocomplexitythrougheffec2vecollabora2on.Collabora2onforImpactDirector,KerryGraham,hasspecificinterestandexperienceinCollec2veImpactandplaced-basedreform.Sheprovideslearning,consul2ng,coachingandevalua2onsupporttoapproximately25Collec2veImpactini2a2vesacrossAustraliaandNewZealand.
HeretoThere:MarkCabajisthePresidentoftheconsul2ngcompanyFromHeretoThereandanAssociateofTamarack,anins2tuteforcommunityengagementinCanada.Markhas25yearsexperienceinsocialchangeinEuropeandCanada,includingadvisingtheUnitedNa2ons.FornineyearsMarkwasVicePresidentoftheTamarackIns2tuteandtheExecu2veDirectorofVibrantCommuni2esCanada,Canada’sna2onalini2a2vetoapplyCollec2veImpactin100ci2estomove1,000,000householdsoutofpoverty.Mark’scurrentfocusisondevelopingprac2calwaystounderstand,planandevaluateeffortstoaddresscomplexissues.Heispar2cularlyinvolvedinbuildingtheprac2ceofdevelopmentalevalua2on. 3
The Developmental Evalua.on Consor.um: SketchVideos(Userac.ve):SketchVideosisavisualthinkingconsultancythatusesgraphicrecordingandanimatedvideotocommunicatecomplexideasinasimpleandengagingway.Ithasextensiveexperienceworkingacrossarangeofsocialissuesandisfamiliarworkingwithdiversegroupsandorganisa2onsfromboththepublicandprivatesector.SketchVideo’sDirector,MaXhewMagain,isaskilledvisualconsultantandhasextensiveexperienceingraphicrecordingandvideoproduc2onacrossarangeofsectorsincludingthegovernmentandsocialsectors.
4
• DevelopmentalEvalua2onisaformofevalua2onthatistailoredtosuitinnova2veini2a2vessuchasAreaPartnerships.
• Itisdesignedtohelpreviewandevolveini2a2vesthatareworkingoncomplexissues,par2cularlyattheearlystageoftheirdevelopment(henceitsname).
• ThepurposeofDevelopmentalEvalua2onisnotjusttounderstandwhatishappeningandwhathasbeenachievedtodate,butalsotoprovideasourceofinforma2onthroughwhichthoseengagedinanini2a2vecanreflectonandadapttowhattheyarelearningandimproveandinnovateonareal2mebasis.
WHAT?
SOWHAT?WHATNEXT?
• Assuch,developmentalevalua2onisbothanassessmentandalearningprocess.Ittakesintoaccountthechangingcontextinwhichac2vi2esarebeingundertakenandtherangeoffactorsthatinfluenceandinformcomplex,collabora2veini2a2ves.Itisnotbasedontheimplementa2onofafixedsetofassessmenttools,butratherinvolvestheapplica2onofevalua2vethinkingandareflec2velearningprac2ce.Itlooksatwhathasbeendone,whathasbeenlearnedandwhatneedstocomenexttoprogressanini2a2ve.
• Itseekstoembedareflec2ve,ac2onlearningprac2ceinthewaythatini2a2vesworkonadaytodaybasistoprogresstheirworkandsupportadapta2onandsystemsbasedchange.1
WHATISDEVELOPMENTALEVALUATION?
KeyDevelopmentalQues.ons:
©DevelopmentalEvalua2onConsor2um2016
WHAT?
SOWHAT?WHATNEXT?
KeyDevelopmentalQues.ons:Usetheschema*casatriggertosupportreflec*veprocess
©DevelopmentalEvalua2onConsor2um2016
TheoryofChangeMap:
©DevelopmentalEvalua2onConsor2um2016
TheoryofChangeMap:
Whatusetoolfor:Useschema2cofemergingtheoryofchangetomapprogressandtestalignmentofac2vitytoobjec2ves.Howusetool:• Putposteruponthewall.• Getstakeholderstousedifferent
colouredpostitnotesordotstoiden2fyareasontheposterwheretheyfeelyouaregefngtrac2on,whereyouarechallenged,areastofocusonnext…
• ReviewpaXernsanduseasthebasisforareflec2vediscussion[What?Sowhat?Whatnext?]
©DevelopmentalEvalua2onConsor2um2016
Changesinindividualprac2ce
Changesinorganisa2onal
prac2ce
Changesinhowservicesoralliances
work
Changesinhowservicenetworks|allianceswork
together
Prac.ceChangeExplora.onTool:
©DevelopmentalEvalua2onConsor2um2016
Changesinindividualprac2ce
Changesinorganisa2onal
prac2ce
Changesinhoworganisa2onsand
communitymemberswork
together
Changesinhowtheoverallsystem
works
Prac.ceChangeExplora.onTool:Whatusetoolfor:Useschema2ctoiden2fychangesinthinking,behaviourandac2vityacrossthedifferentlayersofthesystemthatyouareseekingtoinfluenceorchange.Howusetool:• Putposteruponthewall.• Getstakeholderstowriteonpost-it
notesanddescribechangesthattheyareobservingacrossthesystem(onechangeperpost-itnote)
• Getthemtos2ckthenotesontherelevantpartsoftheposter.
• ReviewpaXernsanduseasthebasisforareflec2vediscussion[What?Sowhat?Whatnext?]
©DevelopmentalEvalua2onConsor2um2016
Ourpurpose:
Whoweare:
Whatwehavedone:
Whathashelpedus: Whathasheldusback:
Whatwehavelearnedalongtheway:
Whatwehaveachievedsofar:
“BentoBox”Reflec.onTool:
©DevelopmentalEvalua2onConsor2um2016
Reflec2ons:Ques2ons: Whatcomesnext:
©DevelopmentalEvalua2onConsor2um2016
Ourpurpose:
Whoweare:
Whatwehavedone:
Whathashelpedus: Whathasheldusback:
Whatwehavelearnedalongtheway:
Whatwehaveachievedsofar:
“BentoBox”Reflec.onTool:
Reflec2ons:Ques2ons: Whatcomesnext:
Whatusetoolfor:Useschema2ctoreflectonpurpose,progresstodateandthepathwayforward.
Howusetool:• Useinsmalldiscussiongroups5–10people.• Givegroupsposters.• Getgroupstoworkthroughthethreecolourcoded
sec2onsofthefirstposterinturn.Givethem15minutesforeachofthethreesec2ons.
• PairupgroupsandgetGroup1toexplainthestorylaidoutontheirposter.Givethem10minutestodothat.GetthelisteningGroup2tolistentotheexplana2onsilently.Thengivethem10minutestoaskques2onsaboutthestorytheyhaveheard.DonotallowGroup1toanswertheques2ons.Justgetthemtowritethemdowninthefirstsec2onofPoster2.ThengetthepairedGroupstoreverserolesandrepeatthesesteps.
• Giveeachgroup15minutestoreflectontheques2onstheywereaskedandiden2fynextstepsbasedontheirreflec2ons.
• Notethetoolscanalsobeusedinsinglegroups.
Poster1:
Poster2:
©DevelopmentalEvalua2onConsor2um2016
Context: Purpose&Objec2ves:
Mechanismorchangelever(s):
Ini2a2veorPlaporm:
KSFs:
KSFs:
RoleofBackboneTeam:
Minimumspecifica2onsforIni2a2veorPlaporm:andRollout:
Minimumspecifica2onsforBackboneTeam:
• Leadershipandengagementaround:‒ Needforchange‒ Collabora2vemodel
• Evidencegroundedapproach• Coordina2onofac2vityandservices• Adap2velearningmodel(reflec2on
andtes2ngofoutcomesandrapidadap2onofresponse)
CentralBackboneReflec.onTool:
©DevelopmentalEvalua2onConsor2um2016
CentralBackboneReflec.onTool:
Whatusetoolfor:Useschema2ctoreflectonthepurpose,structureandtermsofreferenceofanini2a2veandtherequiredaXributesofthesuppor2ngbackbonefunc2on.
Howusetool:• Givegroup(s)posters.• Getgroupstoworkthroughthefourcolourcodedsec2onsinturn.Givethem15minutesforeachofthefour
sec2ons.• Ifusingmul2plegroupsgetthemtopairupgroupsandgetGroup1toexplainthestorylaidoutontheirposter.
Givethem10minutestodothat.GetthelisteningGroup2tolistentotheexplana2onsilently.Thengivethem10minutestoaskques2onsaboutthestorytheyhaveheard.DonotallowGroup1toanswertheques2ons.Justgetthemtowritethemdown.ThengetthepairedGroupstoreverserolesandrepeatthesesteps.
• Giveeachgroup15minutestoreflectontheques2onstheywereaskedandupdatework.• Thengiveeachgroup15minutedtocompletewhitesec2onsoftheposter.• ReviewpaXernsanduseasthebasisforareflec2vediscussion[What?Sowhat?Whatnext?]
©DevelopmentalEvalua2onConsor2um2016
