ReportfromtheCoreCounciltotheFacultySenateonCoreAssessment2015-2016

TableofContents

Introduction ………………………………………………………………………. 2WhatWeLearnedAboutCoreAssessmentin2015-2016…….... 3IssuesThatNeedFurtherAttention………………………………………. 4RecommendationsfortheFuture………………………………………….. 5AppendixA-DescriptionofUALRCore…………………………………... 7AppendixB-ChargeoftheCoreCouncil…………………………………..9AppendixC-ProgressonCoreAssessmentforAcademicYear2014-2015andFall2015……………………………………………………… 10AppendixD-CommunicationStructurefortheCAACs,Liaisons,AndCoreCouncil…………………………………………………………………….11AppendixE-OutlineofCoreAssessmentPlanfor2015-2016……… 12AppendixF-LiaisonReports………………………………………………………..separatefileAppendixG-CAACReports………………………………………………………..separatefileAppendixH-TimelineforFutureAssessment……………………………..separatefile

1

ReportfromtheCoreCounciltotheFacultySenateonCoreAssessment2015-20165-6-16

MembersoftheCoreCouncilRebeccaStreettandBrianMitchell,CALSElizabethSloanDavidsonandLindaStauffer,CEHP NickJovanovicandMichaelDeAngelis,CEITMarkFunkandJoeFelan,COBBelindaBlevins-KnabeandTustytenBensel,CSSC

Introduction

ThisreportcoversprogressonCoreAssessmentforacademicyear2015-2016.AdescriptionofthecoreisincludedinAppendixA.TheCoreCouncilisastandingcommitteeoftheFacultySenateandhasthechargeofoverseeingcoreassessment(seeAppendixBforthecharge).Workoncoreassessmentstartedin2014-2015(AppendixC).CoreAreaAssessmentcommitteeswereestablishedforeachcurricularareaandonememberoftheCoreCouncilwasappointedtoserveasaliaisonandserveasthecommunicationlinkbetweentheCAACandCoreCouncil.(SeeAppendixD)During2015-2016eachcurricularareawasaskedtopilotrubricsdevelopedinSpring2015forassessingtheirKnowledgegoal(s).RhetoricandWritingandSpeechcommunicationwereassignedthefirstskillsgoal.Thepurposeofthepilotwastotesttherubricsandtodevelopprocessesthatcouldbefollowedforfutureassessments.SeeAppendixEfortheinstructionstoeachCAACoutliningthisprocess.EachCAACwasaskedtosubmitareportdescribingthepilotresults.RepresentativesofeachCAACmetwiththeCoreCouncilinSpring2016andfollowingthosemeetingseachliaisonwroteabriefreportansweringcommonquestionstofacilitatecomparisonsacrossreports.ThesereportsalsoincludecourseenrollmentsforSpring,Summer,andFall2015.TheliaisonsreportsareincludedinAppendixFandtheCAACreportsareincludedinAppendixG.ThetimelineforfutureassessmentisincludedinAppendixH.

WhatWeLearnedAboutCoreAssessmentin2015-2016

Overall:Aremarkableamountofprogresshasoccurredoverthecourseofa2-yearperiod.ThecoreassessmentprocessisfacultyledandfacultydrivenwhichisinlinewiththechargefromtheFacultySenate.OurcoreassessmentprocessisconsistentwiththeHigherLearningCommission’semphasison“acultureofcontinuousimprovement”

2

Rubrics-Allgroupsreportedsuccessusingtherubricstoscorestudentwork.Mathmentionedmodifyingthewordingoftherubrictoaidinscoring.Speechmentionedaddingmoredetailtotherubrictofacilitatebetteridentificationofstudentstrengthsandweaknesses.Allgroupswereabletosuccessfullyrateusingthe4-pointscale(however,notecommentunderissuesbelow)Participation:Themajorityoftheprogramsparticipatedandaremakingsteadyprogressonassessment.Someprogramschosenottoparticipate(Music)andsomeprogramsparticipatedminimally(someoftheSocialSciences).PartofthechargeoftheCoreCouncilistodevelopare-approvalprocessforcorecourses.Oneofthecriteriaforre-approvalwillbesatisfactoryparticipationinthecoreassessmentprocess.Selectionofstudentwork:TheCoreCouncilrecommendedthatsectionsofcorecoursesberandomlyselectedforassessmentandthattheselectionoccurafterthestartofthesemester.Thiswastoensurethattherewasno“teachingtothetest.”Thisworkedbetterforsomegroupsthanothers.Severalgroupsusedrandomnumbergeneratorstoselectsections.However,othergroupsdidnotselectrandomly.Insomecasesitwaschallengingtogetinstructorsunfamiliarwithassessmenttosubmitartifactsthatcouldbescoredusingtherubrics.Severalgroupsexpressedconcernsoverhowthiswouldworkwithconcurrentcoursesandadjuncts.Therearealsocasesinwhichthereisonlyonesectionofacourse.TheCoreCouncilcontinuestorecommendthatsectionsofcorecoursesberandomlyselectedforassessmentandthattheselectionoccurafterthestartofthesemester.Raters:CAACsreportedawidevarietyinthetypesofindividualswhoratedthestudentworkincludingadjuncts,graduatestudents,andfull-timefaculty.Inmostcases,exceptforCAACswithonlyoneprograminthecurricular,arearaterswereoutsidethediscipline.TheCoreCouncilrequestedthattheraterscomefromoutsidethediscipline,whenpossible,basedontheassumptionthatiftherubricsandassignmentinstructionswerewellwrittenthenarateroutsidethedisciplinecouldobjectivelyratethestudentwork.CompensationforRaters:Somegroupswereabletopayasmallstipendtoratersandothersrecommendedit.SeerecommendationsbelowunderCompensationandFacultyRole.Reliability-Reliabilityofratersisnecessaryforconfidenceintheassessmentresults.Many,butnotallcurricularareas,reportedreliability.Reliabilityneedstobeapriorityinthenextassessmentcycle.

3

Typesofassignments-CommonrubricscanbeappliedtomanytypesofassignmentsandtheCAACsputthattothetest.Multiplechoice,fillintheblank,labreports,papers,problemquestions,finalessays,electronicportfoliowithmultipleassignments,andvideorecordingswereallused.Themainissuewasthealignmentoftheassignmentwiththelearningobjectivesandnotwhethertherubriccouldbeusedwithawidevarietyofassignments.Timeestimates-Timeestimatesrangedfrom2minutesto15minutestoscoretheworkofonestudent.Theestimatesdependedonthetypeofstudentwork.TheCoreCouncilanticipatesthatwithexperiencetheseestimateswilldecline,butwewanttocontinuetotrackthesenumberssowehavearoughideaoffacultytimerequired.

IssuesthatNeedFurtherAttentionConcurrentcourses:Someprogramshavecloserelationshipswiththeirconcurrentinstructorswhileothersdonot.Oneissuethatwefaceishowtomakesureconcurrentinstructorsparticipateincoreassessmentandincludetherelevantlearningobjectivesintheirsyllabi.Adjuncts:Whilesomeprogramsincludedadjunctsintheassessmentpilotothersdidnot.Inaddition,fewtonoonlinesectionswereincluded.Allmodalitieswillneedtobeincludedinthefuture.Assessmentmanagementsystem:Currentlywedonothaveanelectronicassessmentsystem.Withoutasystemissuessuchasstorage,accesstostudentwork,andrubricaggregationwillneedtobeaddressed.Rubricscale:Allrubricsarecurrentlybasedona1to4scale.Weneedtoclarifythedifferencebetweenstudentworkthatdoesnotreachthethresholdforaratingof1andworkthatdoes.Includingbothinthesamecategoryconfoundsthecomparisonswiththeotherpointsonthescale.Addinganotscorablecategorycouldsolvethisproblem.Samplesize-BoththeHumanitiesandFineArtswantdirectiononthisissue.Theansweristhatnecessarysamplesizeisdirectlydependentonthequestionsthatareasked.Lowersamplesizesareacceptableifmeasuresaremorereliable.Seerecommendationsbelow.Studentwork-Findinganassignmentorassignmentsthatalignedwiththelearningobjectiveswaschallengingforsomeareas.Manygroupsareconsideringcommonassignmentsforthefuture.Anotherpotentialsolutionisforinstructorstobemoreintentionalinconnectingassignmentstolearningobjectives.Inthefutureallgroupswillneedtohaveassignmentsthataddressthelearningobjectives.Thesedonothavetobecommonassignments.Humanitiesstatedthat

4

withoutacomprehensivefinalassignmentthelearningobjectivescouldnotbeassessedandthisproblemwillneedtobeaddressedinthefuture.Itisnecessarytoassessthelearningoutcomesandobjectives.Gradesversusassessment:Someconfusionoverdifferenceingradesversusassessment.Gradesandstudentlearningoutcomesassessmenthavedifferentpurposesandfuturefacultydevelopmentmayneedtoaddressthedifferencesbetweengradesandassessment.Pre-posttestmethodology:TwocurricularareasBiologyandHistorymentionedthevalueofpreandposttests.Thismethodologycouldprovidevaluableinformationaboutthechangethatdoesordoesnottakeplaceinthesemester.Itcanalsoprovideinformationaboutthereliablyofmeasuresthatcanbeusedtomakeestimatesaboutneededsamplesize.

RecommendationsfortheFuture

Facultydevelopment:Futuresessionsonfacultydevelopmentneedtocovertherelationshipsbetweenassignments,rubrics,andlearningobjectives;thedifferencebetweengradesandassessment,Inaddition,theCoreCouncilrecommendsafacultydevelopmentsessionaddressingtheissuesofreliability,samplesize,rubricconstructionetc.Thereareseveralwebresourcesthatarerelevantandwecouldtapintocampusexpertiseforthesession.Alternatively,wecouldadoptmetricthatiscommonlyusedwhichis20%ofthetotalsample.Theadvantageofthismetricisitgivesanimmediateanswerandthedisadvantageisthatthenumbercanbehigh.CompensationandFacultyRole:TheCoreCouncil,respondinginparttoquestionsbymembersoftheCAACs,draftedlegislationfortheFacultySenatethataddressedtheissueofhowassessmentdataisused.Thelegislationstatedthatprogrammaticstudentoutcomesassessmentdatawasnottobeusedinevaluatingindividualfacultyforpromotion,tenure,orannualreview.However,participationinassessmentactivitiescouldbeusedinevaluationindividualfacultyforpromotion,tenure,orannualreview.Thechancellorvetoedthislegislation,however,heencouragedustocontinuetoworkontheissueandthatworkwillcontinuenextyear.Thislegislationrecognizedthatparticipationinassessmentispartwhatisexpectedoffull-timefaculty.TheCoreCouncilrecognizesthatcompensationmightbeappropriateincasesinwhichadjuncts,andgraduatestudentsserveasratersofstudentwork.Smallamountsofcompensationmightalsobeappropriateforfull-timefacultywhohaveaheavyassessmentworkloadorareparticipatinginextratrainingorfacultydevelopment.However,itisimportantthatfacultyowntheassessmentprocess.Whateversystemofcollecting,analyzing,andimplementing

5

assessmentdataisadoptedineachcoreprogramistheresponsibilityoftheentirefaculty.

6

AppendixA

DescriptionofUALRCoreImplementedJanuary20159curricularareasScienceCommunication-WrittenFineArtsSocialSciencesUSHistory/GovernmentHistoryofCivilizationHumanitiesMathematicsCommunication-SpeechThecoreaddressesthefollowingeducationalgoals:Skills1. Communication(oral,written,visual,secondlanguage;professionalself- presentation).2. Criticalthinking,quantitativereasoning,andsolvingproblemsindividually andcollaboratively.3. Informationtechnology(locating,retrieving,evaluating,synthesizing).Knowledge1. Theconcepts,methodologies,findings,andapplicationsofmathematics andthesocialandnaturalsciences,engineeringandtechnology.2. Theconcepts,methodologies,andtheglobalculturalheritageoftheartsand humanities.Values1. Ethicalbehaviorandpersonalresponsibility.2. Civicresponsibility.3. Globalandculturalunderstanding

7

MapofEducationalGoalsRepresentedintheUALRCore

CurricularAreas

EducationalGoals

Communication-Written

Communication-Speech Math

FineArts

Human-ities Science

SocialSciences

HistoryofCivilization

USTraditions

Knowledge1:Math&Science

X X X x x

Knowledge2:Arts&Humanities

X X x

Skill1:Communication

X X X x x x x x x

Skill2:CriticalThinking

X X X x x x x x x

Skill3:InfoTechnology

X X X x x x x x x

Value1:Ethical

X X X x x x x x x

Value2:CivicResponsibility

X X x x x x x x

Value3:Global&Cultural

X x x x

8

AppendixBChargeoftheCoreCouncil

Leg-FS-2012-13_11CouncilonCorePoliciesAmendedMarch11,2016TheUALRCouncilonCoreCurriculumandPolicies (henceforth'Council')astanding committeeofthe FacultySenate,will beresponsiblefor:

1.Maintainingthe criteriafor determiningwhetheracourseisappropriateto satisfy acurricularareainthe UALRStandard Core;2.Maintainingasubmission andapproval processfor UALRStandardCorecourse inclusion;3.Developingandmaintainingare-approvalprocess,includingatimeline,for UALRStandardCorecoursecontinuation;4.Maintainingaplan to assesseducationaloutcomesofthe UALRStandardCore;5.Overseeing college/programcourse inclusionbeyond the UALRStandard Coretoensure thecourseworkmeets the StateMinimumCore;6.Providingadetailed,written reviewto submittersfor alldecisions;7.Grantingexemptionsto the UALRStandard Corerequirementstoprograms, whereappropriate,asoutlinedinpart C.8.Developingapplicationandexemptionformsreflectingtheseguidelines.

9

AppendixC

ProgressonCoreAssessmentforAcademicYear2014-2015andFall2015

Fall2014.Thegeneralplanofusingcommonrubricswithincurricularareaswasadopted.Eachcurricularareahascommoneducationalgoalsandlearningoutcomesforthosegoalsandrubricswillbebasedonthelearningoutcomes.Thelearningobjectivesforaparticularcoursewithinacurricularareavary.Spring2015.Eachcurricularareaformedacommitteewitharepresentativefromeachdepartmentwithacorecourseinthatcurriculararea.TheCAACs(CoreCurricularAssessmentCommittees)includedamemberoftheCoreCounciltoserveasaliaison.Thefirsttaskwastocompleteasyllabusanalysisofallcorecoursestodeterminehowwhethercoursesyllabicoveredtherelevantlearninggoalsandobjectives.Inaddition,theCAACsweretodevelopaplanforcollectingstudentartifactsacrossavarietyofassignmentsandtodesignacommonrubricbasedonthecommonlearningoutcomestoassessthem.Thisprocessproducedthefirstassessmentoutcomes.Everygroupdiscoveredthatmostsyllabididnotincludetherelevantlearningoutcomesandobjectives.TheCoreCouncilrequestedthatinthefutureallcorecoursesincludethelearningoutcomesandobjectives.Fall2015.TheCAACswereaskedtodevelopaplanforhowtocollectdataandtocollectpilotdata.EachareawasaskedtosubmitareportdueFebruary19,2016reportingbothontheprocessofdatacollectionandonhowwelltheirrubricworked.

10

CoreCouncil

LiasionsHistoryofCivilization USTraditions SocialScience Math

CommunicationWritten

CommunicationSpoken Hummanities FineArts

CAACs History History Chemistry CriminalJustice MathRhetoricandWriting Speech English MassCommunication

PoliticalScience Physics Sociology Philosophy ArtAnthropology GenderStudies Religion MusicBiology Anthropology TheatreEarthScience PsychologyAstronomy Geography

PoliticalScience

Science

Appendix D 11

AppendixE

OutlineofCoreAssessmentPlanfor2015-2016

1.LearningObjectivesAllcorecourseswillincludethelearningobjectivessubmittedtotheCoreCouncilintheirsyllabi.Syllabiwillalsoincludethegoalsandoutcomesoralinktothem.2.Pilot

Lastspring(2015)allareassubmittedarubricorbeganworkingonarubric.Thecompletedrubricisnecessaryforthisassessmentprocess.Therubricassessedtheknowledgegoal1or2andaskillsgoalinthecaseofSpeechandRhetoricandWriting.

Eachcurricularareawilldevisearandomizedwaytoselectasectionofeachcourseinthecore.Theirmethodofselectionwillbecoveredintherequiredreport.

Theinstructorsintheselectedsectionswillprovide3studentartifacts,onehigh(Awork),medium(CWork),andlow(Fwork)fromanassignmentthatmatchesthegoalandoutcomesassessedbytherubric.Weanticipatethatthiswilloccurtowardtheendofthesemester.

Theinstructorfromthesectionsendstotheassessors(membersoftheCAAC)theworkofallstudentsinthatsectiononthatassignmentdividedinto3categories:high(Awork),medium(Cwork),andlow(Fwork).

Assessorsrandomlypickonefromeachcategoryandscore.WiththeexceptionofMath,Speech,andRhetoricandWritingassessorswillnotscoretheartifactsfromtheirowndiscipline.Inallcases,assessorswillnotscoreartifactsfromtheirowncourses.Thismeansthatthedisciplinerepresentativewillneedtoprovideadetailedscoringkeyforthecontentoftheassignment.

Atthispointwearetestingtherubricandtheprocessofassessment,nothowwellthestudentsdid.

Forreliabilitytwopeoplewillscoreeachartifact.Percentagreementwillbecalculated.Thereportwillcoverreliabilityandhowdisagreementswereresolved.

Thisisaniterativeprocess.

Iftherubricischangedthentheassessorscangobacktothestudentworkthatwasnotscoredandtestthenewrubric.

12

Continuousimprovementmeansitneverends.Thesecondroundofassessmentbeginsinthespringwiththeimmediategoaloftestinganychangesandthelong-termgoalofassessmenttolookatstudentlearningbeginninginFall2016.

c.Timeline.

Fall2015:Eachcurriculararea:Planhowtocollectdata,includingawaytorandomlysectionwhichsectionsofcoursestoincludeiftherearemultiplesectionsdevisingandfine-tuningtherubric,collectingartifacts,andscoringdata.Scoringthedatamaytakeplaceinearlyspringifartifactsarecollectedattheendofthefallsemester.

Reportdue:February19,2016

Thenrevisetherubricinthespringsemester.

4.Concurrentsectionsofcorecourses

TheCoreCouncilhasrequestedcopiesofsyllabifromconcurrentcoursesfromtheProvost’soffice.Copieswillbeprovidedtotherelevantdepartments.

GuidelinesforSpring2016CAACReportsDue:February19,2016SubmitreporttotheCoreCouncilLiaisonIncludeinthereports:1.DatainthespreadsheetprovidedbytheCoreCouncil.Pleaseincludeanysuggestionsforimprovements.2.Describetheprocessfollowedtorandomlyselectsections.Includesuggestionsforimprovements.3.Howdoyouknowtherubricworked?Provideexamples.4.Describeproblemsandsolutionsfortheproblemswiththerubricscoringprocessandinterpretationsofthedata.5.Providetimeestimateofhowlongittakestoscoretheworkofonestudentforonelearningoutcome.

13

6.Explainhowthestudentworkthatwascollectedcoversthelearningobjectives,andassociatedlearningoutcomes.Pleaseincludeanyrecommendationsforfuturechanges.

14