Academic Assessment at UTB

Steve WilsonDirector of Academic Assessment

Academic Assessment

Assessment of Student Learning Outcomes, Non Instructional Outcome Assessment, Delivery Method Assessment,5-Year Program Reviews, andGeneral Education Program Assessment

Now fall under Academic Affairs and is coordinated by the Director of Academic Assessment (see handout)

OUTCOME ASSESSMENT

SACS COC Principles of Accreditation Requirements 3.3.1.1 requires programs to:1. Identify expected outcomes2. Assess the extent to which it achieves these

outcomes3. Provide evidence of improvement based on

analysis of the results

SACS COC also requires assessment to be continuous and on-going. It should never stop.

UTB, therefore, will need to show that programs are striving for continuous improvement up to the moment it no longer exists and is merged with UTPA into UTRGV

UTB Definition

Program SLO Assessment is the systematic and ongoing process of gathering and interpreting information to discover if programs are meeting intended outcomes and then using the information to verify good practices and enhance or improve student learning.

It occurs in Academic Yearly Cycles

It has three Steps• Planning• Analysis• Closing the Loop

Planning

Analysis

Loop Closing

Planning

1. Identify an Assessment Liaison and/or Assessment Committee

2. Faculty meeting to discuss program mission, knowledge/skills/values graduating students should have learned

3. Faculty should then develop Student Learning Outcomes or SLOs

SLOs are expectations that describe the knowledge, skills and values that students should display when they complete your program

SLOs should be realistic, detailed, measureable, and should use action verbs developed from Blooms Taxonomy.

4. Bring faculty together to develop a curriculum map, showing how course level SLOs are aligned to program level SLOs

Allows faculty to indicate which courses support the achievement of specific outcomes as well as where embedded assessment can occur

5. Develop Assessment Methodology and Timelines

• Includes how each outcomes will be assessed (embedded assignments/test, exit exam, portfolio, survey of students).

• Benchmarks for success

• Not all outcomes have to be assessed every AY – cyclical.

• Use multiple approaches; don’t rely on just an exit exam.

• Be comprehensive not superficial.

6. Report: Form A on SharePoint Site

Data Analysis1. Collect and Analyze Data

Quantitative: by reporting tallies, percentages, or sophisticated statistical results Qualitative: by reporting summaries of minute papers, reflective writing, or open-ended surveys.

Results should be examined in a manner that can provide feedback by verifying what worked well and identifying areas that need improvement.

2. Report: top section of Form B on SharePoint Site

3. Develop an Improvement Plan

• Faculty meeting at the end of the academic year to discuss the results of the analyses and reflect on ways to improve student learning during the next academic year cycle.

4. Report AY 2013-2014 Continuous Plan on bottom section of Form B

AY 2014-2015 Exception

Normally improvement plans are completed on second section of Form B

Because the Merger will take place in 8/2015, AY 2014-2015 Continuous Improvement Plans are not necessary. However, to wrap UTB assessment up into a nice package, we want departments to write a “lessons learned” report, based on reflections of the 2014-2015 assessment findings. • Form will be available on SharePoint Site

Closing the Loop

1. While the improvement plan is being implemented, programs should collect data to assess whether the plan worked, effectively improving student learning in those areas identified during the analyses.

2. Report Form C on SharePoint site

You WILL need to submit a Form C to close the AY 2013-2014 loop

You will not close the AY 2014-2015 loop

School/College Assessment Committee

Each department Assessment Liaison will be part of a college level assessment committee. The Associate Dean will chair the committee. The committee will meet once a month to discuss assessment related activities, including SLO assessment, to ensure sufficient progress is being made to meet the appropriate deadlines, and to report any issues or problems to the relevant officers, like the Dean of the College, Director of Academic Assessment or the Executive Director of the ORIPE.

Action Steps

Faculty meeting to discuss AY 2013-2014 analyses, the AY 2013-2014 Improvement Plan, and the AY 2014-2015 Assessment Plan

Collect Data and Analyze for AY 2013-2014 Improvement Plan

Collect Data and Analyze for AY 2014-2015 Assessment Plan

Faculty meeting to discuss AY 2014-2015 analyses , if the AY 2013-2014Improvement plan was successful , and lessons learned from the 2014-2015 analysis

8/2014 9/2014 10/2014 to 4/2015 5/2015 6/2015

Deliverables

  AY 2013 – 2014 Improvement PlanAY 2014-2015  Assessment Plan

  AY 2014-2015 Analysis Report

Close the AY 2013-14 loopAY 2014-2015 Lessons LearnedReport

NON-INSTRUCTIONALGOAL ASSESSMENT

UTB Definition

Non-Instruction Goal Assessment is the systematic and ongoing process of gathering and interpreting information to discover if academic departments are meeting their intended goals and then using the information to verify good practices and enhance or improve services.

Note: This is a Departmental Level Assessment, not a Program Level Assessment

It occurs in Academic Yearly Cycles

It has three Steps• Planning• Analysis• Lessons Learned

Planning

Analysis

Lessons

Planning

1. Chairs or Departments should ID2-3 short term goals that need improvement or attention

2. Normally, goals are aligned to the University’s strategic plan – we don’t have one

3. So 2014-2015 goals should be aligned to:

• Chancellor’s Framework for Excellence

• Texas Gov’t Code 2114: Customer Service

• Improvement of Dept. Effectiveness/Efficiency Measures

• Your own value (e.g. items identified in Program Review)

SMART Goals are:

Specific (Describe exactly what will be accomplished)Measureable (Success can be assessed)Actionable (Action steps can be implemented to achieve goal)Realistic (The endpoint is achievable)Timely (Can be done in one academic year)

2. Once goals have been identified, Chairs or Departments should:

• Plan achievable action steps• Determine how each goal will be

measured to determine success• Identify the criterion (or target)

that will be used to determine success

• Identify who is responsible for overseeing the assessment of the goal

• Identify required resources

3. Report Form A on SharePoint Site

Data Analysis1. Collect and Analyze Data

Quantitative: by reporting tallies, percentages, or sophisticated statistical results Qualitative: by reporting summaries of minute papers, reflective writing, or open-ended surveys.

Results should be examined in a manner that can provide feedback by verifying what worked well and identifying areas that need improvement.

2. Report top section of Form B on SharePoint Site

UTB/UTPA Merger Exception

Normally you would complete the improvments and closing the loop steps

Because the Merger will take place in 8/2015, AY 2014-2015 Continuous Improvement Plans are not necessary. However, to wrap UTB assessment up into a nice package, we want departments to write a “lessons learned” report, based on reflections of the 2014-2015 assessment findings.

School/College Assessment Committee

Each department Assessment Liaison will be part of a college level assessment committee. The Associate Dean will chair the committee. The committee will meet once a month to discuss assessment related activities, non-instructional goal assessment, to ensure sufficient progress is being made to meet the appropriate deadlines, and to report any issues or problems to the relevant officers, like the Dean of the College, Director of Academic Assessment or the Executive Director of the ORIPE.

Action Steps

Chair should identify about 3-5 short-term goals that need improvement/attention

Collect Data and Analyze for AY 2013-2014 Non-Instructional Goal Assessment Plan

Chairs should reflect on lessons learned from the 2014-2015 analysis

8/2014 9/2014 10/2014 to 4/2015 5/2015 6/2015

Deliverables

  AY 2013 – 2014  Non-Instructional Goal Assessment Plan 

  AY 2014-2015 Analysis Report

Close the AY 2013-14 loopAY 2014-2015 Lessons LearnedReport

COMPARING DELIVERYMETHODS

SACS COC requires Universities to compare distance and other delivery methods to campus-based courses as part of their program assessment, including assessments of student learning outcomes

It occurs in Academic Yearly Cycles

It has three Steps• Planning• Analysis• Lessons Learned

Planning

Analysis

Lessons

Planning

1. Chairs should appoint an Assessment Liaison. Given the large amount of work required by assessment, it is recommended that the Liaison chair a departmental assessment committee so that the work is manageable.

2. At the beginning of each academic year, the assessment liaison and/or assessment committee, working with department chair, will identify courses that deliver the same content using different delivery methods.

3. The assessment liaison and/or assessment committee will then coordinate with the instructors of those courses to identify student work that can be used to assess student attainment of PROGRAM outcomes as well an assessment methodology and benchmarks.

PROGRAM LEARNING

OUTCOMES

3. Report

Data Analysis1. Collect and Analyze Data

Quantitative: by reporting tallies, percentages, or sophisticated statistical results Qualitative: by reporting summaries of minute papers, reflective writing, or open-ended surveys.

Results should be examined in a manner that can provide feedback by verifying what worked well and identifying areas that need improvement.

2. Report

UTB/UTPA Merger Exception

Normally, Delivery Methods Assessment would finish with improvement and closing the loop steps.

Because the Merger will take place in 8/2015, AY 2014-2015 Continuous Improvement Plans are not necessary. However, to wrap UTB assessment up into a nice package, we want departments to write a “lessons learned” report, based on reflections of the 2014-2015 assessment findings.

School/College Assessment Committee

Each department Assessment Liaison will be part of a college level assessment committee. The Associate Dean will chair the committee. The committee will meet once a month to discuss assessment related activities, including delivery method assessment, to ensure sufficient progress is being made to meet the appropriate deadlines, and to report any issues or problems to the relevant officers, like the Dean of the College, Director of Academic Assessment or the Executive Director of the ORIPE.

Action Steps

Faculty will identify courses for assessment, methodology, and benchmarks

Collect Data and Analyze for AY 2013-2014 Non-Instructional Goal Assessment Plan

Chairs should reflect on lessons learned from the 2014-2015 analysis

8/2014 9/2014 10/2014 to 4/2015 5/2015 6/2015

Deliverables

  AY 2013 – 2014  Non-Delivery Method Assessment Plan 

  AY 2014-2015 Analysis Report

Close the AY 2013-14 loopAY 2014-2015 Lessons LearnedReport