Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality

8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality

1/49

Setting Institutional Standards for Student

Achievement:

Meeting the Call for Academic Quality

Fred Trapp, Cambridge West Partners

Carolyn Arnold, Chabot College

Daylene Meuschke, College of the Canyons

Elaine Kuo andAndrew LaManque, Foothill DeAnza Community

College District

Robert Pacheco, MiraCosta College


2/49

Some RP Resources

Learning Assessment Listserv (www.cccnext.net)

RP Group Listserv (www.cccnext.net)

Perspectives (www.rpgroup.org)

Cloud Housing Session

Docs http://tinyurl.com/mjdu9ha
http://www.cccnext.net/http://www.cccnext.net/http://www.rpgroup.org/http://tinyurl.com/mjdu9hahttp://tinyurl.com/mjdu9hahttp://tinyurl.com/mjdu9hahttp://www.rpgroup.org/http://www.cccnext.net/http://www.cccnext.net/


3/49

Outcomes for the Session

Discover some best practices in setting college-

wide standards for achievement.

Brainstorm ways to integrate the standards intoplanning and decision-making processes at your

institution.

Evaluate the reasonableness of your colleges

institution-set standards.


4/49

How Well Roll

Trends and a Literature Review (10-15 minutes each)

Three College/District Stories (10-15 minutes each)

Interactive Portion (25 minutes)

Reflect on your own institution, its culture, history, mission to consider yourown institution-set standards. Compare, borrow, validate


5/49

Institution SetStandards: Trends &

Literature

Frederick Trapp

Cambridge West Partnership, LLC

Long Beach City College (retired)

October 2013


6/49

Professional Literature/National Trends

The three Es of public agencies

1. Economy

2. Efficiency

3. Effectiveness

How high or low is the effectiveness

bar is set to define quality service?


7/49

ACCJC Annual Report, 2013

Student achievement performance metrics

14b Successful student course completion rate

15b Student retention percentage

16b Student degree completion

17b Student transfer to 4-year colleges

18b Student certificate completion

A standard is the level of performance set by the institution

to meet educational quality and institutional effectiveness

expectations. This number may differ from a performance

improvement goal which an institution may aspire to meet.


8/49

Standard I.B.2

The institution sets goals to improve its effectiveness consistent

with its stated purposes. The institution articulates its goals and

states the objectives derived from them in measurable terms so

that the degree to which they are achieved can be determined

and widely discussed.


9/49

Standard I.B.3

The institution assesses progress toward achieving its stated

goalsand makes decisions regarding the improvement of

institutional effectiveness in an ongoing and systematic cycle of

evaluation, integrated planning, resource allocation,

implementation, and re-evaluation.

How bad does performance have to get before we

decide it is unacceptable and we need to do something?


10/49

US Dept. of Ed., New Regulations

34 Code of Federal Regulations (CFR) & USDE Guidelines for

602.16(a)(1)(i).

Standards effectively address success with respect to student

achievement in relation to the institutions mission, which may

include different standards for different institutions or programs,as established by the institution, including, as appropriate,

consideration of course completion, State licensing examinations,

and job placement rates.

Whether institutionally-developed standards to demonstratestudent success are being used by the accreditorin the

accreditation assessment, and the institutions performance with

respect to student achievement is assessed.


11/49

ACCJC Direction to Teams

Evaluation teams will examine student achievement data at

the programmatic and institutional levels. The institution must

set standards of satisfactory performance for student

achievement and evaluate itself against those standards at

both levels.1. Examine the institutional set-standards for student

achievement and assess their appropriateness

(reasonableness)

2. Consider the set-standards in the context of satisfactory

performance, goals for improvement (achievement and

learning).

3. Cite this information as evidence in describing its evaluation

of how well the institution accomplishing its mission.


12/49

ACCJC Direction to Teams

Describe in the team report

Not the basis of a recommendation or

a sanction


13/49

Health Care- Setting Standards

National Quality Forum (1999)

700 best in class quality measures, rooted in evidence-based

practice

Medical Loss Ratio (80-85% of premium goes to care)

Centers for Medicare & Medicaid Services (CMS)- AccountableCare Organizations (ACOs) initiative performance domains

Patient/caregiver experience

Care coordination/patient safety

Preventive care At-risk populations

Moving from fee-for-service to pay-for-performance using

national performance benchmark data and a sliding scale.


14/49

Performance Scorecards and Dashboards

Strategicmeans visually to monitor achievement of objectives Rests on some cause and effect beliefs

Association for Institutional Research (AIR) study of 66 higher

education dashboards

Category % Using

Financial Indicators 80%

Admissions 79%

Enrollment 77%

Faculty 77%

Student Outcomes 72%

Student Engagement 58%Academic information 55%

Physical Plant 38%

Satisfaction 35%

Research 35%

External Ratings 21%


15/49

Setting the Performance Standard

Levels set by1. Benchmarking (www.NCCBP.org)

2. Industry standards

3. Recent performance history

5-year average

3-year rolling average

Color codes for performance (stop light)

Red- poor; yellow- below target; green- on target

Performance expressed as a range

=>5% above- very good

=


16/49

Performance Funding for Higher Education

In place

30%

In

transitio

n8%

Considering

32%

No Action

30%

50 States

National Council of State Legislatures


17/49

Measuring Institutional Effectiveness

through Institutional Standards One

Colleges Approach

Daylene M. Meuschke, Ed.D.

Director, Institutional Research

College of the Canyons

Strengthening Student Success Conference

October 2013


18/49

The Plan / TimelineMarch 2013*Required to establishstandards

*Convened College PlanningTeam sub-group

*Established indicators andstandards

April-June 2013

*Presented Indicators-Senate,College Planning Team, Boardof Trustees

*Obtained feedback onPerformance Indicators

April 2013-present -*Held monthly meetings

*Using RP Groups Principles ofRedesign to guide planningactivities

Source: www.photo-dictionary.com


19/49

Source: www.2.acs.ncsu.edu

StudentLearning /

Institutional

Effectiveness

ACCJCAnnual

Survey

Scorecard

InstitutionalData

Existing Data Sources
http://www.2.acs.ncsu.edu/http://www.2.acs.ncsu.edu/


20/49

COCs Performance Indicators

and Institutional Standards

Coming soonMetric for Career Technical Education


21/49

Planning Guided by RP Groups

Principles of Redesign1. Accelerate entry into coherent programs of study

2. Minimize time required to get college-ready

3. Ensure students know the requirements to

succeed4. Customize and contextualize instruction

5. Integrate student support with instruction

6. Continually monitor student progress and

proactively provide feedback

7. Reward behaviors that contribute to completion

8. Leverage technology to improve learning and

program delivery


22/49

Infusing the Standards

College-wide

College Planning Team

Performance Indicators Sub-Committee

Skills for Success CommitteeRoadmap Project (AAC&U)

Grants (e.g. Title III)

Departments

Academic and Administrative Program Reviews

Documented in Research Brief #57

More to Come!


23/49

Next Steps:

Revise Indicators

Add CTE

Revise Standards Completion (success)

Continue Disaggregating

Connect to departments

Some are clean, clear Others (e.g., transfer)-

more problematic

Source: www.burkemuseum.org

Just scratching the surface

Plan

ImplementEvaluate
http://www.burkemuseum.org/static/polarbear/biblio.phphttp://www.burkemuseum.org/static/polarbear/biblio.php


24/49

Any questions?

Thank you!


25/49

Measuring up to the StandardsCreating Meaningful Student Groups to measure

progress towards Institutional Standards

Carolyn Arnold, Ph.D.

Coordinator, Office of Institutional Research

Planning, Review, and Budget Council (PRBC)

Chabot College


26/49

Chabot Institutional Standards

Setting the standards

Analyzed long term trends for the 5 standards

Chose five year rolling averages Reflect recent trends

Smooth out extreme highs or lows

Sometimes above standard; sometimes below

Assumption no matter where we set them How can we improve these outcomes?


27/49


Challenge 1 to improving the outcomes

Degrees, Transfers, and Certificates are long-term outcomes Take semesters or years to complete Completion numbers fluctuate year to year

Most college initiatives focus on short termobjectives: More students taking basic skills courses

Increasing success in College English, Math

How can short and long term be connected?


28/49


Challenge 2 to improving the outcomes

Not all students aiming for these outcomes Have different goals

Going towards goals at different speeds

Starting from different places

How can we take into account student differencesin goals, speed, and starting places when we

attempt to improve outcomes? We do want to improve!


29/49

Chabot Strategic Plan Goal

Increase the number of students who achieve theireducational goal in a reasonable time (outcomes) Institutional Research supposed to measure the number

by clarifying pathways and providing moreinformation and support (methods) College initiatives supposed to improve the number


30/49

Not One Number!

Why we created distinct student groups

To take into account student differences in goals, speed, levels

To monitor progress and outcomes of studentcohorts with different educational goals

To determine reasonable time to completion foreach group

To target college initiatives and support servicesneeded by each group


31/49

Creating educational goal

groups for new students

Groups based on:

Their educational goal Self-reported educational goal

How long they will take

How many units they take (FT/PT) Where they start (assessment in English)


32/49

Creating

educational goal groups

Laser(FT) College

Laser(FT) Basic Skills

Laser(FT) Not Assessed

Seeker (PT) College

Seeker(PT) Basic Skills

Seeker (PT) Not Assessed

Explorer

Career-builder FT

Career-builder PT

Skills-builder

Student Ed Goal GroupsEngl ish

AssessmentUnitsEd Goal

College

Basic SkillsNot Assessed

College

Basic Skills

Not Assessed

Full-

time

Part-

time

6+ units

Transfer

or

Degree

(GE)

UndecidedCertificate or

Job training

Cert/Job/Und/

Pers Devel

Full-time

Part-time 6-11 units

Under 6 units


33/49

10 student

educational goal groups Laser (FT) College

Laser (FT) Basic Skills

Laser (FT) Not Assessed

Seeker (PT) College

Seeker (PT) Basic Skills

Seeker (PT) Not Assessed

Explorer

Career-builder FT

Career-builder PT

Skills-builder


34/49

Chabot Fall 2012 New StudentsPercentage in each Educational Goal Group

8%

22%

3%

5%

25%

10%

10%

2%

3%

12%

Laser FT College

Laser FT BasSkls

Laser FT NoAssess

Seeker PT College

Seeker PT BasSkls

Seeker PT NoAssess

Explorer

Career Builder FT

Career Builder PT

Skills Builder


35/49

Student ed goal groups:

Look and behave differently by

Age Gender

Race-ethnicity Day vs. evening

Early Decision

Financial Aid

Pct assessed in Math Math assessment levels

Types of major


36/49

Student ed goal groups:

Taking differences into account How do we know what is the reasonable

time it will take each group to achieve their

educational goal? First, answer challenge #1 to meeting the

standards outcomes

Connect long term outcomes with short termobjectives by setting milestones


37/49

Student ed goal groups

progress at different rates Early Engagement

Enrolled in basic skills

Momentum

Completed basic skills Persistence

Progress Completed College Math or English

Earned Transferable units

Earned CTE units

Completion Earned certificates or degrees

Transfer ready / transferred

Employed


38/49

Student ed goal groups

allow us to: Get beyond one number for each

outcome.

10 student groups Measure progress along the way.

Progress milestones

Determine reasonable time to milestones

and completion for each group based onpast cohorts

Tailor college initiatives for each group andsee effect of initiatives.


39/49

Strengthening Student Success Conference 2013

Elaine Kuo, Ph.D.Andrew LaManque, Ph.D.

Setting ACCJC Institutional

Standards at Foothill College


40/49

Legal Foundation Summary

The requirement for Institutional Standards flow from

Federal Regulations

The Regulations are not new (2009) but the

Commission has stated that the Federal interpretationchanged last year.

The new interpretation was included in evaluation team

training in Spring 2013.

40


41/49

The Accreditation Standards Process

Colleges setInstitutionalStandards

ACCJC collectsannual data onInstitutionalStandards

Visiting teamsevaluate whetherStandards are met

and reasonable

Recommendationsto meet the

standards or makethem reasonable?

41


42/49

The Process

Accreditation team

Shared governance groups

Planning and ResourceCouncil (PaRC)

Academic Senate

Classified Senate

Imbed in college planning,

program review

The Data

Most recent term or year

Fall 2012 or 2011-12

Longitudinal2007-08 to 2011-12

Disaggregation

Ethnicity

ProgramSources

CCCCO data mart

FHDA IR&P

42

Foothill Approach


43/49

The Data: Setting the Standards

Completion based on passingcourse with A, B, C, or P grade;includes all courses offered in Fallterm

Table 1. Student Course Completion Rate

Fall 2012 75%

Five-Year Average 76%

Five-Year Average by Ethnicity-Low 61%

Five-Year Average by Program-Low 60%

Institutional Standard Recommendation 55%

Source: CCCCO Data Mart, FHDA IR&P

Fall rates: 2011: 74%

2010: 76%

2009: 77%

2008: 79%43


44/49

The Process: Shared Governance Review

DiscussmethodologywithAccreditationTeam

Presentation anddiscussion atmain shared

governance body

Methodology andrecommendationsaccepted by mainshared governancebody and collegepresident

44

Institutional-set standards

scheduled for annual review

Key players include:

ALO, VPI, IR,

Academic Senate

President, Classified

Senate President


45/49

The Process: Program Review

Shared Governance

Integrated Planning& Budget Taskforce

Program ReviewTemplates

Student equity andinstitutional

standards section

Program ReviewCommittee

45

Prompts include:

Comment on your programs course success data, including any differences in completionrates by student demographics as well as efforts to address these differences.

Data sources: Program review data sheets, CCCCO data mart


46/49

Institution and Program Standards

For discussion:

How should a department course success /

completion rate reported during program reviewrelate to the institution standard?

If the department is below the institutionalstandard should there be a plan for getting it up

to the institutional standard?

46


47/49

Shaping the Dialogue

Three Roles, One Product

Three Roles: Scrivener

Time Keeper

Facilitator

Product: Summary Sheet

Notes

Three takeaways from the discussion

One Muddiest Point


48/49

Shaping the Dialogue

How were the institutional set standards developed at

your institution?

How might these standards be integrated with your

colleges decision-making and planning processes?

How would you justify the reasonablenessof the

institutional set standards in the context of your mission?


49/49

Wrap Up:What Have We Learned?

Discover some best practices in setting college-wide

standards for achievement.

Brainstorm ways to integrate the standards into planning

and decision-making processes at your institution.

Evaluate the reasonableness of your colleges institution-

set standards.

Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality

Documents

Transcript of Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality