www.emmaus.edu

MAKING THE MOST OF YOUR SELF-

STUDY PROCESSDr. Lisa Beatty

ABHE ANNUAL MEETINGFEBRUARY 21-23, 2018

www.emmaus.edu

How do you eat an elephant?

One bite at a timeNo elephants were harmed in the development of this presentation.

www.emmaus.edu

The Hallmarks of an Effective Self-Study Report

An explicit evaluation of the organization in light of the Comprehensive Integrated Standards and related Essential Elements

Use of information and data to create evidence to support the organization’s self-evaluation

Tone and content characterized by honest evaluation, not public relations

Developing an Evidence-Based, Evaluative Self-Study

Getting Started• Set Goals• Self-Study Timeline• Committee Structure

Self-Study Process• Conceptual Framework• Committee Culture• Ensuring Effective Self-

Study

Required Documents• Compliance Document• Assessment Document• Planning Document

Getting Started

www.emmaus.edu

Goals for Self-StudyWhat does the institution hope to achieve through the process?

Example: 2015 Self-Study Goals1. Reaffirm accreditation status with the Association for Biblical Higher

Education.2. Evaluate institutional progress on recommendations from the 2006

ABHE Team Report.3. Identify opportunities for institutional improvement.4. Assist faculty and staff in developing a global view of the institution.

www.emmaus.edu

Self-Study TimelineConsiderations…• Ensure that data evidence is available when needed.• Build in time to understand and interpret the standards.• Provide ample time for real research and careful writing.• Be sure to include time for reflection on findings.

www.emmaus.edu

Self-Study TimelinePROJECT DEADLINESelf-study process planning March/April 2014

Establish self-study committee May 2014

Project kick-off May 2014

Orientation to ABHE standards Summer 2014

Collection of supporting documents Fall 2014

Self-study research Fall 2014

Self-study writing and initial edits Spring 2015

Review and revision Summer 2015

Development of online resource site Summer 2015

Self-study review across the institution September/October 2015

Final edits and publication November 2015

Submission to ABHE December 1, 2015

Meetings ScheduleDate TaskMonday, August 18, 3:00-4:00pm Orientation/Timeline reviewThursday, September 11, 3:30-5:00pm Review Standards/Essential ElementsThursday, September 25, 3:30-5:00pm Review Standards/Essential ElementsThursday, October 9, 3:30-5:00pm Review Standards/Essential ElementsTuesday, October 21, 3:30-5:00pm Review document review and interview plans

Thursday, November 6, 3:30-5:00pm Review Standard 9 draftThursday, November 20, 3:30-5:00pm Extra TimeThursday, December 4, 3:30-5:00pm Review Standards 7 & 8 draftsThursday, January 15, 3:30-5:00pm Review Standards 1 & 3 draftsThursday, January 29, 3:30-5:00pm Review Standard 10 draftThursday, February 12, 3:30-5:00pm Review Standard 11 draftThursday, February 26, 3:30-5:00pm Review Standard 6 draftThursday, March 19, 3:30-5:00pm Review Standards 4 & 5 draftsThursday, April 9, 3:30-5:00pm Review Standard 2 draftThursday, April 30, 3:30-5:00pm Extra TimeThursday, September 24, 3:30-5:00pm Review second draftsThursday, October 15, 3:30-5:00pm Review second draftsThursday, November 19, 3:30-5:00pm Review final draft w/ Cabinet

www.emmaus.edu

Committee StructureConsiderations…To facilitate explicit evaluation, some committee members should

have comprehensive knowledge of the areas being assessed.Benefits of this approach

– Saves time in understanding data– Helps keep the committee from making naïve evaluative

judgments based on incomplete understanding of relevant issues

www.emmaus.edu

Committee StructureTo facilitate honest evaluation, some committee members should

have roles outside of those areas being assessed.Benefits of this approach– Provides fresh perspectives– Limits “group-think”– Keeps everyone honest

www.emmaus.edu

Self-Study CommitteeName ResponsibilityVice President Academic Affairs ChairAssociate Dean, Gen Ed and Assessment

Standards 1, 3, 4 & 5Intercultural Studies Department Chair

Bible Department Chair Standards 2a & 11Counseling Faculty MemberTeacher Education Department Chair Standards 2b & 6Student Accounts Manager/Staff AccountantCounseling Department Chair

Standards 7 & 8Librarian

Director, Emmaus Distance LearningStandards 9 & 10

Teacher Education Faculty MemberMath/Science Faculty Member Technology/Statistical SupportAdministrative Assistant Supporting DocumentsEnglish Faculty Member Research/EditingStudent Student Input

Self-Study Process

www.emmaus.edu

What questions are we trying to answer?

One-dimensional compliance question…Are we meeting the ABHE Standards?

Multi-dimensional evaluative questions…To what extent or with what level of quality are we meeting the ABHE

standards?What evidence supports our findings?

www.emmaus.edu

Conceptual Framework

We defined…• Telling as simple description or unsubstantiated evaluation, and• Showing as relevant description, explicit evaluation, and the

necessary evidence to support findings.

Example #1: Student DiversityThe Enrollment Department demonstrates the institution’s commitment to diversity by proactively recruiting at venues with the potential to reach ethnic minority students, including Hispanic college fairs, African-American college fairs, and Indian Brethren conferences. The college promotes ethnic diversity by offering a $2000 Student of Color Scholarship to qualified applicants. Additionally, Emmaus actively recruits international students, thus increasing diversity of national origin. The College promotes this diversity through International Student Scholarships ($6000) and International Christian School Scholarships ($1000). These proactive strategies have led to increasing levels of diversity over the past five years:

Race/Ethnicity 2011 2012 2013 2014 2015 White 85% 83% 75% 75% 73% Hispanic 6% 6.7% 10.4% 5.3% 6.3% Black 1.2% 2.4% 2% 5.3% 5.5% Asian 2% 1.9% 1.5% 0.8% 1.6% Hawaiian/Other Pacific Islander - - 1% 0.8% 0.7% American Indian 0.4% 0.4% 1% 0.8% 1.6% Two or more races - 0.9% 1% 3% 2.8% Other 0.8% 0.9% 1.5% 0.8% 3.2% Not Provided 2.8% 1.4% 3.1% 4.4% 5.2%

Source: Registrar’s reports

The Emmaus student body is significantly more diverse than the general population in Dubuque County and more ethnically diverse than other local four-year private colleges and universities, except for the University of Dubuque:

Fall 2014 Emmaus Bible

College

Dubuque County

Clarke University

Loras College

University of Dubuque

White 74.5% 92.3% 86.2% 82.6% 67.4% Hispanic 5.6% 2.1% 4.4% 6.0% 7.7% Black 5.6% 1.6% 3.3% 2.6% 11.9% Asian 1.4% 1.2% 0.7% 0.8% 2.0% Hawaiian/Other Pacific Islander 0.9% 0.0% 0.0% 0.1% 0.1% American Indian 0.9% 0.0% 0.1% 0.1% 0.5% Two or more races 3.2% 2.8% 1.7% 1.5% 3.4% Race Ethnicity Unknown 7.4% 2.3% 4.5% 5.9% Non-Resident Alien 0.5% 1.3% 1.9% 1.0%

Source: National Center for Educational Statistics; Demographics Starter Report for Dubuque County, IA, Cubit Planning, Inc.

Example #1: Student Diversity

Employee turnover in the past three years has created some obstacles to success. For example, due to the high turnover rate in the Enrollment Department, the Enrollment Marketing/Recruitment Plan has not been formally updated since 2013 (the Enrollment team has continued to implement changes and improvements in the enrollment cycle). Additionally, the rapid turnover rate diverted some attention and resources from direct recruitment efforts to training and supporting new employees. While difficult to quantify, these challenges have had a negative impact on enrollment success.

The college administration is confident in the personnel now serving in the area of enrollment. In August 2015, the College named Laurel Rasmussen, Director of Enrollment (effective October 1, 2015). Ms. Rasmussen served previously as Emmaus registrar and admissions director, and she brings significant experience from her roles at other higher education institutions including Wheaton College, University of North Carolina Charlotte, and Clarke University. The College is confident that Ms. Rasmussen will provide the leadership and stability needed for consistent enrollment growth.

Example #2: Enrollment Personnel Turnover

www.emmaus.edu

Committee CultureEstablish a committee culture that values…• Questions• Openness• Truth• Research• Gracious interaction• A global perspective

www.emmaus.edu

Committee CultureLessons learned in establishing a positive culture…• Set ground rules.• Reference institutional core values related to the evaluative

process.• Take the time necessary to orient participants to the process.• Invest in training.• Articulate purposes and procedures over and over and over… • Empower committees to gather evidence.• Never criticize individuals.

www.emmaus.edu

Ensuring Effective Self-StudyEstablish checkpoints to assess for evaluation and supporting evidence.

Initial chapter drafts– Identify strengths and challenges for each Comprehensive

Integrated Standard.– Cite sources for all evidence provided.– Connect identified strengths and challenges to the body of the

text.– Articulate recommendations for all challenges.

www.emmaus.edu

Ensuring Effective Self-StudyDuring the editing process– Share the document across the institution to corroborate

evaluation and discover unconsidered evidence.– Identify descriptive and evaluative holes.– Connect evidence to all evaluative statements.

Before the final edit– Communicate findings with the institution’s administrative team.– Discuss identified strengths, challenges, and recommendations.– Finalize action plans.

Required Documents

www.emmaus.edu

Required DocumentsCompliance Document

– Report of findings from your self-study process– See Self-Study Guide, Appendix A, for suggested outline

Assessment Document– Comprehensive Plan for Assessment of Student Learning

and Institutional Effectiveness– Plan should include (1) intended outcomes, (2)

achievement benchmarks, (3) assessment tools– Reporting of results should include (1) data summary, (2)

data analysis, and (3) recommendations for changes or improvements

www.emmaus.edu

Required DocumentsPlanning Document

– Institutional plans designed to address issues identified through assessment or self-study processes

– Should address short-term and longer-term plans– Should be informed by mission and goals with particular

attention to institutional vision (where do we want to be in 5 years, 10 years…?)

www.emmaus.edu

www.emmaus.edu

Accreditation Research “Another useful study might examine the constraining and enabling factors that influence accreditation outcomes and effects. It appears that the institutional motivation for pursuing accreditation may function as such a factor. Institutions pursuing accreditation for the purpose of quality improvement or increased credibility may experience more positive change and fewer negative effects than institutions that are motivated primarily by access to federal funding.”

CATFISH AND GOLDFISH IN THE SAME BOWL: PERCEIVED OUTCOMES AND EFFECTS OF ACCREDITATION AT THE INSTITUTIONAL LEVEL, Dr. Lisa L. Beatty

www.emmaus.edu

Final ThoughtsEngage in the work for the purpose of improving your institution.Engage in the work for the glory of God.

And whatever you do, do it heartily, as to the Lord and not to men, knowing that from the Lord you will receive the reward of the inheritance for you serve the Lord Christ.

Colossians 3:23, 24

www.emmaus.edu

https://www.abhe.org/accreditation/accreditation-documents/

Dr. Lisa L. Beatty / Vice President for Academic Affairslbeatty@emmaus.edu / 563-588-8000 x1103

Tips To Organize

The Final Product of your Self-Study

Start Knowing The End

�Read the Self-Study Guide�Have a plan for the final product

Final 5 Documents:

1. Compliance Document (with Regulatory Requirements Evaluation)

2. Assessment Plan3. Planning Document4. Statistical Abstract5. Exhibits

EXHIBITS

�Only want excerpts or specific pages�Don’t send 100+ exhibits �Can send handbooks, catalog, etc. as stand-

alone documents�Send files 25M or less�Number exhibits

EXHIBITS

�No internet links to documentation or repository

�Utilize links to website �Only send what is asked for�Refrain from scanning documents�Combine files into PDF files

OPTION 3

�We want your Self-Study documents!

�Least favorite option

�Name exhibits with numbering system

�Send the documents as best as you can

OPTION 2

oCopy/paste excerpts into Word file

o Scan 300dpi and PDF

o Save As: PDF

OPTION 2

�Table of Contents

�Name files:

• 2018.SS Institution Name (FL) Exhibits 1-20

• 2018.SS Institution Name (FL) Exhibits 21-36

OPTION 1

Adobe Acrobat DC (about $150/year)

OPTION 1

�Ease of use

�Combines any type of documents

�Auto bookmarks

NAME EACH FILE:

�2018.SS Institution Name (FL) Planning Doc

�2018.SS Institution Name (FL) Compliance

�2018.SS Institution Name (FL) Statistical Abstract

�2018.SS Institution Name (FL) Assessment

�2018.SS Institution Name (FL) Exhibits

OR2018.SS Institution Name (FL) Exhibits 1-20

2018.SS Institution Name (FL) Exhibits 21-30

QUESTIONS?