How Do CCCU Adult Programs Measure Up?

Dr. Cynthia Tweedell, CCCU Center for Research in Adult Learning

Dr. Mary Moretto, Goshen CollegeDr. Shirley Roddy, Mid-America Christian University

Dr. George Howell, Indiana Wesleyan University

• Partnership between Indiana Wesleyan University and Council for Christian Colleges and Universities.

What is the Center for Research in Adult Learning?

Richard Ellis– John Brown University, ARDon Finn– Regent University, VASteve Holtrop– Huntington University, INLori Jass– Bethel University, MNAudrey Kelleher– Belhaven University, MSToni Pauls– Warner Pacific University, ORShirley Roddy– Mid-America Christian Univ., OKGeorge Howell– Indiana Wesleyan Univ., INAnita Underwood– Nyack College, NYCameron Wold– Colorado Christian Univ., CODeborah Wright– Montreat College, NCMimi Barnard– CCCUCynthia Tweedell– Center for Research

Steering Committee

• Learning outcomes assessment for adults• Benchmarking• Retention of adult students• Spiritual transformation of adults• Best practices in online learning• Teaching diverse learners• Alumni surveys

Research Agenda

• Sponsor at least one conference per year.– May 10-12, 2010 In Cincinnati

• Publish at least one monograph per year.– Integration of Faith and Learning for Adults coming in May

• Provide benchmarking data on adult education.• Assist in student learning outcomes assessment• Serve as a resource for research in adult programs.• Coordinate projects at the request of institutions

www.indwes.edu/cral

Strategic Goals

• Adult Student Learning Outcomes– 3rd year of project– About 10 schools participating– Common writing prompt– Beginning and graduating students– 3 faculty from each school as scorers– Critical Thinking, Written Communication, Christian

Worldview– How does your school compare to other adult programs?

Benchmarking from the CCCU Center for Research in Adult Learning

Scoring Rubric

CriticalThinking

Christian World View

Communication

5 Identifies issuesChallenges assumptionsThorough analysis

Clearly demonstrates an understanding of a Christian perspective. (Frequent references to biblical principles)

Free from distracting spelling, punctuation, and grammatical errors.Very well organizedMeaning is clear

4 General connections, analysis and identification of issues

Frequently refers to a Christian perspective. Student has a reasonable understanding of Christian perspective. (Some references to biblical principles.)

Few spelling, punctuation and grammatical errors.Fairly well organizedA few places where meaning is a little unclear.

3 Some analysisVague identification of issue

Makes some mention of a Christian perspective. Student indicates some understanding of a Christian perspective. (Biblical principles referred to somewhat.)

Most spelling, punctuation, and grammar are correct, though some errors remain.Organization may detract from meaning. Some places unclear.

2 Incomplete analysisFragmented understanding of issue

Demonstrates little understanding of a Christian perspective. (Biblical principles hardly or not mentioned.)

Spelling, punctuation, and grammatical errors are distracting.Organization and meaning unclear.

1 Vague analysisBasic lack of understanding

Makes no reference to a Christian perspective. Student does not appear to have an understanding of a Christian perspective. (Biblical principles not mentioned.)

Many spelling, punctuation and grammatical errors, making reader unable to follow ideas. Lacks organization.Meaning is very unclear.

Score:

2008 Results: 6 Colleges

N MeanStd. Dev

Critical Thinking Pre 109 *3.27 0.80

Post 127 *3.49 0.72Written Communication Pre 109 *3.06 0.83

Post 126 *3.36 0.70Christian Worldview Pre 109 1.92 0.97

Post 126 1.83 0.92

2009 Results: 5 Colleges

N MeanStd. Dev

Critical Thinking Pre 107 *3.10 0.61

Post 84 *3.34 0.73Written Communication Pre 107 1.77 0.80

Post 84 1.97 0.82Christian Worldview Pre 107 *2.82 0.77

Post 84 *3.49 0.75

What Do Colleges Do With The Results?TWO CASE STUDIESMID-AMERICA CHRISTIANINDIANA WESLEYAN

Mid-AmericaCHRISTIANUniversity

Response To Data

• Review MACU’s presentation of data in comparison to other participating institutions

• Consider that the results could be reflective of the reality.

• Initiate dialogue with the College of Adult and Graduate Studies Chairs.

MACU’s presentation of data

• Determine if MACU’s data indeed matched with the presented data from the other participating institutions in time, setting and resources available to the student during their response.

• Consider the impact of the sample size and MACU’s method of randomization for possible influence on results

Conduct and Participate in Subsequent Studies

• Attempt to replicate the study by participating in subsequent multi-institutional studies

• Compare ourselves to ourselves--Replicate our own institution’s study to determine if we get similar results.

IN THE MEANTIME…

• Considering that the data may be reflective of student achievement in the areas measured, it does not seem prudent to wait until further studies have been conducted.

• Immediately put into place measures that will increase students’ level of achievement in the areas reflective of need

What we have already done

• Initiated discourse with chairs, using the research results as a tool

• Established remediation courses in math and English

• Standardized our own method of data gathering and compared ourselves to ourselves!

School of Business & LeadershipCollege of Adult &

Professional Studies

Assessment ProcessQuestions for School of Business & Leadership Program Directors & Faculty (Graduate, Bachelor and Associate)• Critical thinking

– How can we build on existing threads? • Christian worldview

– How can we increase the Christian Worldview of our students?• Written communication

– How can we improve our students’ written skills?• The CCCU Adult Student Learning Outcomes Project was run

in parallel with College and School initiatives related to the outcomes of this project

Assessment Process: Critical Thinking

Critical thinking: How can we build on existing threads? • Review by faculty of critical thinking resources

– Selection narrowed to five textbooks– Two textbooks selected by the School of Business

& Leadership• Ongoing faculty development activities on

critical thinking

Assessment Process: Critical Thinking

• Faculty and Academic Leaders selected the following critical thinking textbooks to reinforce the critical thinking thread in programs:– Undergraduate Business Programs Browne, M. &

Keeley, S. (2010). Asking the Right Questions: A Critical Guide to Critical Thinking (9th Ed.). Upper Saddle River, NJ: Pearson/Prentice Hall.

– Graduate Business Programs Paul, R. & Elder, L. (2006). Critical Thinking: Learn the Tools the Best Thinkers Use. Upper Saddle River, NJ: Pearson/Prentice Hall.

Assessment Process: Critical Thinking

• Critical thinking resource for our course writers– Nosich, G.M. (2009). Learning to think things

through: A guide to critical thinking across the curriculum (3rd Ed). Upper Saddle River, NJ: Pearson Prentice Hall

Assessment Process: Christian Worldview

Christian Worldview: How can we increase the Christian Worldview of our students?• Ongoing evaluation of the integration of faith in curriculum

– Faculty Devotions and Biblical Principles in Business– Curriculum threads that reinforce knowledge, skills and

disposition (be, know and do) of students’ Christian Worldview – Develop character of students by a Christ-centered

example/role model • Faculty member’s expression and role modeling of Judeo Christian

values in the classroom

• Faculty development activities to improve faith integration

Assessment Process: Written Communication

Written communication: How can we improve our students’ written communication skills?• School of Business & Leadership Faculty committees developed

scoring guides for written work to better explain expectations to students on written work and to help faculty be more objective in their assessment.– Met objectives/requirements of assignment (35%)– Critical thinking (35%)– APA formatting (15%)– Mechanics (Grammar, Spelling, Word choice, Punctuation and Sentence

structure ) (15%)• Ongoing faculty development activities on grading written work

Assessment Process: Written Communication

Written communication: How can we improve our students’ written communication skills?

– Faculty task force examined our Liberal Arts Requirements (General Education courses) and made recommendation to revise the number of credits required in English /Composition/Speech/Literature

• Going to College’s Academic Affairs Committee in first quarter of 2010

– Writing lab – MAPP (Make a Paper/Project)– Online Student Resources with Prentice Hall Reference

Guide