2004 Southern Business Administration Association Meeting, Myrtle Beach, South Carolina
The UCF College of Business Approach to Assessing Student Learning
Taylor Ellis, Ph.D. Patrick Moskal, Ph.D.Associate Dean Director, Testing & Evaluation
University of Central Florida
Presentation Outline1. UCF background.
2. Overview of the assessment process in the College of Business Administration.
3. Development of student learning outcomes.
4. Description and role of our Testing Lab in the assessment process.
The UCF College of Business Approach to Assessing Student Learning
UCF Background Large metropolitan university with over 44,000 students. College of Business Administration (CBA) offers 7
undergraduate, 10 Masters, and 5 Doctoral degree programs.
~140 full-time and 60 part-time faculty in CBA. ~8,000 students enrolled in CBA each year.
The UCF College of Business Approach to Assessing Student Learning
Issues Faced in UCF CBA Number of students. Required courses in core. Pre-requisites in majors. Computer Testing Lab.
The UCF College of Business Approach to Assessing Student Learning
CBA Assessment ProcessUniversityAssessment
Committee (UAC)
CBA DepartmentalReview Committee
(Dept. Chairs)
CBA Teaching &Learning Committees
(Dept. level)Director of Testing &
Evaluation
Develop learningobjectives,
outcomes, andperformance
measures
Supports T & L committees develop SLO’s
Associate Dean ofUndergraduate
Programs
CBA representative to UAC (UAC Chair)CBA’s DRC Chair
The UCF College of Business Approach to Assessing Student Learning
CBA Assessment Process (continued)
The UCF College of Business Approach to Assessing Student Learning
Director of Testing &Evaluation
Develop learningobjectives,
outcomes, andperformance
measures(department level)
Student Records(University Database)
CBA AccreditationDatabase
Data Analyses
Reports(to CBA, UCF,Accreditating
Agencies, and Stateof Florida)
Paper-basedassessments (e.g.,
written exams,papers,
presentations,portfolios,
standardized exams
Computer-basedassessment in
Testing Lab
Supports T & L committees develop SLO’s
Associate Dean ofUndergraduate
Programs
Assessment Process
CBA Departmental Review Committee (DRC) DRC consists of CBA Department Chairs. DRC Chair participates on the UAC and reports CBA
assessment results. Supported by Director of Testing and Evaluation.
The UCF College of Business Approach to Assessing Student Learning
Assessment Process
CBA Teaching and Learning Committees Selected for each degree program area. Establish student learning outcomes (SLOs) and ensure
data collection. Participate in an annual review of assessment results,
and plans for upcoming year.
The UCF College of Business Approach to Assessing Student Learning
Assessment Process Each degree program will develop 4-10 general
learning objectives (expectations). Each objective will be evaluated by multiple
outcome measures (exam questions or assessment types).
Summative scales will be developed to measure most learning outcomes.
Allows for evaluation in multiple classes. Allows for pinpointing specific weak areas.
The UCF College of Business Approach to Assessing Student Learning
Assessment Process
Degree Program Learning Expectations
Expectation 1 Expectation 2 Expectation 3, etc.
Class 1 Class 2 Class 3, etc.
Affective 1 Cognitive 1 Behavior 1
Questions & Beliefs 1-? Questions 1-? Questions & Projects 1-?ProjectsCollaboration score
The UCF College of Business Approach to Assessing Student Learning
Assessment GoalsEstablish student learning outcomes for each degree
program, based on the CBA Mission.Acquire data that provide direction for program
improvement and meet both AACSB and regional accreditation requirements.
Develop comprehensive data collection and reporting system.
Collect multiple measures over time (data triangulation).Utilize existing data where possible. Cause minimal intrusion on normal faculty workload.
The UCF College of Business Approach to Assessing Student Learning
Why Do We Need Student Learning Outcomes?
Provide feedback to students about their progress. Communicate instructional goals to stakeholders (e.g.,
trustees, community, university administration, faculty, accrediting agencies, parents).
Provide the means for assessing student learning.
The UCF College of Business Approach to Assessing Student Learning
Learning Objectives
Begin by developing a list of goals and objectives, such as the following “essential competencies” for all UCF Business graduates.
BE2010 Competencies:TeamworkCommunicationCreative ThinkingAdapting to changeDiversityEthics
The UCF College of Business Approach to Assessing Student Learning
Learning Objectives From our goals, we develop specific learning objectives. An objective is defined as a quantifiable and/or observable achievement accomplished under specifiable conditions.
How do we develop objectives? Start by answering basic questions like:
What do we want our students to know when they finish our course or program?
What should our students be able to do when they finish our course or program?
The UCF College of Business Approach to Assessing Student Learning
Objectives Vs. Outcomes In general, an objective is what we hope that we will
see, and an outcome is what we did see. Learning objectives define the outcomes of
instruction in clear terms. Outcomes must be both observable and measurable. All objectives/outcomes should include three primary
components: the action or behavior to be measured, conditions of learning, and standards of performance.
The UCF College of Business Approach to Assessing Student Learning
You Want SMART Objectives Specific (Clear and definable terms describing the
knowledge, skills, and attitudes)
Measurable (Reliable data that can be assessed in multiple ways)
Attainable (Feasible to be accurate; need not be perfect)
Realistic (Expected standards must be practical)
Time-bound (When will results will be observed?)
The UCF College of Business Approach to Assessing Student Learning
Types and levels of student learning outcomes Cognitive: intellectual skills.
Knowledge, comprehension, application, analysis, synthesis, evaluation (from Bloom, 1956).
Affective: what a student believes (values). Behavioral: psychomotor skills.
The UCF College of Business Approach to Assessing Student Learning
General Knowledge and Skills Examples
Communication Organizational Quantitative Critical Thinking Interpersonal Skills and Awareness Group Dynamics Motivational and Leadership Ability to Learn and Train Others Ability to Manage Information Technology Global Awareness Understanding Issues of Diversity Ethical Awareness
The UCF College of Business Approach to Assessing Student Learning
CBA Assessment Types Course embedded assessments of individual and group
knowledge and skills (e.g., exams, projects). Program level exams (e.g., doctoral candidacy exams). Certification and standardized exams. Direct observation. Teamwork. Student portfolios. Internships and service learning. Student records. College, AACSB, and employer surveys. Placement/employer data.
The UCF College of Business Approach to Assessing Student Learning
CBA AccreditationDatabase
Data Analyses
Student Records(Peoplesoft)
Exam Records(InQsit & WebCT):
selected test questions,student responses,
course and student info,applicable studentlearning outcomes
Other Assessment Records:assessment type (e.g.,
portfolio, oral presentation,written communication,collaborative project),
student score onassessment) course andstudent info, applicable
student learning outcomes
Learning OutcomeRecords:
student learning outcomes,outcome classification
(Bloom), department andcourse info
Reports
Faculty Informationcredentials, course syllabi
CBA Accreditation Database Development Plan
CBA Assessment Database Maintain results for each student. Prefer automated input from electronic testing. Allow input from additional sources. Allow data analyses. Create reports specific to each degree program
and for diverse stakeholders.
The UCF College of Business Approach to Assessing Student Learning
CBA Testing Laboratory
140 networked PCs Positioned to maintain exam security. Netsupport School Pro software to manage PCs
(e.g., monitor, block unapproved access). WebCT and InQsit1 packages available for exams. Online student exam scheduling.
1 Developed at Ball State University.
The UCF College of Business Approach to Assessing Student Learning
CBA Testing Laboratory
Over 7,500 exams given in fall 2003.
Over 12,500 exams given in spring 2004.
Goal is to automatically extract specified program evaluation and accreditation data from course exams.
The UCF College of Business Approach to Assessing Student Learning
Testing Software Features
InQsit: Easy to use and generate tests (Word, text files, Publisher’s test banks).
WebCT: Complete course management system.
Test Question Types: T/F, multiple choice, multiple response, short answer, matching, identification of objects on chart/photo (InQsit), limited essay grading (InQsit), ungraded essay, survey.
The UCF College of Business Approach to Assessing Student Learning
Testing Software Features (continued)
Random selection of questions from a question pool. Randomization of questions and answer choices. Adaptive scoring. Use of audio and video in tests. Start and stop times may be set by date and time. Analysis of Results: automatic student grading and item
analysis.
The UCF College of Business Approach to Assessing Student Learning
Testing Lab Components Under Development
Lab entrance will have student ID reader. Students swipe card or enter PID for computer verification.
Proctor software will allow the faculty/proctor PC to display:Student name and photo.Test the student is authorized to take.Computer assigned for exam.Seat reservation duration.
The UCF College of Business Approach to Assessing Student Learning
For more information, contact:
Patrick J. Moskal, Ph.D.University of Central Florida College of Business AdministrationUndergraduate Student Services, BA-II, Room 101GOrlando, Florida [email protected]
The UCF College of Business Approach to Assessing Student Learning
Top Related