Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D....
-
Upload
harry-gilbert -
Category
Documents
-
view
218 -
download
1
Transcript of Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D....
![Page 1: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/1.jpg)
Learning Outcomes Made EasyUsing the Best Tools
Jeffrey D. Keith, Ph.D.
J. Kelly Flanagan, Ph.D.
Russell T. Osguthorpe, Ph.D.
Danny R. Olsen, Ph.D.
Tom Mallory
Jeff Fox, Ph.D.
Brigham Young University, Provo UT
![Page 2: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/2.jpg)
2
![Page 3: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/3.jpg)
11 Academic Colleges 56 Academic Departments 400 Degree Programs 1,600+ Faculty 8,000+ Courses
33,000+ Students
3
Carnegie Classification: Research University (High Research Activity)
Northwest Commission on Colleges and Universities
30 Specialized Accrediting Organizations
BRIGHAM YOUNG UNIVERSITY
![Page 4: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/4.jpg)
4
U.S. Regional Accreditors
![Page 5: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/5.jpg)
5
Assessment Charge
Publish expected learning outcomes for all of their courses and programs
Provide evidence that the expected learning outcomes are realized by students
Demonstrate how such data collection and analyses leads to continuous improvement of student learning, the curriculum and the university
Colleges and universities across the country are being asked to:
![Page 6: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/6.jpg)
6
Why “Student Learning Outcomes”?
1. Ethics: Assessment should ultimately (first and always) be about student learning Student major selection Aligning institutional, department goals Student “take-aways”
2. Pressure: Stakeholders Accrediting entities Government agencies Employers
![Page 7: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/7.jpg)
7
What are Student Learning Outcomes?1. Definition: Statements defining what students
should know, understand and/or be able to do as a result of their learning experiences in a degree program Cognitive (knowledge) Behavioral (skills) Affective (attitudes)
2. Assessment: What counts as evidence that a given level of competence has been achieved.
3. Clear and assessable statements of the “essential and enduring knowledge, abilities, and attitudes or dispositions” that enable a learner to apply what he or she has learned to situations encountered in the real world.
![Page 8: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/8.jpg)
Direct & Indirect EvidenceDIRECT MEASURES Imbedded performance
assessment mapped to learning objectives
Course and homework assignments
Exams, quizzes Standardized tests Term papers, reports Research projects Capstone projects, senior
theses, exhibits or performances
Pass rates, scores on licensure, certification, area tests
Rubric scores
INDIRECT MEASURES Course evaluations Alumni questionnaire data Senior survey data National Survey of Student
Engagement data Employer surveys Graduate school placement
rates Department exit interviews Focus group interviews with
students, faculty, etc. Job placement Registration or course
enrollment information8
![Page 9: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/9.jpg)
9
Evidences of Learning on Pathways to Graduation
![Page 10: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/10.jpg)
10
University Goal AwarenessTo Publish Course Level Learning Outcomes by August 31, 2011
Law
Relig
ious
Ed
Man
agem
ent
Educa
tion
Engin
eerin
g
Fine
Arts
Socia
l Scie
nces
Physic
al S
ci
Life
Scie
nces
Human
ities
Nursin
g
Unive
rsity
0%
20%
40%
60%
80%
100%
22%
40% 44%60% 61% 62% 68% 71% 72% 73%
80%
63%
By College as of June 1
![Page 11: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/11.jpg)
11
Published, Mapped Course Outcomes
October 2010
April 18 June 1 July 21 Aug 31 GOAL
0%
20%
40%
60%
80%
100%
9%22%
29% 32%
100%
0.0830000000000001
% Courses with Published Outcomes% Course Outcomes Mapped to Program Outcomes
2011
![Page 12: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/12.jpg)
12
Obstacles & Challenges Culture of assessment disparity across campus. Quality of stated “Expected Learning Outcomes”. Obtaining faculty consensus within departments. Faculty engagement in the process as part of the
status quo without any extra compensation. Moving from a faculty-centric to a student centric
language of stated outcomes. Appropriate channels of evidence—direct, indirect. Demonstrating closing the assessment loop Applied Technology—cost, time, scalability, usability
![Page 13: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/13.jpg)
Design Goals• Ease of use
Editing, viewing, and linking of program and course outcomes must be easy
• Leverage existing IT web services Security, identity, course information
• Data accessibility Existing university applications
Course catalog, registration, syllabus builder
![Page 14: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/14.jpg)
14
Achieving Design Goals
• Ease of use Usability
• Leverage existing OIT web services University IT department adopted SOA SOA registry
• Data accessibility Adopted a service model
Common interfaces for interoperability Service registered in SOA registry
![Page 15: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/15.jpg)
Learning Outcomes
Service
Learning Outcomes Web site
Exams
Syllabus Builder
Discussions
CourseContent
Program Assessment
Tool
CourseRegistration
CourseCatalog
Course Assessment Tool
CourseEvaluations
![Page 16: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/16.jpg)
Challenges
• Course Learning Outcome Outliers Certificates Emphases GE Service Course
• Building a Learning Outcomes Assessment tool that meets disparate needs.
![Page 17: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/17.jpg)
![Page 18: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/18.jpg)
Faculty Involvementhttp://www.youtube.com/byuctl#p/u/0/-oy6ztc1kq0
![Page 19: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/19.jpg)
Course Catalog
![Page 20: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/20.jpg)
Course Evaluations
![Page 21: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/21.jpg)
Program Assessment Tool
Just Beginning Design…
![Page 22: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/22.jpg)
Course Content
![Page 23: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/23.jpg)
Discussions
![Page 24: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/24.jpg)
Exams
![Page 25: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/25.jpg)
Syllabus Builder
![Page 26: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/26.jpg)
26
![Page 27: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/27.jpg)
LO Website 1
![Page 28: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/28.jpg)
![Page 29: Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom.](https://reader036.fdocuments.net/reader036/viewer/2022062422/56649eb45503460f94bbb6ee/html5/thumbnails/29.jpg)
29