Post on 30-Mar-2015
Second day sessions
First sessionSecond session Third Session
How to write LOs at Program and Course Levels
Relationships between Teaching Methods and Assessments Methods and LOs
KPIs in General and Application and Itegration with LOs
74
How to write learning outcomes for How to write learning outcomes for program levels and courses?program levels and courses?
Session 1Session 1
NCAAANCAAA
75
Where do L.O. come from?
Learning OutcomesLearning Outcomes flows out flows out from……from……
……students’ needs assessmentstudents’ needs assessment… … and employers needsand employers needs
(cause and effect chart)(cause and effect chart)76
EffectsEffects
Learning Outcomes Learning Outcomes
CauseCause
StudentStudent
NeedsNeeds
77
Where do L.O. come from?
Learning OutcomesLearning Outcomes flows out flows out from……from……
……the Mission Statementthe Mission StatementWhat must students do to demonstrate that What must students do to demonstrate that
the Institution/Program Mission is the Institution/Program Mission is accomplished?accomplished?
(cause and effect chart)(cause and effect chart)78
EffectsEffects
Program and CourseProgram and Course
Learning OutcomesLearning Outcomes
CauseCause
MissionMission
NeedsNeeds
79
Theory Into Practice5 Questions for Instructional Design5 Questions for Instructional Design
1.1. What do you want the student to be able What do you want the student to be able to do? (Outcome)to do? (Outcome)
2.2. What does the student need to know in What does the student need to know in order to do this well? (Curriculum)order to do this well? (Curriculum)
3.3. What activity will facilitate the learning? What activity will facilitate the learning? (Pedagogy)(Pedagogy)
4.4. How will the student demonstrate the How will the student demonstrate the learning? (Assessment)learning? (Assessment)
5.5. How will I know the student has done How will I know the student has done this well? (Criteria)this well? (Criteria)
80
Let’s Write a Learning Outcome
• We’re taking a friend desert camping We’re taking a friend desert camping for the first time (not roughing it too for the first time (not roughing it too much).much).
• What do they need to know?What do they need to know?
81
1. What do you want the student to be able to do?
This question asks you to This question asks you to develop the outcome.develop the outcome.
For Example: Student Student identifiesidentifies, , consultsconsults and and evaluatesevaluates reference books appropriate to reference books appropriate to the topic in order to locate the topic in order to locate background information and background information and statistics.statistics.
82
Learning Outcomes Formula
Verb Verb
oror
Action PhraseAction Phrase “In Order In Order To”To”
=Great Great
Learning Learning OutcomesOutcomes
What students need to know?““Student identifies, Student identifies, consults and evaluates consults and evaluates reference books reference books appropriate to the topic”appropriate to the topic”
Why do they Why do they need to know need to know this?this?““locate background locate background information and information and statistics.”statistics.”
In Order to
+
OrOr
Why?
83
Establish DraftEstablish Draft
Provide PrepareProvide Prepare
Tabulate WriteTabulate Write
Schedule UpdateSchedule Update
Audit ArticulateAudit Articulate
Align CollectAlign Collect
Construct GenerateConstruct Generate
ListList Produce Produce
Compile DocumentCompile Document
Demonstrate DevelopDemonstrate Develop
Suggested VerbsSuggested Verbs
84
Consider MaximizeConsider Maximize
MaintainMaintain Reflect Reflect
ContinueContinue Review Review
Ensure Enlarge Ensure Enlarge
Understand ExamineUnderstand Examine
Strengthen ExploreStrengthen Explore
Encourage DeepenEncourage Deepen
Some of these verbs can be used if tied to Some of these verbs can be used if tied to specific actions or quantificationspecific actions or quantification
Verbs Not To UseVerbs Not To Use
85
Example 1
Poor Learning Outcome:Poor Learning Outcome:
• Students will name the three types of rock in order to differentiate among the three.
Good Learning Outcome:
• Students will compare and contrast Students will compare and contrast the characteristics of the three types the characteristics of the three types of rocks in order to differentiate of rocks in order to differentiate among the three. among the three.
86
Example 2
Bad Learning Outcomes:Bad Learning Outcomes:• Discover that Al Fawo University offers a welcoming
and helpful environment which can fulfill their educational, cultural and social needs in order to recognize the university’s role in lifelong learning.
• Use OVID in order to access materials not available Use OVID in order to access materials not available at Al Fawo university Library.at Al Fawo university Library.
Good Learning Outcome:Good Learning Outcome:• Utilize retrieval services in order to obtain materials Utilize retrieval services in order to obtain materials
not owned by Al Fawo University Librarynot owned by Al Fawo University Library.87
Let’s Write a Learning OutcomeLet’s Write a Learning Outcome
• We’ll concentrate on how to build a We’ll concentrate on how to build a tenttent
• Why do we want our friend to be able Why do we want our friend to be able to properly build a tent?to properly build a tent?
88
Let’s Write a Learning OutcomeLet’s Write a Learning Outcome
• Now let’s write the learning Now let’s write the learning outcomeoutcome
• What is our verb (use the right What is our verb (use the right domain)?domain)?
• Why?Why?
89
Let’s Write an Assignment-Specific Learning Outcome
What does the student need to know?What does the student need to know?
• Why do they need to know this?Why do they need to know this?
See example and activity See example and activity 90
Student needs Learning outcomes
Student needs example: “Student needs Student needs example: “Student needs to learn how to fish in the ocean to to learn how to fish in the ocean to survive on the island.”survive on the island.”
1.1.What are some learning outcomes that What are some learning outcomes that fit this student’s needs?fit this student’s needs?
2.2.Create a student need for your Create a student need for your specialized course.specialized course.
3. Create five learning outcomes that 3. Create five learning outcomes that are directly based on this need. are directly based on this need. Bubble MapBubble Map9
1
Student Need
92
Relationship between teaching methods Relationship between teaching methods and assessment methods with learning and assessment methods with learning
outcomes.outcomes.
Session 2Session 2
NCAAANCAAA
93
Teaching Methods and LOs
• Focuses on learning outcomes in debates on teaching methods in higher education
• “Teaching methods” are not an end in themselves, they are a means to an end.
• They are the vehicle(s) we use to lead our students towards particular learning outcomes.
• We evaluate our teaching methods against the learning outcomes that we are seeking for our students.
94
Teaching Methods and LOs
• The first step in operationalizing it is to clarify the learning outcomes at which we are aiming.
• The second step involves developing a contingency approach to the choice of teaching methods whereby “fitness for purpose”
95
Teaching Methods and LOs• When selecting any teaching and learning method it
is important to ensure that the method will enable the students to achieve what are intended as learning outcomes.
• There are different kinds of methods available:– effective in building up subject knowledge– contribution to developing generic skills
96
Quality of Teaching
NCAAA Standard 4, paragraph 4.6:
• Teaching must be of high quality with appropriate strategies used for different categories of learning outcomes and student learning styles.
• Differentiated Instruction 97
Teaching Methods MappingTeaching Methods Mapping
Courses Codes
101102103104105106107108
TeachingMethods
Lecture
Small Groups
Discussion
Project
Activity
Debate
Research
Lab
Guest Expert
Demonstration98
Key for Learning Outcomes
Learning outcomes must be…. measurablemeasurable and meaningful to be assessed accurately. meaningful to be assessed accurately.
Who will know? Who will know? Student? Faculty? Student? Faculty?
How will I know? How will I know? Evidence Evidence
What evidence is needed? DemonstrateWhat evidence is needed? Demonstrate
Key is ASSESSMENTASSESSMENT
99
NCAAA: L.O. + AssessmentNCAAA: L.O. + Assessment
NCAAA Standard 4, paragraph 4.4:NCAAA Standard 4, paragraph 4.4:
Student assessment processes must be appropriate for the intended learning outcomes and effectively and fairly administered with independent verification of standards achieved.
100
Quality Assurance of Assessment Quality Assurance of Assessment
of Learning Outcomesof Learning Outcomes….….
May be achieved:• By direct observation – inspection of
assessments before they are set, and of samples of marked work
• Indirectly – by examining the specification of assessment processes
• Via feedback – from students, from employers, from external examiner, from professional bodies.
101
NCAAA Domains of Learning
Strategies
Knowledge Knowledge FactsFacts Concepts, theoriesConcepts, theories ProceduresProcedures
Cognitive SkillsCognitive Skills Apply skills when askedApply skills when asked Creative thinking and problem Creative thinking and problem solvingsolvingInterpersonal Skills and Interpersonal Skills and ResponsibilityResponsibilityResponsibility for own learningResponsibility for own learningGroup participation and leadershipGroup participation and leadershipAct responsibly-personal and Act responsibly-personal and professional situationsprofessional situationsEthical standards of behaviorEthical standards of behavior
Communication IT and Numerical Communication IT and Numerical SkillsSkills Oral and written communicationOral and written communication Use of ITUse of IT Basic math and statisticsBasic math and statistics
Psychomotor SkillsPsychomotor Skills
Assessment Strategies…Assessment Strategies…
102
Quality Assessment of L.O.sQuality Assessment of L.O.sRequiresRequires
• That a range of assessment methods is used, so as to cover different LOs and different learning styles
• That specifications for assessment, and criteria for grading, are transparent and appropriate to the level of the course within the NQF
• That second-marking or moderation is routinely used
• That there is clarity about the grounds on which students may appeal a mark, and that students are aware of this10
3
Quality Assessment of L.O.sQuality Assessment of L.O.sRequires….Requires….
• That deadlines for the return of marked work are published and adhered to
• That full and constructive feedback on assessments is provided to students, linked to the LOs and to the assessment criteria
• That guidelines for dealing with plagiarism, and with requests for time extensions and mitigating circumstances, are clear and are adhered to
104
Start at the EndStart at the End
Teaching strategy and methods depend on the assessment methods utilized. The assessments utilized depends on the learning outcome VERB that guides the assessment process to validate student learning and direct teaching methods.
Teaching Assessment Teaching Assessment LearningLearning Objective Objective Student Student
Methods Methods Methods Methods Outcomes Outcomes Content Content Needs Needs
105
AssessmentsAssessments
Teaching-Teaching-
MethodsMethods
LearningLearningContentContent
ObjectiveObjective
11stst Learning LearningOutcome Outcome 22ndnd LO LO
55thth LOLO
3rd LO
44thth LO LO
106
Introductory Assessment MapIntroductory Assessment Map
107
Intermediate Assessment MapIntermediate Assessment Map
108
Assessment MapAssessment Map
109
NCAAANCAAA
Key Performance Indicators,
Application and Integration with
Learning Outcomes
Session 3110
……a type of measure of performancea type of measure of performance
……a Key Success Indicatora Key Success Indicator (KSI) (KSI)…a measure of results and efficiency
Quantifiable performance measures used to define success and measure progress toward the achievement of goals.
What are KPIs???
111
Success IndicatorsSuccess Indicators
Bubble Map Group ActivityBubble Map Group Activity(think Quality Indicators)(think Quality Indicators)
……for a carfor a car……for a jobfor a job……for dinnerfor dinner……for a business tripfor a business trip……for a housefor a house……for a vacationfor a vacation
112
Good KPIs are Good KPIs are SMARTSMART• Specific – clear and well-defined• Measurable – show distance from
attainment• Agreed – shared by stakeholders• Realistic – fit available time, money,
etc• Time-based – set a specific time-scale
Sometimes ‘achievable’ is used, not ‘agreed’,but ‘achievable’ implies ‘realistic’,
so you might use ‘achievable’ and ‘relevant’113
FIRST……
An An intended learning outcome intended learning outcome must be must be selected and then a KPI may be chosen selected and then a KPI may be chosen which best reflects thewhich best reflects the intended student intended student performance performance ... or intended student intended student achievement achievement … or the student outcome… or the student outcome
114
KPIs are like….KPIs are like….
……like like yeastyeast when it permeates when it permeates bread and makes it rise…bread and makes it rise…
……like like saltsalt seasoning food, it seasoning food, it changes the entire flavor..changes the entire flavor..
……like like lightlight in the darkness.. in the darkness..
They create a context for actionThey create a context for action115
KPIs are not independent
KPIs do not stand alone or operate in a KPIs do not stand alone or operate in a vacuum; a clear understanding of vacuum; a clear understanding of what is possible is requiredwhat is possible is required “upper “upper and lower limits.”and lower limits.”
What are “limits”???What are “limits”???(ie., speed limits for cars)(ie., speed limits for cars)
KPIs are “Married” to KPIs are “Married” to benchmarksbenchmarks
116
Limits are…
LimitsLimits refer to “BENCHMARKSBENCHMARKS” which are essential to make KPIs useful; both internal and external…
They provide a basis for They provide a basis for ANALYSISANALYSIS……or as we see in the sporting or as we see in the sporting
business…business…
“competitive analysis”117
SebringSebring Racetrack Benchmarks
Example:Example:
DaytonaDaytona500500
SebringSebring
118
Benchmarks prevent Benchmarks prevent race car crashesrace car crashes
Benchmark Cones Control SpeedBenchmark Cones Control Speed Cars are forced to slow down in order Cars are forced to slow down in order to safely make the hairpin curve. to safely make the hairpin curve. How does How does
this apply to teaching and learning?this apply to teaching and learning?
=
119
Benchmarking for Benchmarking for quality: Why?quality: Why?
• Benchmarking fixes reference points relative to other measurements
• In the context of quality, benchmarking specifies reference values for KPIs
• Benchmarking is used to ensure that the KPIs are realistic
• Ensure KPIs are sufficiently demanding
120
Benchmarking for Benchmarking for quality: quality: What?What?
• Judgments Judgments about quality are based on comparisons with similar good institutions
• The levels of performancelevels of performance identified are benchmarks that a used for comparisons.
• Aggregate national figures for KPIs can also serve as benchmarks
• Should establish other benchmarks for matters of importance to them
• Remember:Remember: benchmarks should be both benchmarks should be both internal and externalinternal and external
121
KPI & Benchmarks(for Dr. Bothyna’s nursing program)
Learning Outcome: The nursing student will use new Disposable Gloves at The nursing student will use new Disposable Gloves at all times when nursing patients and whenever body fluids are present. all times when nursing patients and whenever body fluids are present.
KPI The student uses new disposable gloves 100% of the time while The student uses new disposable gloves 100% of the time while nursing patients and whenever body fluids are present.nursing patients and whenever body fluids are present.
Target Benchmark Target Benchmark 100% Goal
KPI Finding Benchmark KPI Finding Benchmark 95% Assessment finding
Internal Benchmark Internal Benchmark 98% Medical College
External Benchmark External Benchmark New Target Benchmark New Target Benchmark
98% Cairo University100% New Goal
AnalysisAnalysis: How is this data interpreted?: How is this data interpreted? What is the improvement plan to reach the new goal?What is the improvement plan to reach the new goal?
122
KPIs for learning outcomesKPIs for learning outcomes
Using a Using a Bubble MapBubble Map, write a course , write a course learning outcome for one of your learning outcome for one of your courses in the center circle. courses in the center circle.
Next write teaching methods for Next write teaching methods for learning outcome in the learning outcome in the connected circles.connected circles.
On the extended lines write an On the extended lines write an assessment method with a KPI and assessment method with a KPI and benchmarks.benchmarks.
123
Extra information that may Help
124
Program Learning Program Learning OutcomesOutcomes
What is essential to this degree??
Why are students enrolling??
What does the public think??
What the STUDENTSTUDENT expects to know and/or do via the Program…
125
ProgramProgram Outcomes Outcomes
What the What the STUDENTSTUDENT will will achieveachieve and and performperform at the at the end of the program.end of the program.
These are general, specialized learning These are general, specialized learning outcomes for all students in all outcomes for all students in all departments of the college program.departments of the college program.
Institutional learning outcomes Institutional learning outcomes are general are general learning outcomes for all students.learning outcomes for all students.
126
CourseCourse Outcomes Outcomes
What the What the STUDENTSTUDENT will will achieveachieve and and performperform at the at the end of a specific course.end of a specific course.
These are highly specialized learning These are highly specialized learning outcomes for all students in a given course.outcomes for all students in a given course.
127
Purposes of Program and Course Learning Outcomes
Measurable Improvement is #1Measurable Improvement is #1
Evaluate programs/courses, not Evaluate programs/courses, not personnelpersonnel
Assist with QA and accreditationAssist with QA and accreditation…
Initiate “best practices”…Initiate “best practices”…
Set learning to KPIs & benchmarksSet learning to KPIs & benchmarks…
128
Harden, R.M. (2001). AMEE Guide No. 21. Curriculum mapping: a tool for transparent and authenticteaching and learning. Medical Teacher, 23 (2), 123-137.
Hobson, E.H. (2005). Changing pedagogy. Presentation at SACS-COC Institute on Quality Enhancement and Accreditation, Orlando, FL, July 24-27, 2005.
Curriculum Alignment?Consistency and Intentionality
Learning OutcomeLearning Outcome
Learning ObjectivesLearning Objectives
129
ExamplesExamples
Student OutcomeStudent Outcome Student will demonstrate ability to use 10 conventions of safe science research in a science lab.
What are the differences???What are the differences???What are the different teaching methods?What are the different teaching methods?
What are the different assessments?What are the different assessments?
Student OutcomeStudent Outcome Student will be able to identify 10 safety rules
for the science lab in a paragraph.
Teaching Objective Teaching Objective teacher will teacher will teachteach & & demonstratedemonstrate 10 science lab safety rules 10 science lab safety rules
130
Conclusion
Thank you for your time and reflections
Dr Bothyna Zakaria MurshidDr Greg Maffet
131