Anders Jönsson (LUT) & Nikos Mattheos (OD) Malmö University
23
Anders Jönsson (LUT) & Nikos Mattheos (OD) Malmö University Dynamic assessment and ‘Interactive Examination’
description
Dynamic assessment and ‘Interactive Examination’. Anders Jönsson (LUT) & Nikos Mattheos (OD) Malmö University. How can professional-directed education foster reflecting and self-assessing practitioners?. - PowerPoint PPT Presentation
Transcript of Anders Jönsson (LUT) & Nikos Mattheos (OD) Malmö University
Anders Jönsson (LUT) & Nikos Mattheos (OD)
Malmö University
Dynamic assessment and
‘Interactive Examination’
Dynamic assessment and
‘Interactive Examination’
How can professional-directed education foster reflecting and self-
assessing practitioners?
How can professional-directed education foster reflecting and self-
assessing practitioners?
Jönsson, A. & Mattheos, N.
Malmö University
The students must be given the opportunity to practice these skills, as well as be assessed on them.
The students must be given the opportunity to practice these skills, as well as be assessed on them.
- a structured assessment methodology
- aims to evaluate students’
content specific skills in parallel to their
metacognitive
- a structured assessment methodology
- aims to evaluate students’
content specific skills in parallel to their
metacognitive
Jönsson, A. & Mattheos, N.
Malmö University
The Interactive Examination
1. Describe the methodology,
2. Present some results from last years cohorts,
3. Highlight some interesting differences between the two education centres.
1. Describe the methodology,
2. Present some results from last years cohorts,
3. Highlight some interesting differences between the two education centres.
Jönsson, A. & Mattheos, N.
Malmö University
This presentation will:
teacherteacherdoctodoctorr
Autumn 2004: Autumn 2004:
3434 3rd semester 3rd semesterdental students dental students
174174 1st semester 1st semesterstudent teachersstudent teachers
The Interactive Examination
6 steps
Self-assessment
Personal task Comparison task
EvaluationAssessmentFeedback
Jönsson, A. & Mattheos, N.
Malmö University
Method6 steps
1. Self-assessment
Likert-like questions from
1 (poor) to 6 (excellent)
1. Self-assessment
Likert-like questions from
1 (poor) to 6 (excellent)
The Interactive Examination
Self-assessment Personal task Comparison task
EvaluationAssessmentFeedback
- 11 - 11 competenciescompetencies
-match to instructors’match to instructors’judgementjudgement
-13 13 competenciescompetencies
- match to examination match to examination resultsresults
Methodstep 1
The Interactive Examination
Self-assessment Personal task Comparison task
EvaluationAssessmentFeedback
2. Personal task
An authentic problem from professional life
2. Personal task
An authentic problem from professional life
-A patient walks A patient walks is is your clinic and...your clinic and...
Internet based Internet based case case presentationspresentations
Actual ClassroomActual Classroomscenariosscenarios
A series of short A series of short Internet video filmsInternet video films
Methodstep 2
3. Comparison task
a. identify differences between own and ‘expert’ answer,
b. reflect on the reasons for these differences,
c. define own needs for further learning.
3. Comparison task
a. identify differences between own and ‘expert’ answer,
b. reflect on the reasons for these differences,
c. define own needs for further learning.
The Interactive Examination
Self-assessment Personal task Comparison task
EvaluationAssessmentFeedback
Methodstep 3
The Interactive Examination
Self-assessment Personal task Comparison task
EvaluationAssessmentFeedback
4. Evaluation
Asking the students to evaluate the whole
experience, through a standardised form
4. Evaluation
Asking the students to evaluate the whole
experience, through a standardised form
Standardised form – 10 fieldsStandardised form – 10 fields
8 identical – 2comparable8 identical – 2comparable
Methodstep 4
5. Assessment of students- personal task,
- comparison task.
5. Assessment of students- personal task,
- comparison task.
The Interactive Examination
Self-assessment Personal task Comparison task
EvaluationAssessmentFeedback
Methodstep 5
6. Personalized feedback
Comments on: - students’ self-assessment
-the personal task
–the comparison document.
6. Personalized feedback
Comments on: - students’ self-assessment
-the personal task
–the comparison document.
The Interactive Examination
Self-assessment Personal task Comparison task
EvaluationAssessmentFeedback
Methodstep 6
ResultsEvaluation Results from the Likert-like questions
that were identical in both centres:Results from the Likert-like questions that were identical in both centres:
Median scoreOD
Median score
TE
Learning experience 8 6
Clear expectations 6 5
Chance to show what you know
7 6
Help to prepare for work
6 6
Difficulty 6 6
ResultsEvaluation
ResultsSelf-assessment
Students as a group:Students as a group:
Self-assessment
OD(n = 34)
TE(n = 166)
In agreement 38% 16%
Higher 38% 72%
Lower 24% 12%
Jönsson, A. & Mattheos, N.
Malmö University
ResultsSelf-assessment
ResultsSelf-assessment
Individual students:Individual students:
Self-assessment
OD(n = 34)
TE(n = 166)
Significantly higher (p<0.05) 35% 78%
Significantly lower (p<0.05) 17% -
Jönsson, A. & Mattheos, N.
Malmö University
ResultsSelf-assessment
ResultsSelf-assessment
Regression analysis:Regression analysis:
Self-assessment
OD
Self-assessment
TE
Gender no relation no relation
Group no relation no relation
Success on exam no relation eta = 0.50
(p<0.001)
Clinical instructor no relation -
Jönsson, A. & Mattheos, N.
Malmö University
ResultsSelf-assessment
Jönsson, A. & Mattheos, N.
Malmö University
Qualitative analysis:comparison document
-differencesdifferences
-argumentsarguments
-learning needs,learning needs, which and how?
Resultsqualitative
analysis
Jönsson, A. & Mattheos, N.
Malmö University
differences in:differences in:- formform- content- content- attitudesattitudes
- interpretationinterpretation
Qualitative analysis:comparison document
Resultsqualitative
analysis
- differences..!differences..!
- mainly contentmainly content- few attitude- few attitude- few form few form - hardly any hardly any interpretationinterpretation
Learning objectives:Learning objectives:mainly content driven mainly content driven ”need to know more...”
- similarities..!- similarities..!
- mainly attitudemainly attitude- some contentsome content- some interpretationsome interpretation- few formfew form
Learning objectives:Learning objectives:need more in-serviceneed more in-servicetrainingtraining
Resultsqualitative
analysis
Jönsson, A. & Mattheos, N.
Malmö University
ResultsComparison task
Reflecting...Reflecting...
- differences in the nature of the - differences in the nature of the assessed task? assessed task?
- differences in the institutional differences in the institutional learning culture? learning culture?
- differences in student’s perception differences in student’s perception of the ”end product”?of the ”end product”?
Future research...Future research...
- longitudinal observation of longitudinal observation of self-assessment skills?self-assessment skills?
- Evaluation of interventions?Evaluation of interventions?
- investigate student’s investigate student’s perception of the ”end perception of the ”end product”?product”?
Jönsson, A. & Mattheos, N.
Malmö University
Jönsson, A. & Mattheos, N.
Malmö University
Contact:
Anders Jönsson
Nikos Mattheos
Dynamic assessment and
‘Interactive Examination’
Dynamic assessment and
‘Interactive Examination’
