Nikos Mattheos Malmö University, Sweden
description
Transcript of Nikos Mattheos Malmö University, Sweden
![Page 1: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/1.jpg)
IADR Honolulu Nikos MattheosApril 2004
Nikos MattheosNikos MattheosMalmö University, SwedenMalmö University, Sweden
Interactive softwareInteractive software and self-assessment skillsand self-assessment skills
![Page 2: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/2.jpg)
April 2004 IADR Honolulu Nikos Mattheos
Self assessment ability:Self assessment ability:
•Evaluate professional actions – process /outcomeEvaluate professional actions – process /outcome
• Compare to standardsCompare to standards
• Identify weaknessesIdentify weaknesses
• Define new learning objectivesDefine new learning objectives
![Page 3: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/3.jpg)
April 2004 IADR Honolulu Nikos Mattheos
self assessment ability self assessment ability
reflectionreflection feedbackfeedback
![Page 4: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/4.jpg)
April 2004 IADR Honolulu Nikos Mattheos
The Leo:The Leo:A free Internet-based platform for assessing and A free Internet-based platform for assessing and
training of self-assessment skills training of self-assessment skills in oral health education in oral health education (A. Nattestad – R. Attström)(A. Nattestad – R. Attström)
http://tmk.odont.ku.dk/leo/http://tmk.odont.ku.dk/leo/
![Page 5: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/5.jpg)
April 2004 IADR Honolulu Nikos Mattheos
![Page 6: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/6.jpg)
April 2004 IADR Honolulu Nikos Mattheos
![Page 7: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/7.jpg)
April 2004 IADR Honolulu Nikos Mattheos
![Page 8: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/8.jpg)
April 2004 IADR Honolulu Nikos Mattheos
Student’s essayStudent’s essay
Primary feedback: Primary feedback: Expert’s essayExpert’s essay
reflectionreflection
improvementimprovement?
![Page 9: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/9.jpg)
April 2004 IADR Honolulu Nikos Mattheos
LeoLeon=26n=26
StaticStaticn=26n=26
BBaasseelliinnee
week 1 week 2 week 3 week 4 week 5
AAsssseessssmmeenntt
CaseCase
11
CaseCase
11 1
1CaseCase
22CaseCase
33CaseCase
44
CaseCase
22CaseCase
33CaseCase
442
2 3
3
4
4
![Page 10: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/10.jpg)
April 2004 IADR Honolulu Nikos Mattheos
BBaasseelliinnee
• background data, demographicsbackground data, demographics
• competence with computers (score 1-50)competence with computers (score 1-50)
Baseline measurements: Baseline measurements:
![Page 11: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/11.jpg)
April 2004 IADR Honolulu Nikos Mattheos
Weekly evaluations: Weekly evaluations:
•use of the software use of the software
• attitudes, acceptance, expectationsattitudes, acceptance, expectations (Visual Analogue Scale 1-16)(Visual Analogue Scale 1-16)
1
1 2
2 3
3 4
4
![Page 12: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/12.jpg)
April 2004 IADR Honolulu Nikos Mattheos
AAsssseessssmmeenntt
The Interactive Examination:The Interactive Examination:
• self assessment as staring point (Internet)self assessment as staring point (Internet)
• oral + essay + written comparison with expertoral + essay + written comparison with expert
Mattheos N, Nattestad A, Falk Nilsson E, Attström R. The Interactive Examination: Assessing students’ self-assessment ability. Med Educ 2004, in press.
![Page 13: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/13.jpg)
April 2004 IADR Honolulu Nikos Mattheos
ResultsResults
• Improved performance in general Improved performance in general
• No significant difference between groupsNo significant difference between groups (moderate agreement between 2 assessors)(moderate agreement between 2 assessors)
• Time on task correlates to written performanceTime on task correlates to written performance
• Computer competence correlates to written performanceComputer competence correlates to written performance
![Page 14: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/14.jpg)
April 2004 IADR Honolulu Nikos Mattheos
• students judged 3 out of 4 cases as “easy”students judged 3 out of 4 cases as “easy”
• students were positive on Internet cases, students were positive on Internet cases, but some complained about workload but some complained about workload
• Attitude swiftsAttitude swifts
ResultsResults
![Page 15: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/15.jpg)
April 2004 IADR Honolulu Nikos Mattheos
1. What do you think of the Internet application you used? (VAS 1-16) Leo group Non Leo group difference
meanafter case 1
10,15
8,3
p>0,05
meanafter case 4
10,5 10,8 p>0,05
difference p>0,05 p = 0,0148
2. How much do you think that the software you used will help you at the exams?
Leo group Non Leo group difference
meanafter case 1
11,2 8,8 p = 0,0091
meanafter case 4
10,9 9,9 p>0,05
difference p>0,05 p>0,05
3. How much have you learned through the Internet cases?
Leo group Non Leo group difference
meanafter case 1
6,7 5,43 p>0,05
meanafter case 4
10 9,43 p>0,05
difference p = 0,0032 p = 0,0004
![Page 16: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/16.jpg)
April 2004 IADR Honolulu Nikos Mattheos
1. What do you think of the Internet application you used? (VAS 1-16) Leo group Non Leo group difference
meanafter case 1
10,15
8,3
p>0,05
meanafter case 4
10,5 10,8 p>0,05
difference p>0,05 p = 0,0148
2. How much do you think that the software you used will help you at the exams?
Leo group Non Leo group difference
meanafter case 1
11,2 8,8 p = 0,0091
meanafter case 4
10,9 9,9 p>0,05
difference p>0,05 p>0,05
3. How much have you learned through the Internet cases?
Leo group Non Leo group difference
meanafter case 1
6,7 5,43 p>0,05
meanafter case 4
10 9,43 p>0,05
difference p = 0,0032 p = 0,0004
![Page 17: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/17.jpg)
April 2004 IADR Honolulu Nikos Mattheos
1. What do you think of the Internet application you used? (VAS 1-16) Leo group Non Leo group difference
meanafter case 1
10,15
8,3
p>0,05
meanafter case 4
10,5 10,8 p>0,05
difference p>0,05 p = 0,0148
2. How much do you think that the software you used will help you at the exams?
Leo group Non Leo group difference
meanafter case 1
11,2 8,8 p = 0,0091
meanafter case 4
10,9 9,9 p>0,05
difference p>0,05 p>0,05
3. How much have you learned through the Internet cases?
Leo group Non Leo group difference
meanafter case 1
6,7 5,43 p>0,05
meanafter case 4
10 9,43 p>0,05
difference p = 0,0032 p = 0,0004
![Page 18: Nikos Mattheos Malmö University, Sweden](https://reader037.fdocuments.net/reader037/viewer/2022102719/56814216550346895dae251f/html5/thumbnails/18.jpg)
April 2004 IADR Honolulu Nikos Mattheos
Reflections…Reflections…
• Human feedback irreplaceable?Human feedback irreplaceable?• Content and cases too easy?Content and cases too easy?• Time not enough? Time not enough? •Assessment procedure not sensitive enough ?Assessment procedure not sensitive enough ?