What are we to do?
description
Transcript of What are we to do?
![Page 1: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/1.jpg)
What are we to do?
Jim Crossleyfor Maastricht 2012
![Page 2: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/2.jpg)
Outline
• A crisis in WPBA
• A brief history in lenses
• Construct-aligned scales
• Some implications
![Page 3: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/3.jpg)
Outline
• A crisis in WPBA
• A brief history in lenses
• Construct-aligned scales
• Some implications
![Page 4: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/4.jpg)
Crisis
UK Academy of Medical Royal Colleges
2010“The profession has rightly
lost all confidence in Workplace Based
Assessment”
![Page 5: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/5.jpg)
Outline
• A crisis in WPBA
• A brief history in lenses
• Construct-aligned scales
• Some implications
![Page 6: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/6.jpg)
Getting a clear picture
![Page 7: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/7.jpg)
![Page 8: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/8.jpg)
First lens - constructs
?
![Page 9: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/9.jpg)
Getting a clear picture
![Page 10: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/10.jpg)
Getting a clear picture
![Page 11: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/11.jpg)
Domain of Interest
Test Sample
Test Sample
Sampling
![Page 12: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/12.jpg)
Second lens - sampling
?
![Page 13: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/13.jpg)
Getting a clear picture
![Page 14: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/14.jpg)
![Page 15: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/15.jpg)
Outline
• A crisis in WPBA
• A brief history in lenses
• Construct-aligned scales
• Some implications
![Page 16: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/16.jpg)
Surgical skills study
• 3 instruments• 1635 assessments• 437 cases• 15 index procedures• 5 specialities• 85 surgical trainees• 155 judges
Modelled reliability with
3 cases & 2 observers
• PBA, G=0.81
• NOTTS, G=0.59
• OSATS, G=0.65
![Page 17: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/17.jpg)
PBA response scale• Level 1 Unable to perform the
procedure, or part observed, under supervision
• Level 2 Able to perform the procedure, or part observed, under supervision
• Level 3 Able to perform the procedure with minimum supervision (needed occasional help)
• Level 4 Competent to perform the procedure unsupervised (could deal with complications that arose)
• Very Poor
• Poor
• Good
• Very good
![Page 18: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/18.jpg)
educ
atio
nal
anch
ors
slid
ing
clin
ical
anc
hors
ACAT 35 12CBD 8 3mini-CEX 7 3raters to reach G=0.7
It generalises
![Page 19: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/19.jpg)
Getting a clear picture
![Page 20: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/20.jpg)
Third lens – judge alignment
?
![Page 21: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/21.jpg)
Outline
• A crisis in WPBA
• A brief history in lenses
• Construct-aligned scales
• Some implications
![Page 22: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/22.jpg)
Understanding development
Please join in!
![Page 23: What are we to do?](https://reader036.fdocuments.net/reader036/viewer/2022062323/56816112550346895dd06e3b/html5/thumbnails/23.jpg)
judgement and obscurity
Now we see as through a glass darkly; then we shall see face to face. Now I know in part; then I shall know fully, even as I am fully known.1 Corinthians 13:12