Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training
-
Upload
vance-giles -
Category
Documents
-
view
21 -
download
0
description
Transcript of Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training
![Page 1: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/1.jpg)
Identifying and Assessing Identifying and Assessing Benchmarks in the Sequence Benchmarks in the Sequence of Psychology Education and of Psychology Education and
TrainingTraining
Nadya A. Fouad, Ph.D.
University of Wisconsin-Milwaukee
![Page 2: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/2.jpg)
THANK YOUTHANK YOU
• Many thanks to Cathi Grus, APA Education Directorate and Nadine Kaslow, Emory University and the members of the Benchmarks Workgroup and Assessment Workgroup for their hard work and contributions to competency movement
![Page 3: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/3.jpg)
CasesCases• Student A: Applying for internship. Faculty in
the program have had concerns about her from the middle of first year. Concerns have included ethical decision making, boundaries with clients, and poor self-reflection. Academic grades are in the A and B range, though, so the faculty vote to allow her to apply for internship. Letters are vaguely positive, because faculty cannot totally identify her strengths or challenges.
![Page 4: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/4.jpg)
CasesCases• Student B: 2nd Year doctoral student end of 2nd
year practicum. Faculty instructor has had some concerns about student, but all reports from on-site supervisor have been glowing. End of April, on-site supervisor asks for a meeting to discuss serious reservations about the student, including violations of confidentiality, seeking a personal, sexual relationship with a client, and manipulating the evaluation process.
![Page 5: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/5.jpg)
CasesCases
• Student C: 1st year doctoral student in first practicum notes in journal of fear of African-American clients. Responses to ethics vignette indicate lack of knowledge about ethics code and poor ethical decision-making. Initial consultation with supervisor indicates that student had been misrepresenting client contact.
![Page 6: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/6.jpg)
• Overview of Competency Movement• Competency Benchmarks• Competency Assessment Toolkit
OverviewOverview
![Page 7: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/7.jpg)
CONTEXTCONTEXT
![Page 8: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/8.jpg)
a “culture of competence”a “culture of competence”Roberts, Borden, Christiansen & Lopez (2005)Roberts, Borden, Christiansen & Lopez (2005)
• a shift within professional psychology toward an emphasis on the acquisition and maintenance of competence as a primary goal
![Page 9: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/9.jpg)
Culture of CompetenceCulture of Competence• Recent years have witnessed a burgeoning
interest in a competency-based approach to student learning outcomes in professional psychology
• Educational programs are expected to produce competence
• Professional credentialing bodies are expected to certify individuals as competent
• Policy makers laud competence• Consumers increasingly demand it
![Page 10: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/10.jpg)
Are all equal?Are all equal?
![Page 11: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/11.jpg)
Culture of CompetenceCulture of CompetenceIt is time to embrace a culture of the
assessment of competence
• fosters learning
• evaluates progress
• assists in determining curriculum and training program effectiveness
• advances the field
• protects the public
![Page 12: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/12.jpg)
Goal: Competent PsychologistsGoal: Competent Psychologists
Choosing A Therapist That's Right for You!
![Page 13: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/13.jpg)
A Pedagogical ShiftA Pedagogical Shift
• Traditional models of education and training focus on learning objectives
• Objective: aim or goal
• Curriculum is designed to meet goals• Competency models focus on outcomes
• Outcome: result, final state, achievement
• Measurement of student learning
![Page 14: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/14.jpg)
Culture of CompetenceCulture of Competence• We can learn a lot from the efforts of our
colleagues in education and the other health professions (medicine, pharmacy, nursing, dentistry)
• Assessment of Competence Toolbox (American College of Graduate Medical Education (ACGME)
• APA Guidelines for the Undergraduate Psychology Major (APA, 2007)
![Page 15: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/15.jpg)
What is competence in professional psychology? What does it look like?
How is it best assessed?
![Page 16: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/16.jpg)
Key Competency Initiatives in Key Competency Initiatives in Professional Psychology Professional Psychology
1990s 2000 2002 2006
CoA
revi
ses
G&
P
Mod
el C
urric
ulum
for T
rain
ing
in C
ouns
elin
g Ps
ycho
logy
AP
A E
thic
s co
de re
vise
d
AP
PIC
Com
pete
ncie
s C
onf.
AD
PTC
CC
TC P
ract
icum
Com
pete
ncie
s
BEA
Tas
k Fo
rce
on
Ass
essm
ent o
f Com
pete
nce
NS
CP
P C
ore
Com
pete
ncie
s
![Page 17: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/17.jpg)
**Rodolfa, E. R., Bent, R. J., Eisman, E., Nelson, P. D., Rehm, L., & Ritchie, P. (2005). A cube model for competency development: Implications for psychology educators and regulators. Professional Psychology:
Research and Practice, 36, 347-354.
![Page 18: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/18.jpg)
Key Competency Initiatives in Key Competency Initiatives in Professional Psychology Professional Psychology
1990s 2000 2002 2006
CoA
revi
ses
G&
P
Mod
el C
urric
ulum
for T
rain
ing
in C
ouns
elin
g Ps
ycho
logy
AP
A E
thic
s co
de re
vise
d
AP
PIC
Com
pete
ncie
s C
onf.
AD
PTC
CC
TC P
ract
icum
Com
pete
ncie
s
Com
pete
ncy
Ben
chm
arks
Wor
k G
roup
BEA
Tas
k Fo
rce
on
Ass
essm
ent o
f Com
pete
nce
NS
CP
P C
ore
Com
pete
ncie
s
Com
pete
ncy
Ass
essm
ent
Tool
kit
![Page 19: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/19.jpg)
Competency Cube Competency Cube
• General Consensus • 12 competency areas• BUT– what does each competency look like
across a developmental trajectory?
![Page 20: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/20.jpg)
Assessment of Competency Assessment of Competency Benchmarks Work GroupBenchmarks Work Group
• Chair: Nadya Fouad, Ph.D.• Two-day meeting September 2006• Gathered individuals knowledgeable
about domains of competence• 32 member work group
![Page 21: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/21.jpg)
GoalsGoals• Build on existing knowledge and advance the shift
to a “culture of competence”• Promote excellence in professional education and
training• Operationalize a developmental model of
competence in professional psychology• To better inform understanding of “entry level to
practice” in light of the APA policy on Education and Training Leading to Licensure
![Page 22: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/22.jpg)
CaveatsCaveats
• Applies most directly to those preparing for the practice of health service provision, i.e., those who will seek licensure
• Not meant to be prescriptive, a tool for programs to implement if they chose and in accordance with their model of education and training
• Assessment of competence must be balanced with the primary mission of the program: education and training
![Page 23: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/23.jpg)
Product of the GroupProduct of the Group
• A document that delineates competency benchmarks, or measurable standards of performance, that are developmental and integrated through the sequence of professional education and training
![Page 24: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/24.jpg)
How are the Benchmarks How are the Benchmarks Organized?Organized?
Core Foundational and Functional Competencies• Essential Component: what are the critical
elements of/what knowledge/skills/attitudes that make up this competency?
• Behavioral Anchor: what would it look like if you saw it (essential component)?
![Page 25: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/25.jpg)
Essential ComponentsEssential Components Professionalism: Professional values and ethics as
evidenced in behavior and comportment that reflects the values and ethics of psychology, integrity, and responsibility.
B. Deportment
Readiness for Practicum
Readiness for Internship
Essential Component:Understands how to conduct oneself in a professional manner
Essential Component:Professionally appropriate communication and physical conduct, including attire, across different settings
![Page 26: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/26.jpg)
Essential Component Essential Component
• Readiness for Entry to Practice:• Consistently conducts self in a
professional manner across and settings and situations
![Page 27: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/27.jpg)
Behavioral AnchorsBehavioral AnchorsProfessionalism B. Deportment
Readiness for Practicum
Readiness for Internship
Demonstrates appropriate personal hygiene and attireDistinguishes between appropriate and inappropriate language and demeanor in professional contexts
Demonstrates awareness of the impact behavior has on client, public and profession Utilizes appropriate language and demeanor in professional communicationsDemonstrates appropriate physical conduct, including attire, consistent with context
![Page 28: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/28.jpg)
Behavioral AnchorsBehavioral Anchors
Professionalism B. DeportmentReadiness for Entry to Practice• Verbal and nonverbal communications are
appropriate to the professional context including in challenging interactions
![Page 29: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/29.jpg)
BenchmarksBenchmarks
• Benchmarks document underwent period public comment (fall 2007)
• Group charged by APA Board of Educational Affairs to review comment and made recommendations (spring/summer 2008) (e.g 15 competencies)
• Final version went to BEA Fall 2008• Manuscript in press in TEPP
![Page 30: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/30.jpg)
Competency Assessment for Competency Assessment for Toolkit for Professional Toolkit for Professional
Psychology Psychology • Chair: Nadine Kaslow, Ph.D.• Six members in work group• Charge from APA Board of Educational Affairs:
Develop a “Toolkit” for professional psychology• Purpose: Promote broader implementation of
competence assessment and provide information about application of assessment methods to the assessment of competence
• Coordinated effort with Benchmarks Work Group
![Page 31: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/31.jpg)
Components of ToolkitComponents of Toolkit• Background and Introduction• Assessment Method Fact Sheets
• Description• Use specific to core competencies, formative
vs. summative, developmental level• Implementation• Psychometrics• Strengths/Challenges• Future Directions
![Page 32: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/32.jpg)
Assessment Methods in Assessment Methods in ToolkitToolkit
• 360 evaluation• Portfolio• OSCE• Structured Written & Oral
Exams• Case Presentation• Simulation/Role Play• Competence Evaluation
Rating Form• Self-Assessment• Ratings of live or recorded
performance
• Standardized Client Interview
• Client/Patient Process/Outcome Measure
• Consumer Satisfaction Survey
• End of Rotation Performance Review
![Page 33: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/33.jpg)
Annual review
Case review
s
Process/outcom
e data
Rating F
orms
Consum
er surveys
OS
CE
Portfolio review
s
Ratings P
erformance
Record review
s
Self-assessm
ent
Sim
ulations/role plays
Stand. patient nterview
s
Stand. oral exam
s
Written exam
s360 E
valuations
1 2 1 1 1 2 3 1 2 2 2 1 1
CompetencyProfessionalism: Deportment
Note: 1 = very useful method, 2 = useful, 3 = potentially useful
![Page 34: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/34.jpg)
Table 1Toolkit Assessment Measures, Foundational and Functional Competencies, Formative and Summative Evaluation, and Developmental Level
MethodOverall Broad Competencies Useful
Essential Components of Competencies Useful
Formative and Summative Evaluation (notes types of evaluation most useful for most competencies
Developmental Level (notes levels most useful for)
360-Degree Evaluation
FoundationalProfessionalismReflective practiceRelationshipsEthical and legal standards and policyInterdisciplinary systemsFunctionalSupervisionTeachingAdministrationAdvocacy
FoundationalIndividual and cultural diversityFunctionalIntervention
FormativeSummative
Readiness for internshipEntry level to practiceAdvanced credentialing
![Page 35: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/35.jpg)
Components of Toolkit (cont.)Components of Toolkit (cont.)
• Grid of Assessment Methods and Competencies Best Used for
• Glossary of Terms• Reference
![Page 36: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/36.jpg)
Toolkit: Next StepsToolkit: Next Steps
• Plan for Dissemination• On-line, downloadable resources for
education and training programs methods to assess competence
• Manuscript in press TEPP• Presentation at education and training
council meetings• http://www.apa.org/ed/graduate/
competency.html
![Page 37: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/37.jpg)
Challenges?Challenges?
• fostering an environment that supports enhancing excellence through assessment at the institutional, programmatic, and individual level; leadership to use results to strengthen
• Training in “difficult conversations”
![Page 38: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/38.jpg)
Challenges?Challenges?
• Recognize and articulate multiple roles trainers engage in and potential impact on evaluation
• Knowledge of a range of assessment tools, access to resources “tool kits”
![Page 39: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/39.jpg)
CasesCases• Student A: Applying for internship. Faculty in
the program have had concerns about her from the middle of first year. Concerns have included ethical decision making, boundaries with clients, and poor self-reflection. Academic grades are in the A and B range, though, so the faculty vote to allow her to apply for internship. Letters are vaguely positive, because faculty cannot totally identify her strengths or challenges.
![Page 40: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/40.jpg)
CasesCases• Student B: 2nd Year doctoral student end of 2nd
year practicum. Faculty instructor has had some concerns about student, but all reports from on-site supervisor have been glowing. End of April, on-site supervisor asks for a meeting to discuss serious reservations about the student, including violations of confidentiality, seeking a personal, sexual relationship with a client, and manipulating the evaluation process.
![Page 41: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/41.jpg)
CasesCases
• Student C: 1st year doctoral student in first practicum notes in journal of fear of African-American clients. Responses to ethics vignette indicate lack of knowledge about ethics code and poor ethical decision-making. Initial consultation with supervisor indicates that student had been misrepresenting client contact.
![Page 42: Identifying and Assessing Benchmarks in the Sequence of Psychology Education and Training](https://reader036.fdocuments.net/reader036/viewer/2022062517/568133fb550346895d9aefe9/html5/thumbnails/42.jpg)
Questions/CommentsQuestions/Comments
• Preparing….