Annual BEAP Report March 20, 2010 27 th Annual BPD Conference Atlanta, GA.
-
date post
21-Dec-2015 -
Category
Documents
-
view
234 -
download
1
Transcript of Annual BEAP Report March 20, 2010 27 th Annual BPD Conference Atlanta, GA.
Annual BEAP ReportMarch 20, 2010
27th Annual BPD Conference
Atlanta, GA
Introduction
Foundation Curriculum Assessment Instrument
Purpose of the FCAI
1. Provides Pre/Post test in seven major curricular areas of a BSW program
2. Seeks to help the program assess “value added” from entry to exit
3. Assists with identification of curricular areas that may need attention
4. Provides national comparative data
Curricular components Curriculum Area Number of
Questions
Practice 13
Human Behavior & Social Environment 10
Policy 9
Research 9
Ethics and values 8
Diversity 8
Social and Economic Justice 7
Sample HBSE Question
• The concept “person-in-environment” includes which of the following:a. Clients are influenced by their environment
b. Clients influence their environment
c. Behavior is understood in the context of one’s environment
d. All of the above
Sample Practice Question
• Determining progress toward goal achievement is one facet of the _____ stage.
– a. Engagement – b. Evaluation – c. Assessment – d. Planning
Testing History – 2006-2010 • Version 3 1 Test 65 ?s n= 305• Version 4 1 test 55 ?s n= 381• Version 5 1 test 82 ?s n= 286• Version 6 1 test 74 ?s n= 36• Version 7 1 test 72 ?s n= 318• Version 8 1 test 64 ?s
n=1576
TOTAL =2902
Reliability Testing – Version 8
–Tested in two junior practice classes–Students tested twice, 2 weeks apart–Pearson’s correlation coefficient: r
= .86
– a
Item difficulty index
• Overall difficulty or average should be around .5 (Cohen & Swerdlik, 2005)
• FCAI = .523 (n=415)
• “This is a very good difficulty level for the test.Not likely to misrepresent the knowledge level of test takers”.
Expansion beyond BSW
• Based upon CSWE assertions related to educational levels in social work education, we expanded testing to three additional groups: – MSW foundation students:
• entering • exiting
– Advanced standing students: • entering
Mean scores by Educational level – Version 8
Program level
N Mean Score
Std Dev Min Max
BSW entering 441 31.34 7.27 10 57
BSW exiting 522 40.62 7.25 9 58
MSW F entering 277 37.01 7.40 16 54
MSW F exiting 248 41.15 7.84 11 59
Advanced Standing entering
88 41.57 6.24 25 55
Instrument Discriminant Analysis
Pre/post test for differenceBSW & MSW Foundation
Groups by pre/post scores
Mean Pre test Score
Mean Post test score
SD
t-value
Sig.
BSW ENTERING/BSW EXITING
31.34 40.62 7.3
19.76
.000
MSW F ENTERING/MSW F EXITING
37.01 41.15 7.6
6.219
.000
Tests of Difference by Educational Level
Groups by Educational level
Mean MSW Exit Score
Mean BSW Exit Score
SD t-value
Sig/P-value
41.15 40.62 7.5 .924 Not Sig
Tests of Difference by Educational level
Groups by Educ. Level
BSW exiting
Advanced Standing
SD t-value Sig.
40.62 41.57 6.7
1.157 Not Sig.
BSW Entering BSW Exiting t-test p-value
441 522
Total Score (64)
31.734 40.62 -19.77 .000
Practice (13) 6.91 9.51 -20.01 .000
HBSE (10) 5.54 6.65 -10.25 .000
Policy (9) 3.49 4.71 -10.98 .000
Research (9) 3.41 4.84 -12.89 .000
Ethics (8) 4.18 5.47 -13.72 .000
Diversity (8) 3.94 4.71 -7.44 .000
Social & Econ Justice (7)
3.88 4.73 -8.35 .000
Domain Comparisons- Version 8 Overall
MSW F Entering
MSW F Exiting
t-test p-value
N 277 248
Total Score (64) 37.01 41.15 -6.22 .000
Practice (13) 8.44 9.44 -5.69 .000
HBSE (10) 6.09 6.69 -4.05 .000
Policy (9) 4.36 5.01 -3.40 .000
Research (9) 4.39 5.19 -4.87 .000
Ethics (8) 4.65 5.21 -4.95 .000
Diversity (8) 4.68 4.52 1.17 .244
Social & Econ Justice (7)
4.39 5.09 -5.00 .000
Domain Comparisons- Version 8 Overall
BSW Exiting
Advanced Standing
t-test p-value
N 522 88
Total Score (64) 40.62 41.57 -1.16 .248
Practice (13) 9.51 9.65 -0.61 .544
HBSE (10) 6.65 6.62 0.11 .913
Policy (9) 4.71 4.90 -0.93 .352
Research (9) 4.84 4.72 0.61 .539
Ethics (8) 5.47 5.60 -0.85 .396
Diversity (8) 4.71 4.98 -1.48 .140
Social & Econ Justice (7)
4.73 5.10* -2.23 .026
Domain Comparisons-Version 8 Overall
BSW Exiting
MSW F Exiting
t-test p-value
N 522 248
Total Score (64) 40.62 41.15 -0.92 .356
Practice (13) 9.51 9.44 0.47 .640
HBSE (10) 6.65 6.69 -0.34 .733
Policy (9) 4.71 5.01* -2.19 .029
Research (9) 4.84 5.19* -2.52 .012
Ethics (8) 5.47* 5.21 2.57 .011
Diversity (8) 4.71 4.52 1.60 .109
Social & Econ Justice (7)
4.73 5.09* -3.08 .002
Domain Comparisons- Version 8 Overall
Field Practicum/Placement Assessment Instrument
(FPPAI)
• In response to evaluate social work program outcomes related to the 2008 EPAS competencies and practice behaviors
• The need for a field/practicum assessment that measures competencies related to practice behaviors
Need for Field Evaluation
• Initial Piloting was conducted May 2008
• Second Piloting was conducted in Fall 2008 /Spring 2009
• Third Piloting phase was conducted Fall 2009
• Full implementation: Fall 2010
FPPAI Piloting Phases
• 55 Likert Scale questions measuring practice behaviors linked to the EPAS 2008 competencies.
• Qualitative feedback form for each domain available for program use.
• Will be available online and in print format.
• Individual program outcomes report with national comparisons.
• Individual program outcomes report with national comparisons for EPAS 2008 Competencies & Practice Behaviors
• Can be used as a final field assessment and mid-test/post test design.
• MSW pilot in the planning stages.
Assessment Methodology
FPPAI Scale
FPPAI: Instrument Sample (Quantitative)
FPPAI Sample: Qualitative Feedback Form
Educational Policy
2.1.1—Identify as a professional social worker and conduct oneself accordingly
Practice Behavior
use supervision and consultation
Measures
BEAP-Exit
BEAP-FPPAI
BEAP-Employer
BEAP-Graduate
BEAP & EPAS 2008
• 204 participants from 18 schools and 18 states participated in the FPPAI pilot project.
• Overwhelming positive feedback from participating schools.
• Incorporated pilot feedback survey results into final pilot revisions.
• Cronbach’s Alpha reliability test of internal consistency was completed, (n=204, .962) reporting excellent internal consistency.
• Fall online MSW foundation pilot
FPPAI: Overview & Key Findings
If your program is interested in being a part of the national ”MSW Foundation Online” pilot, please access the FPPAI website:
http://www.lcsc/bchristenson/fpai.htm
Dr. Brian Christenson:– [email protected]
Dr. Ruth Gerritsen-McKane:– [email protected]
Call for Foundation Year MSW Participation
http://beap.socwk.utah.edu/
http://www.lcsc.edu/bchristenson/fpai.htm
2009 Report on the Revised BEAP Instruments
Sample Profile Overview of all Respondents
2000-2006
Respondents
2007
Respondents
2008
Respondents
2009
Respondents
Overall
Respondents
Entrance 24,977 3,319 2,287 524 31,107
SWVI at Entrance 25,446 3,309 1,611 348 30,714
Exit 14,304 2,546 2,157 761 19,768
SWVI at Exit 11,936 2,283 1,551 423 16,193
Alumni 4,488 428 337 96 5,349
Employer 1,613 181 118 23 1,935
Totals 82,764 12,066 8,061 2,175 105,066
Sample vs. Population Program Type
2000-2006
Entrance
2007
Entrance
2008
Entrance
2009
EntranceOverall
Program
TypeN % N % N % N % BEAP % CSWE%
BSW Only 747 71.3 98 74.2 60 73.9 83 86.6 72.0 70.6
Combined
Programs
177 28.7 15 25.8 10 26.1 17 13.4 27.8 29.3
Total 924 100 113 100 70 100 100 100 99.8 100
Sample Profile Response Rates
Instrument2000-2006
Average %
2007
Average %
2008
Average %
2009
Average %
Overall
Average %
Entrance 96.03 96.17 96.10 91.42 96.10
Exit 93.56 93.98 94.01 91.98 93.55
Alumni 43.41 44.54 45.06 36.39 42.81
Employer 30.57 29.42 31.40 24.40 30.31
Auspices of College or University(Overall-Entrance)
Auspice Program Percentage Student Percentage
Public(#schools)
51 68.8
Private Denominational(#schools)
34 19.7
Private Non-Denominational(#schools)
15 11.6
Ethnicity at EntranceEthnicity
2000-2006 %
N= 24,977
2007 %
N= 3,319
2008 %
N= 2,287
2009 %
N= 524
Overall %
N= 31,107
European/
Caucasian67.1 62.2 66.9 57.1 65.6
African/Black 17.6 21.2 16.5 25.2 18.4
Native American 9.9 8.1 8.7 5.9 9.5
Other 3.8 3.7 3.8 2.7 3.9
Other Hispanic 3.8 5.6 4.0 6.5 4.2
Asian/Pacific Islander 3.4 3.5 3.9 2.9 3.4
Chicano/Mexican 3.0 3.5 3.5 4.6 3.1
Puerto Rican 1.6 1.7 1.6 3.6 1.6
Note: Percentages equal more than 100% due to multiple self-reported ethnicities.
Financial Aid at Entrance
Aid Source2000-2006 %
N= 24,977
2007 %
N= 3,319
2008 %
N= 2,287
2009 %
N= 524
Overall %
N= 31,107
Self Work/Savings 78.8 76.8 81.0 75.0 77.7
Loan* 77.9 78.5 81.0 89.7 77.0
Grant* 76.3 78.4 72.4 82.0 74.9
Family Assistance 44.0 43.0 41.1 39.5 43.3
*Includes Federal and State only.
Plan to be Employed During BSW Education
(Reported at Entrance)
2000-2006 %
N= 24,977
2007 %
N= 3,319
2008 %
N= 2,287
2009 %
N= 524
Overall %
N= 31,107
Plan to be Employed 79.4 81.0 80.5 73.7 78.9
Educational
Plans
2000-2006 %
N= 14,304
2007 %
N= 2,546
2008 %
N= 2,157
2009 %
N=761
Overall %
N=19,768
Have Future Educational Plans
82.9 83.2 84.6 86.5 82.2
MSW 75.7 77.8 79.4 80.7 75.6
Other MA 13.5 12.0 11.3 12.6 12.8
Other BA 2.9 3.0 2.9 3.0 2.9
Note: Percentages equal more than 100% due to multiple self-reported future educational plans.
Future Educational Plans-Exit
KSV- Exit
Mean Score 2000-2006 2007 2008 2009 Overall
Knowledge 7.28 7.38 7.38 7.25 7.30
Skills 7.35 7.43 7.48 7.31 7.37
Values 8.64 8.69 8.73 8.56 8.65
(Scale of 0-10)
Exit Advising Scores
2000-2006
N= 14,304
2007
N= 2,546
2008
N= 2,157
2009
N= 761
Overall Average
N= 19,768
Course selection/
curriculum planning6.82 7.00 6.95 6.71 6.84
Professional advising 6.66 6.80 6.85 6.44 6.68
Career planning 6.07 6.28 6.29 5.94 6.11
(Scale of 0-10)
Overall Advising Scores by Auspice- Exit
Course Selection and Curriculum Planning
Career Planning Professional Advising
Private Denominational
College or University
(# Schools)
7.46 6.75 7.33
Private-Non
Denominational College or University
(# Schools)
7.20 6.56 7.22
Public College or
University
(# Schools)6.61 5.86 6.40
(Scale of 0-10)
Overall Advising Scores by Program Type-Exit
Course Selection and Curriculum
Planning*
Career Planning* Professional
Advising*
BSW Only 7.08 6.43 7.01
Co-located Programs 6.14 5.18 5.71
N= 20,833t- test*p= .000
(Scale of 0-10)
Current Employment Information Primary Function - Exit
n= number reporting
Primary Function2000-2005 %
n=3536
2006 %
n=688
2007 %
n=731
2008 %
n=563
Overall %
n=5857
Generalist Practice 39.2 38.2 35.3 38.7 38.5
Direct Practice with Individuals, Families, or Groups
25.0 25.7 26.1 25.8 25.0
Training 10.8 11.3 12.2 11.7 11.3
Administration 8.5 7.3 8.3 7.1 8.3
Other Functions 5.0 4.5 6.4 6.2 5.3
Current Employment Information Primary Role - Exit
Primary Role2009 %
n=224
Advocate/Activist 33.5
Broker 22.3
Enabler/Facilitator 16.1
Administrator 9.4
Counselor/Clinician 6.7
Initiator 5.8
Coordinator 3.6
Case Manager 1.3
Convener/Mediator 1.3
n= number reporting
Client Systems - Alumni/ae
2000-2006 %
N= 4,488
2007 %
N= 428
2008 %
N= 337
2009 %
N= 96
Overall %
N= 5,349
Individual Clients 66.1 63.8 67.7 67.4 65.3
Family Clients 61.7 61.7 61.7 62.0 61.2
Group Clients 34.6 36.4 34.7 34.0 34.4
Organization Client 29.3 25.5 28.5 26.3 28.0
Community Clients 24.1 23.8 25.2 23.2 23.9
Other Client systems 2.9 3.0 2.7 2.5 2.9
Income Level Most Represented by Clients - Alumni/ae
Income Level
2000-2006 %
N= 4,488
2007 %
N= 428
2008 %
N= 337
2009 %
N= 96
Overall %
N= 5,349
Poverty or Below 52.0 54.2 54.1 54.5 52.0
No Typical Income 25.2 24.5 18.7 23.8 25.2
Above Poverty and Below Middle Income
17.8 16.7 13.9 16.4 17.8
Middle Income
or Above5.0 4.6 6.1 5.3 5.0
Professional Development During the Last Year
(Reported at Alumni/ae)
2000-2006
N= 4,488
2007
N= 428
2008
N= 337
2009
N= 96
Overall
N= 5,349
Mean # of times used evaluation of client progress 17.31 14.5 14.96 16.27 16.78
Mean # of times used program evaluation 6.21 5.41 5.88 6.36 6.08
Mean # of times used other research techniques 5.44 4.86 5.49 4.16 5.26
Mean # of professional workshops attended 5.11 5.02 4.78 4.55 4.86
Mean # of professional conferences attended 2.58 2.38 2.34 2.70 2.50
Current Employment Information- Field of Practice
Overall Exit
n=7,577
Overall Alumni/ae
n= 4,083
Corrections/Criminal Justice
23.1%
Child Welfare/Child Protection
19.4%
Child Welfare/Child Protection
14.6%
Mental/Behavioral/CMH
15.4%
Youth Services
8.4%
Aging and Gerontological Social Work
10.9%
Mental/Behavioral/CMH
7.1%
Health/Medical
7.6%
Aging and Gerontological Social Work
5.9%
Family Service
7.5%
n= number of responses
Current Employment Information 2009 Exit-Field of Practice
2009 Exit
n= 200
Overall Exit
n=7,577
Child Welfare/Child Protection
16.0%
Corrections/Criminal Justice
23.1%
Aging and Gerontological Social Work
10.0%
Child Welfare/Child Protection
14.6%
Youth Services10.0%
Youth Services
8.4%
Mental Retardation/Developmental Disabilities
9.5%
Mental/Behavioral/CMH
7.1%
Family Service
8.5%
Aging and Gerontological Social Work
5.9%
Current Employment Information 2009 Alumni/ae - Field of Practice
2009 Alumni/ae
n= 58
Overall Alumni/ae
n= 4,083
Child Welfare/Protection
29.3%
Child Welfare/Child Protection
19.4%
Aging and Gerontological Social Work
15.5%
Mental/Behavioral/CMH
15.4%
Alcohol, Drug, or Substance Abuse (and)
Mental Retardation/Developmental Disabilities
6.9%
Aging and Gerontological Social Work
10.9%
Housing (and)
School Social Work
5.2%
Health/Medical
7.6%
Family Service
7.5%
Mean* Knowledge Scores—at ExitBSW only BSW & MSW
Co-locatedTest**
Public7.35 6.82 p<.001
Private-
Denominational 7.59 6.77 p< .001
Private-
Non Denominational
7.65 7.26 p< .001
* Mean score on 10 point scale ** t-test, N= 19,768
Mean* Skill Scores—at ExitBSW only BSW & MSW
Co-locatedTest**
Public
7.41 6.84 p< .001
Private-
Denominational 7.71 6.86 p< .001
Private-
Non Denominational
7.77 7.46 p= .001
* Mean score on 10 point scale
** t-test, N= 19,768
Mean* Value Scores—at ExitBSW only BSW &
MSW Co-located
t-Test**
Public
8.71 8.30 p<.001
Private-
Denominational 8.82 8.36 p=.001
Private-
Non Denominational
8.78 8.73 p=.523
* Mean score on 10 point scale
** t-test, N= 19,768
Region I—Southeast*
Knowledge, Skills and Value Scores by Deciles
*Alabama, Arkansas, Mississippi, Georgia, Florida, South Carolina, Louisiana, Tennessee
Percentile Group
Knowledge Skills Values
Mean Mean Mean
10 5.64 5.62 7.00
20 6.17 6.16 8.00
30 6.75 6.85 8.00
40 7.17 7.31 8.77
50 7.67 7.77 9.00
60 8.00 8.00 9.50
70 8.25 8.31 9.80
80 8.75 8.85 10.00
90 9.33 9.38 10.00
Region 1 Public BSW Only Programs
What BEAP can do for your program
• Student demographics• Assessment on KSV• Employment information• Advising feedback• Professional development of students &
alumni/ae• Peer comparison by region, program type,
auspice & nationally• …and more
BEAP Instruments & EPAS Competencies/Practice Behaviors
2.1.6—Engage in research-informed practice and practice-informed research.
A.Use practice experience to inform scientific inquiry
B. Use research evidence to inform practice
Graduate
C19, 20, 21
E4
Exit
D20
D19
Employer
Q14
FCAI
31-39
FPPAI
23,36,37,38
23,36,37,38
BEAP & EPAS 2008
http://beap.socwk.utah.edu/
http://beap.socwk.utah.edu/site/annual.htm