Program Evaluation Southern Illinois University Carbondale ... · Psychology at SIU Carbondale. Our...

Program Evaluation

Southern Illinois University Carbondale

Counselor Education Program

Community Counseling, School Counseling, and

Marriage, Couple, and Family Counseling

Presented by:

Jennifer Allen, Anthony Moran, Brian Schaffner, Krista Smith, Anancia Stafford, and Joseph

Wulgaert

EPSY 547: Research and Evaluation in Counseling, Spring 2013

Instructor: Dr. Kimemia

Institution Mission:

The following excerpt is taken from the mission statement of Southern Illinois

University, Carbondale Master’s Handbook of Community Counseling, School Counseling, and

Marriage, Couple, and Family Counseling:

The mission of the Counselor Education Program is to prepare professionals in clinical

mental health, school, and marriage, couple and family counseling and to provide

opportunities for ongoing learning and development. The program serves a diverse

student body from the quad-state area of southern Illinois, southeast Missouri, southern

Indiana, and northern Kentucky; regional and national urban centers; and the

international arena. Faculty expect graduates to be knowledgeable and skilled in their

cognitive area of expertise, to be understanding and sensitive to the needs of the diverse

populations they serve, and to be skilled in the creative application of current

technologies. Faculty are committed to preparing ethically responsible counselors who

are critically reflective about their profession and who recognize they must continue to

learn and refine their knowledge, values, and skills throughout their professional lives.

Purpose of the Evaluation:

There are three purposes to this program evaluation:

1. To evaluate the Counselor Education Program of Southern Illinois University in each of

its three specialty tracks: Marriage, Couple, and Family; Community and Mental Health; and

School Counseling. The evaluation will be done from the perspective of graduates of these

tracks, as well as from the perspective of their employers.

2. To evaluate students’ perception of how well they were educated at SIU based on the

Mission Statement and specific tracks of the Counselor Education Program, as well as CACREP

standards as reflected in the Master’s Program Handbook .

3. To examine the accuracy of the self-assessments done by graduates of the program by

comparing their answers to those given by their employers.

Participants

Participants for the program evaluation were graduate students from SIUC Counselor

Education Program and employers who employed graduates from SIUC Counselor Education

Program. For purposes of this program evaluation the evaluation team looked back at the last 8

years of graduates from the program. The rationale for considering only graduates from within

the last 8 years is that the CACREP accrediting board evaluates the program every 8 years. The

evaluation team decided to consider only those participants who have graduated within the last

accrediting process that took place in 2005.

Program Design, Development, and Implementation

Program Design

The Counselor Education Program at Southern Illinois University Carbondale is a

CACREP accrediting program that is designed to help students gain knowledge, skills, and

necessary experiences for their development as professional counselors. Students in the program

are also expected to develop and grow their counseling skills approach when working with

clients.

Procedures

In the initial working stage, the evaluation team collaboratively discussed various options

in evaluating the Counselor Education Program within the Department of Educational

Psychology at SIU Carbondale. Our evaluation team came to the conclusion the best way to

assess the program was to base our program evaluation off of the Counselor Education Master’s

Program handbook of 2012. The team would use the mission statement as well as the track-

specific objectives as the basis for which to create the survey and to evaluate the program;

specifically, its effectiveness in preparing graduates for the counseling profession. Our next step

was to update this survey to address the content and objectives of the Master’s level student

handbook’s mission statement. After a few edits and purposeful updating to both surveys, our

team was able to have two surveys that fit the objectives and content in the mission statement.

Data was collected using Survey Monkey; an internet-based survey management

software. Assessments were sent to the email addresses of Educational Psychology and

Counselor Education graduates gained from the Southern Illinois University Chi Sigma Iota

(CSI) Database, the Educational Psychology student list-serve, the SIUC Division of

Development and Alumni Relations database, as well as a list provided by the Counselor

Education department. In the cover letter sent with each assessment was a request that the

recipient forward an employer copy to their supervisor. In addition, if any employers received

the email they were encouraged to forward the survey invitation to their employee who

graduated from the SIUC Counselor Education Program. Also included in the graduates survey

was a question asking about their employment and if participants could provide contact

information of their employers. Through this medium, the team was able to identify employer

information. Though, several graduates’ comments left little to no specificity thus resulting in a

low number of identifiable employer emails. This was how we got employer information and

were able to send surveys through email.

Our team’s intent was to use a paired sample T-test in looking at the data from both the

graduate students’ responses and that of their employers, but unfortunately that was not possible

with the lack of responses from the employers. As our data came in we noticed that the data we

were gathering was prevalent among students compared to employees. As expected in research

we had to make adjustments based on the data we had collected and ways to compare it.

Instruments

There were two surveys used as instruments during this program evaluation which were

based on surveys used in the past: one from 2007, and one from 2010. Some items in the current

surveys are consistent with the previous surveys but have been constructed to fit the most current

Master’s Program Handbook; whereas, several items are new and original. One survey was sent

to graduates of the Counselor Education Program at SIUC. The second survey was designed for

employers of the graduates. Both surveys had a section of shared demographic questions which

covered areas such as: ethnicity, gender, and employment status. Each survey also had general

questions related to the counseling program, followed by specific questions related to each

individual track. Questions for the individual tracks focused on objectives of the tracks which

are listed in the handbook and are directed around the mission statement of the program.

Program Evaluation:

General Program Objectives:

“The Master of Science in Educational Psychology is designed to develop students’

potential as professional counselors. Faculty in the program recognize the uniqueness of human

beings and are dedicated to helping students obtain the knowledge, skills, and experiences

necessary for development as counseling professionals. Uniqueness is emphasized in the

program in that each student is expected to develop his or her counseling approach with clients.

Faculty adhere to the belief that effective counselors must possess respect for human dignity, a

firm commitment to maximum development of individual potential, and a broad knowledge base

of education and counseling. In order to prepare these kinds of competent counselors the

following are required of all graduates of the program:

1. Respect for the dignity and worth of all individuals in a multicultural society.

2. Commitment to self-development and fulfillment of human potential.

3. Integration of educational and counseling processes.

4. Demonstrated competence in applying counseling skills.

5. Knowledge of related mental health professions.

6. Knowledge of research methods and commitment to counseling research.

In sum, graduates must demonstrate excellence in all skills and knowledge required to

work effectively in a variety of settings in a multicultural context. The faculty is dedicated to

facilitating development of these goals with students in the program.” (SIUC Counselor

Education Program Master’s Handbook 2012, p. 2-3)

Community (in transition to Clinical Mental Health) Counseling Objectives:

“A. To provide knowledge of the foundations of community, clinical mental health, and other

agency counseling including historical, philosophical, societal, cultural, economic, and political

dimensions; roles, functions, and professional identity; structures and operations of professional

organizations, training standards, credential bodies, and ethical codes; and implications of

professional issues such as recognition, reimbursement, and right to practice.”

“B. To foster understanding of the: (a) roles of community and clinical mental health counselors

in a variety of settings and their relationship to other professionals; (b) organizational, fiscal, and

legal dimensions of the settings in which counselors practice; (c) theories and techniques of

community and clinical mental health needs assessment to design, implement, and evaluate

community agency interventions, programs, and systems; (d) general principles of community

and clinical mental health intervention, consultation, education, and outreach; and (e) theoretical

and applied approaches to administration, finance, budgeting management, and staffing.”

“C. To provide knowledge and skills for the practice of Community (in transition to Clinical

Mental Health) Counseling including client characteristics, principles of program development

and service delivery, specialized consultation skills, and effective strategies for client advocacy.”

“D. To foster acceptance of the uniqueness of individuals and implications of a pluralistic

society.” (SIUC Counselor Education Program Master’s Handbook 2012, p. 3-4)

School Counseling Objectives:

“A. To provide knowledge of the foundations of school counseling including its history,

philosophy, and trends; the school counselor's role in relation to other school professionals; and

ethical and legal issues relating to school counseling.”

“B. To foster an understanding of coordination of counseling program components as they relate

to the total school community.”

“C. To provide knowledge and skills for the practice of school counseling including program

development and evaluation, counseling and guidance, and consultation.”

“D. To foster acceptance of the uniqueness of individuals and implications of a pluralistic school

society.” (SIUC Counselor Education Program Master’s Handbook 2012, p. 4)

Marriage, Couple, and Family Counseling Objectives:

“A. To provide knowledge of the most influential factors that shaped marriage, couple, and

family counseling as a specialization including its history, philosophical, and epistemology

premises; professional organizations, training standards, and credentialing bodies; ethical and

legal issues; and the role of marriage, couple, and family counseling in community, educational,

and business settings.”

“B. To provide knowledge and skills for the practice of marriage, couple, and family counseling

including current family systems theories and their application in working with couples and

families; basic interviewing, assessment, goal setting, and case management; family life-cycle

developmental stages and their interaction with factors unique to each family including

composition, ethnicity, socioeconomic status, and religious belief systems, as well as family of

origin and intergenerational influences.” (SIUC Counselor Education Program Master’s

Handbook 2012, p. 4)

Based on the Master’s Handbook 2012, the curriculum for the Counselor Education

program offers many necessary and appropriate courses that meet the needs of the listed

objectives and mission statement of the program. These courses include: professional identity

(Professional, Legal, and Ethical Issues for each specific track), social and cultural diversity

(Cross Cultural Factors Affecting Counseling), human growth and development (Life-Span

Development), career development (Career Development Procedures and Practices), the helping

relationship (Theories of Counseling and Counseling Skill Development), group work (Group

Theory and Practice), assessment (Appraisal in Counseling), research and evaluation (Inferential

Statistics and Research and Evaluation in Counseling), and clinical instruction (School

Counseling Practicum, Clinical Mental Health/ Community Counseling Practicum, and

Marriage, Couple, and Family Counseling Practicum, and Internship in Counseling). Each track

also has specific course work that is appropriate for their specialization. The program also

requires students take appropriate research courses or conduct and write a thesis. Research

courses for students can come from other related fields such as curriculum and instruction for

example. The program also offers opportunities for students to take courses in other related

fields such as rehabilitation, education, special education, psychology, and communication

disorders to name a few.

Data Analysis

Data analysis originally was to use a paired-sample t-test to compare employer and

graduate responses to check for consistency and reliability. There was only one employer

response received, which is not sufficient to do this type of analysis. The data analysis will

almost entirely focus on the graduate survey, as there were more than 20 responses making the

data easier to analyze.

Measures of central tendency will be looked at to analyze the data. Modes will be used to

show the most common response. This information may be particularly useful because of the

small sample size, and will make the data richer. Means will be the primary data used in the

analysis. Means will provide an idea of how participants generally responded to specific

questions on the survey. The survey is based on a scale that ranges from one to five. A response

of one indicates very poor, two indicates poor, three indicates adequate, four indicates good and

five is indicative of very good. Areas that appear not to need much improvement will be

indicated by means of 3.5 or above, with ratings above 4.0 indicating that the program prepares

students well in that area. Areas that may need improvement will be indicated by scores lower

than 3.5, with special attention to areas that are nearer to 3.0 or below.

Results

Demographics of Participants

In the Counselors Education Graduate Program Survey, there were a total of 40

participants. 30 of the participants completed the survey and 10 did not. When asked about

enrollment demographic, 39 out of 40 participants answered. 27 participants were graduates from

the program, 8 were currently enrolled, 3 were previously enrolled but never completed the

program and 1 reported never having been enrolled. When asked about graduation dates the

results showed the following; 4 participants graduated in 2012, 1 participant graduated in 2011, 3

participants graduated in 2010, 4 participants graduated in 2009, 2 participants graduated in

2008, 1 participant graduated in 2007, 3 participants graduated in 2006, no one reported to have

graduated in 2005 and 9 participants reported having graduated prior to 2005. There were 36

participants who responded to the ethnicity question with 5 preferring not to answer, and 31

providing a response. Of the 31 participants who responded, there were 3 African Americans, 1

Mexican American, 1 Asian, and 26 Caucasians. Of 36 participants, 29 were reported to be

female, 8 male and one who preferred not to answer. It shows that a large part of participant who

have already graduated from the Counselors Education program. Most participants have also

graduated prior to 2005 and are Caucasian.

When asked about employment status 18 of 36 participants reported that they were

currently employed in area of specialization within the counseling field. 8 participants were still

enrolled in school and not employed yet while 5 participants were employed in a field other than

counseling and not looking for a counseling position. Only 2 people reported unemployment. So,

most of the participants were employed in a counseling position in a field of choice. There were

16 School Counseling participants, 14 Community Counseling participants and only 4 Couple

and Family Counseling participants.

Participants Evaluation of Program

For the following counselor activities, please indicate to what extent you believe the

counselor education program at Southern Illinois University Carbondale prepares students to

perform each of the following:

Table 1 Very

poor

Poor Adequate Good Very

good

Rating

Average

Rating

Count

Intake

interviewing

0.0%

(0)

3.0%

(1)

45.5%

(15)

36.4%

(12)

15.2%

(5) 3.64 33

Individual

counseling

0.0%

(0)

0.0%

(0) 9.1% (3)

45.5%

(15)

45.5%

(15) 4.36 33

Group counseling 0.0%

(0)

0.0%

(0) 6.1% (2)

39.4%

(13)

54.5%

(18) 4.48 33

Child/adolescent

counseling

3.0%

(1)

18.2%

(6) 27.3% (9)

33.3%

(11)

18.2%

(6) 3.45 33

Family or couple

counseling

0.0%

(0)

9.1%

(3)

33.3%

(11)

45.5%

(15)

12.1%

(4) 3.61 33

Career/lifestyle

counseling

0.0%

(0)

0.0%

(0)

36.4%

(12)

42.4%

(14)

21.2%

(7) 3.85 33

Multicultural

counseling

0.0%

(0)

3.0%

(1) 15.2% (5)

39.4%

(13)

42.4%

(14) 4.21 33

**Special Note** Table 1 is based on a scale that ranges from 1-5. Very Poor represents 1,

Poor represents 2 Adequate represents 3, Good represents 4 and Very Good represent 5.

The averages provided are numerical representations of the responses.

Based on the responses of participants, Table 1 shows that participants percieved that the

program prepares students between well and very well for Group Counseling ( = 4.48),

Individual Counseling ( = 4.32). , and Multicultural Counseling ( = 4.21) . The results also

indicate that the program prepares students between adequate and well, but closer to well in

Family or Couple Counseling ( = 3.61), Career/Lifestyle Counseling( = 3.85) and Intake

Interviewing ( = 3.64). The results suggest that the program prepares students closer to

adequate than well in regards to Child/Adolescent Counseling( = 3.45) .





Table 2 Very

poor


Good

Rating

Average

Rating

Count

Counseling for

persons with special

needs

3.0%

(1)

21.2%

(7)

45.5%

(15)

27.3%

(9)

3.0%

(1) 3.06 33

Crisis counseling 0.0%

(0)

12.1%

(4)

54.5%

(18)

24.2%

(8)

9.1%

(3) 3.30 33

Diagnosis and

treatment planning

0.0%

(0)

15.2%

(5)

36.4%

(12)

39.4%

(13)

9.1%

(3) 3.42 33

Report writing 0.0%

(0)

6.1%

(2)

48.5%

(16)

39.4%

(13)

6.1%

(2) 3.45 33

Case

conferences/Staff

presentations

0.0%

(0)

12.1%

(4)

42.4%

(14)

36.4%

(12)

9.1%

(3) 3.42 33




Based on the responses of participants, means in Table 2 shows that participants felt their

preparation in Diagnosis/Treatment Planning ( = 3.42), Report Writing( = 3.45) , Crisis

Counseling ( = 3.30), and Case Conferences/Staff Presentation ( = 3.42) by the Counseling

Education Program was slightly closer to adequate than well. The respondents perceived their

preparation in Counseling People with Special Needs ( = 3.06) to be just better than adequate.





Table 3 Very

poor


good Rating

Average

Rating

Count

Guidance/psycho-

educational

activities

0.0%

(0)

6.3%

(2) 18.8% (6)

56.3%

(18)

18.8%

(6) 3.88 32

Program

development and

evaluation

0.0%

(0)

9.4%

(3)

34.4%

(11) 53.1%

(17)

3.1%

(1) 3.50 32

Recognize and

respond to a

variety of social

issues

0.0%

(0)

3.1%

(1) 25.0% (8)

62.5%

(20)

9.4%

(3) 3.78 32

Self-evaluate

counseling

competencies

0.0%

(0)

0.0%

(0) 12.5% (4)

65.6%

(21)

21.9%

(7) 4.09 32

Professional

ethical behavior

0.0%

(0)

6.3%

(2) 9.4% (3)

37.5%

(12) 46.9%

(15) 4.25 32

Assume

responsibility

0.0%

(0)

0.0%

(0) 12.5% (4)

46.9%

(15)

40.6%

(13) 4.28 32

Technical

competencies (e-

mail, internet,

web-pages)

0.0%

(0)

12.5%

(4) 37.5%

(12)

37.5%

(12)

12.5%

(4) 3.50 32




Based on the responses of participants, means in Table 3 suggest that the program

prepares students better than well to Self-Evaluate Counseling Competencies ( = 4.09), to

behave ethically ( = 4.25), and to assume responsibility ( = 4.28). The data suggests the

program prepares students between adequate and well but closer to well in Guidance/Pscyho-

educational activities ( = 3.88) and to Recognize/Respond to a variety of Societal Issues( =

3.78). The data suggests that the program prepares students in the middle between adequate and

well when it comes to Technical Competencies ( = 3.50) and Program Development/Evaluation

( = 3.50).

Participants Evaluation of the Program

Participants were asked to provide open-ended responses about what they felt were some

strengths of the program. Out of 25 responses, here are a few examples of the feedback that was

provided:

“That is combines individual, group, and family counseling and allows you to develop

your own style and nurtures you in that style.”

“Self- reflection, group counseling, research”

“It has been awhile, but I remember the group counseling practicum as one of the best

experiences. I enjoyed instructing the demonstration EPSY - 100 course for the other

grad students and discussing it afterward. I think I developed my group leadership skills

and I think this was a strength of the program.”

“Strong marriage & family theory & practicum experiences. Strong group counseling

courses.”


Participants were asked to provide open-ended responses about what they felt were some

improvements that can be made in the program. Out of 25 responses, here are a few examples of

the feedback that was provided:

“Make research real and doable on a on-going practice level. Inspire participation in

politics and policies.”

“More training in treatment planning, there was no crisis counseling class when I was

there (I think this has changed?), the substance abuse class was not required and I

think it should be.”

“School counseling program needs to be much stronger. With school counseling there

needs to be more of an emphasis working with children and in schools, and not so

much of a focus on individual counseling.”

“There is a lack of communication and some of the people in charge don't seem to

take the lead in guiding students where they need to go. There are always questions

that students have that no one seems to be able to answer.”


Participants were asked to provide open-ended responses providing additional comments

pertaining to the program. Out of 13 responses, here are a few examples of the feedback that was

provided:

“There is some talk of some cultural bias within the program.”

“End the thesis/research paper requirement for the program as there is little

application for this as a counselor in a community mental health setting. Focus

more on what it's actually like as a counselor in a community mental health

setting.”

“I am grateful that I had such wonderful professors that were willing to go the

extra mile for us!”

“Counseling has to do with helping people become healthy. I think that a program

that is training people to do that for others should provide supports to insure that

their mental health is a priority during the program. At times it seemed that people

were trying to make it as hard as possible on students instead of caring about their

health and experience.”

Community Counseling Program Evaluation

For graduates of the program in Community Counseling, please evaluate how the

program contributed to your knowledge or understanding of the following:

Table 4 Very

poor


Good Rating

Average

Rating

Count

Historical

foundations of the

specialization

0.0%

(0)

7.7%

(1) 23.1% (3)

46.2%

(6)

23.1%

(3) 3.85 13

Philosophical and

theoretical

foundations within

the specialization

0.0%

(0)

0.0%

(0) 15.4% (2)

53.8%

(7)

30.8%

(4) 4.15 13

The influence of

society within the

specialization

0.0%

(0)

7.7%

(1) 15.4% (2)

61.5%

(8)

15.4%

(2) 3.85 13

The influence of

economy within

the specialization

0.0%

(0)

15.4%

(2) 30.8% (4)

38.5%

(5)

15.4%

(2) 3.54 13

The influence of

politics within the

specialization

0.0%

(0)

23.1%

(3) 30.8%

(4)

30.8%

(4)

15.4%

(2) 3.38 13

Roles, function, 0.0% 7.7% 7.7% (1) 46.2% 38.5% 4.15 13

and professional

identity within the

specialization

(0) (1) (6) (5)

Operations and

structures of

professional

organizations

within the

specialization

7.7%

(1)

15.4%

(2) 7.7% (1)

46.2%

(6)

23.1%

(3) 3.62 13

Training

standards within

the specialization

7.7%

(1)

0.0%

(0) 23.1% (3)

38.5%

(5)

30.8%

(4) 3.85 13

Credentialing

bodies within the

specialization

0.0%

(0)

0.0%

(0) 23.1% (3)

30.8%

(4) 46.2%

(6) 4.23 13

Implications of

professional issues

within the

specialization

0.0%

(0)

7.7%

(1) 23.1% (3)

30.8%

(4) 38.5%

(5) 4.00 13

Relationship of

professionals

within the

specialization with

other mental

health

professionals

0.0%

(0)

0.0%

(0) 30.8% (4)

46.2%

(6)

23.1%

(3) 3.92 13

Strategies for

client advocacy

7.7%

(1)

7.7%

(1) 30.8% (4)

38.5%

(5)

15.4%

(2) 3.46 13

Specialized

consultation skills

7.7%

(1)

7.7%

(1) 15.4% (2)

46.2%

(6)

23.1%

(3) 3.69 13

Service Delivery 0.0%

(0)

0.0%

(0) 7.7% (1)

69.2%

(9)

23.1%

(3) 4.15 13

Increased

appreciation for

individual

uniqueness

0.0%

(0)

0.0%

(0) 0.0% (0)

46.2%

(6) 53.8%

(7) 4.54 13


Poor represents 2, Adequate represents 3, Good represents 4 and Very Good represent 5.


The data from the means in Table 4 suggests that graduates of the Clinical Mental

Health/Community Counseling Specialization perceived that the program did closer to very good

than good when it came to increasing appreciation for individual uniqueness ( = 4.54). The

program did better than good job preparing students for Service Delivery ( = 4.15), knowledge

about Credentialing Bodies within the Specialization ( = 4.23), knowledge about Roles,

Function, and Professional Identity within the Specialization ( = 4.15), and knowledge about

Theoretical Foundations within the Specialization ( = 4.15). The graduates perceived the

program as preparing them well to deal with Implications of Professional Issues within the

Specialization ( = 4.00). The data suggests that graduates perceived themselves as prepared

closer to well than adequate when it comes to Specialized Consultation Skills ( = 3.69),

Operations and Structures of Organizations within the Specialization ( = 3.62) knowledge

about Relationships of Professionals within the Specialization with other Mental Health

Professionals ( = 3.92), standards of training within the specialization ( = 3.85), the Influence

of Economy within the Specialization ( = 3.54), and the Influence of Society within the

Specialization( = 3.85). The data also suggests that graduates perceive their preparation in

regards to the Influence of Politics within the Specialization ( = 3.38) and preparation in

regards to Client Advocacy ( = 3.46) to be closer to adequate than good.

Community Counseling Program Evaluation

For graduates of the program in Community Counseling, please evaluate the following:

Table 5 Very

poor

Poor Adequat

e

Good Very

Good Rating

Averag

e

Ratin

g

Count

The amount of

coursework

required seemed

appropriate to

degree

0.0%

(0)

7.7%

(1)

15.4%

(2) 46.2

% (6)

30.8%

(4) 4.00 13

The program

supported my

research or

professional goals.

0.0%

(0)

7.7%

(1) 0.0% (0)

38.5%

(5) 53.8

% (7) 4.38 13

Preparation

acquired through

supervision

0.0%

(0)

15.4%

(2) 7.7% (1)

23.1%

(3) 53.8

% (7) 4.15 13

Preparation

acquired through

Internship/practicu

m

0.0%

(0)

0.0%

(0)

23.1%

(3)

30.8%

(4) 46.2

% (6) 4.23 13

Overall preparation

for work in a

community

counseling agency

0.0%

(0)

0.0%

(0)

23.1%

(3) 38.5

% (5)

38.5

% (5) 4.15 13

Program advisor

assisted in obtaining

first job position

7.7%

(1) 38.5

% (5) 7.7% (1)

15.4%

(2)

30.8%

(4) 3.23 13

Information

acquired in the

program regarding

addictions

15.4

% (2)

23.1%

(3) 46.2%

(6)

7.7%

(1)

7.7%

(1) 2.69 13


Poor represents 2, Adequate represents 3, Good represents 4 and Very Good represent 5.


Based on the responses of the participants, means in Table 6 shows that graduates

perceived their preparation acquired through Internship/Practicum ( = 4.23), Preparation

through Supervision ( = 4.15) and overall preparation to work in a community counseling

agency ( = 4.15) was better than well at supporting research/professional goals (4.38). The

amount of coursework required for the degree was good ( = 4.00). The graduates perceived the

program as doing a less than adequate job at providing information regarding addictions ( =

2.69) and performed closer to adequate than well when it came to advisors assisting in obtaining

their first position ( = 3.23).

School Counseling Program Evaluation

Immediately after starting your first job as a school counselor, how well prepared were

you in the following areas:

Table 6 Very

poor


Good Rating

Average

Rating

Count

Collaborating

with teachers

0.0%

(0)

16.7%

(2) 25.0% (3)

16.7%

(2) 41.7%

(5) 3.83 12

Collaborating

with

administrators

0.0%

(0) 25.0%

(3)

25.0%

(3)

25.0%

(3)

25.0%

(3) 3.50 12

Collaborating

with parents

0.0%

(0)

25.0%

(3) 25.0% (3)

8.3%

(1) 41.7%

(5) 3.67 12

Participating in

IEP meetings

33.3%

(4)

16.7%

(2) 33.3%

(4)

16.7%

(2)

0.0%

(0) 2.33 12

Administering

standardized

tests

8.3%

(1) 33.3%

(4)

33.3%

(4)

16.7%

(2)

8.3%

(1) 2.83 12

Organization

and time

0.0%

(0)

8.3%

(1) 33.3% (4)

16.7%

(2) 41.7%

(5) 3.92 12

management

Handling

students’

academic issues

8.3%

(1)

16.7%

(2) 33.3%

(4)

16.7%

(2)

25.0%

(3) 3.33 12

Handling

students’

social/emotional

issues

0.0%

(0)

0.0%

(0) 16.7% (2)

33.3%

(4) 50.0%

(6) 4.33 12

Program

development and

evaluation

0.0%

(0)

8.3%

(1) 58.3%

(7)

33.3%

(4)

0.0%

(0) 3.25 12

Coordinating

programs that

consider all

stakeholders in

the school

community

0.0%

(0)

16.7%

(2) 33.3% (4)

41.7%

(5)

8.3%

(1) 3.42 12


Poor represents 2, Adequate represents 3, Good represents 4 and Very Good represents 5.


After graduating from the Educational Psychology Department at SIUC and beginning

their first job, participants felt most prepared in handling students’ social/emotional issues,

average score ( = 4.33). Participants feel very well prepared with organization and time

management ( = 3.92). Next, they felt most prepared in collaborating with teachers ( = 3.83),

parents ( = 3.67), and then administrators ( = 3.50). The subsequent area participants would

consider themselves to be most prepared is in coordinating programs that consider all

stakeholders in the school community ( = 3.42). The last of and the following areas are in

descending order, rated from a little above adequate to poorly prepared: handling students’

academic issues ( = 3.33); program development and evaluation ( = 3.25); administering

standardized tests ( = 2.83); and lastly survey participants felt least prepared in participating in

IEP meetings ( = 2.33).

Table 6.A.

Table 6.B.

Table 6.C.

Table 6.D.

Table 6.E.

Table 6.F.

Based upon the responses from graduates of the Counselor Education Program, the field

experience received at SIU Carbondale was between an adequate and a good amount (Table

6.A.). A majority of the participants indicated they were adequately prepared near well prepared

to work with students from all age groups after graduation (Table 6.B.). In addition, a majority

of the participants indicated they were satisfied as well as very satisfied with the education they

received in the School Counseling track of the Counselor Education Program (Table 6.C.). Of

the following: the history, the philosophy, and the trends of school counseling – participants

specified being most educated in the philosophy, next the trends, and last the history of school

counseling (Table 6.D.). Half of the participants indicated the education they received was

somewhat unaligned with their real-world work experience and a few signified their education

was somewhat aligned (Table 6.E.). Seven participants suggested they were very approachable

for students and teachers who are in need of help, one participant indicated somewhat more

approachable, and four chose approachable (Table 6.F.).

Couples and Family Counseling Program Evaluation

Table 7 Very

poor


good Rating

Average

Rating

Count

Historical

knowledge of the

specialization

0.0%

(0)

0.0%

(0) 0.0% (0)

33.3%

(1) 66.7%

(2) 4.67 3

Philosophical

premises of the

specialization

0.0%

(0)

0.0%

(0) 0.0% (0)

33.3%

(1) 66.7%

(2) 4.67 3

Knowledge of

professional

organization

within the

specialization

0.0%

(0)

0.0%

(0) 33.3%

(1)

33.3%

(1)

33.3%

(1) 4.00 3

Training

standards within

the specialization

0.0%

(0)

0.0%

(0) 33.3%

(1)

33.3%

(1)

33.3%

(1) 4.00 3

Credentialing

bodies within the

specialization

0.0%

(0) 33.3%

(1) 0.0% (0)

33.3%

(1)

33.3%

(1) 3.67 3

Role of

specialization

0.0%

(0) 33.3%

(1)

33.3%

(1)

0.0%

(0) 33.3%

(1) 3.33 3

within business

settings

Role of

specialization

within community

settings

0.0%

(0) 33.3%

(1)

33.3%

(1)

0.0%

(0) 33.3%

(1) 3.33 3

Role of

specialization in

school settings

0.0%

(0) 33.3%

(1)

33.3%

(1)

33.3%

(1)

0.0%

(0) 3.00 3

Theoretical

models within the

specialization

0.0%

(0)

0.0%

(0) 0.0% (0)

66.7%

(2)

33.3%

(1) 4.33 3

Interviewing 0.0%

(0)

0.0%

(0) 33.3%

(1)

33.3%

(1)

33.3%

(1) 4.00 3

Goal Setting 0.0%

(0)

0.0%

(0) 33.3%

(1)

33.3%

(1)

33.3%

(1) 4.00 3

Case Management 0.0%

(0)

0.0%

(0) 33.3%

(1)

33.3%

(1)

33.3%

(1) 4.00 3

Family Life Cycle

Developmental

Stages

0.0%

(0)

0.0%

(0) 33.3%

(1)

33.3%

(1)

33.3%

(1) 4.00 3

Family of origin

influences

0.0%

(0)

0.0%

(0) 0.0% (0)

66.7%

(2)

33.3%

(1) 4.33 3

Intergenerational

influences

0.0%

(0)

0.0%

(0) 0.0% (0)

66.7%

(2)

33.3%

(1) 4.33 3

The means from the data in Table 7 suggest that the graduates from the Marriage,

Families, and Couples Specialization perceive the program as closer very good than good at

preparing them to deal with historical knowledge ( = 4.67) and philosophical premises ( =

4.67)within the specialization. The graduates also appear to perceive their preparation in regards

to inter-generational ( = 4.33) and family of origin ( = 4.33) influences, and theoretical models

within the specialization ( = 4.33) to be better than good. The data suggests that the program

prepared students well in regards to interviewing( = 4.00) , case management ( = 4.00) , goal

setting( = 4.00), knowledge about the Family Life Cycle and Developmental Stages( = 4.00) ,

knowledge about Training and Standards within the Specialization ( = 4.00), and knowledge

about Professional Organizations within the Specialization ( = 4.00). The graduates appeared

to perceive that they were prepared closer to well than adequately when it came to credentialing

bodies within the specialization ( = 3.67). The data suggests that perceptions of preparation in

reference to the roles of the specialization in the business ( = 3.33) and community ( = 3.33)

settings to be closer to adequate than good. The perception of preparation in reference to the role

of the specialization in the school setting on average was adequate ( = 3.00).

Three purposes were proposed for evaluating the Counselor Education Program at SIUC.

Purpose three states “to examine the accuracy of the self-assessments done by graduates of the

program by comparing their answers to those given by their employers”. The evaluation team

was unable to fulfill this purpose because only one response from an employer was obtained. As

indicated by the employer, the employee graduated in 2008 from the program with a school

counseling degree. The survey respondent chose “very good” for all general and track-specific

items. No additional information was provided and all open-ended responses were left blank. At

least two more responses would have been necessary in order to complete data analysis as

originally planned. Due to insufficient data, the evaluation team was unable to satisfy purpose

three of the program evaluation.

Limitations to the data collection process:

Our limitations for this program evaluation go beyond just addressing the data. To begin

we wish to reflect on the idea that students whom were a part of the SIUC Counselor Education

program may be in-fact inclined to answer the survey in a positive fashion and may not be fully

honest about the struggles they faced throughout the program. This can be seen in self-reflecting

surveys. Also the survey itself is a research created survey that has not been tested for reliability

and validity. However, our group hoped to combat this with the fact by our ability to compare

such results to the results of the employer survey using trusted data programs such as SPSS.

Another limitation to our study is the process of research, which can be difficult at times.

After our deadline we found that there was a lack of responses from the employer survey.

Unfortunately with the lack of responses from the employer survey, we did not have enough data

to compare the employer survey responses with those of the graduate survey.

A limitation concerning the internal results of the survey is that other factors contribute to

the satisfaction of the program. For example, depending on the graduating year each participant

may have experienced a different set of professors and courses. Other factors may include

varying regional expectations. For example, receiving one’s education from the SIUC Counselor

Education Program may very well prepare him or her for work in Illinois. However laws,

standards, and guidelines of other regions and states may not fully prepare him or her for

counseling work in that part of the world. Lastly, external factors such as old counseling

frameworks, practices, and ideals may be a contributing factor in the dissatisfaction of the SIUC

Counselor Education Program. These are all uncontrollable factors of which no educational

entity can address, but should be considered when forming conclusions based from the results of

the surveys.

Recommendations and Suggestions

General Counselor Education Recommendations:

The counselor education program appears to be above adequate in all areas assessed in

the general portion of the surveys. The strongest two areas of the program are preparing students

for individual and group counseling, with multicultural counseling also achieving a mean above

good. This indicates that these are the areas graduates feel the program is strong in. The highest

rating was for group counseling. The group practicum is probably the most unique aspect of this

program, and an experience our graduates felt well prepared for. This information could be

useful to the program in future recruiting efforts. The graduates also appeared to perceive the

program as doing a good job of preparing them to self-evaluate their competencies and prepare

them to behave ethically/responsibly. The recommendation here is to keep doing what is

currently being done, because what is done appears to prepare students well.

There were several areas that fell just below the 3.50 threshold. These are areas that

graduates appear to perceive the program as preparing them closer to adequately than preparing

them well. These would not be high priority areas based off the data collected in this survey, but

could still be improved upon. The idea here is that the goal is to have a good program as

opposed to an adequate program. The first area noted was counseling children and adolescents.

In the past there has been no known class offered by the department specifically addressing

counseling children and adolescents. The recommendation would be to offer such a class, and it

is important to note that one is planned for Fall of 2013, which may address this perceived

deficiency. The other three areas that fell just under the 3.50 threshold are Intake Interviewing,

Diagnosis and Treatment, Report Writing, and Case Conferences and Presentations. The data

related to diagnosis may be skewed since a class is now offered that once was not. The mode

was “good”, but there were almost an equal number of “adequate” responses. Report writing,

case conferences and presentations are areas that could easily be better incorporated into the

practicum experiences. In regards to intake interviews, no information was gathered from the

survey that would help generate recommendations on this account. It is important to note that

Clinical/Community Counseling and the Marriage and Family Counselors are required to

shadow two or three interviews and do a minimum of three intakes on their own for a minimum

total of five intakes. Perhaps this number should be increased to better prepare students for

intake interviewing.

Crisis Counseling appears to be an area that could be improved upon. As with diagnosis,

a class has been added in recent years covering this topic. Still, in future surveys it may be

beneficial to distinguish between experience and education on the subject, as there is a class

requirement but no practicum requirement. The Clinical Center makes this option available to

students who have completed their practicum. It may be beneficial to require a certain number

of crisis sessions, though the logistics of that idea may be unrealistic because of the random

nature of crises and actual need for crisis counselors in the Clinical Center. Counseling people

with special needs had the lowest general score, with an average of just above adequate. Adding

classes always seems to be the solution, but realistically there can only be so many classes added.

This is an area that may differ from specialization to specialization. In that regards the program

might be able to improve on this by adding aspects of this to existing classes or offering an

elective seminar once per year on the topic.

Clinical Mental Health/Community Counseling Specialization:

Overall the participants perceived the Clinical Mental Health (CMH) track to prepare

them well in relation to specific aspects of the mission statement. All but three of the 22 areas

rated where above the 3.50 threshold, and nine areas had averages above 4.00. The program

appears to already well prepare students in service delivery, appreciation of individual

uniqueness, information about credentialing bodies, the implications of professional issues and

the roles, function, and professional identity in CMH. Preparation through Internship/Practicum,

Preparation through Supervision, and overall preparation to work in a community counseling

agency was better than well at supporting research/professional goals. The amount of

coursework required for the degree was good.

There were four areas that were below the threshold but not very far below the threshold

with the exception of information gained on addictions. As per before, the participant responses

indicate that these are areas that could be improved upon, but are lower priority. These were

Influence on Politics and Strategies for Client Advocacy. Both of these particular areas are

covered in a class within the specialization called “Legal and Ethical Issues in Clinical and

Mental Health Counseling”. They both seem somewhat related in that advocacy is often aimed

at changing the sociopolitical structures that create barriers for clients. The program might better

prepare students in both these areas by assigning a paper in this class that addresses both of these

interrelated issues within the specialization. Advocacy could also be addressed in the

multicultural class, as issues that clients need advocated for often result from cultural differences

between dominant and minority cultures. The perception of being helped by their advisor to

acquire their first job fell closer to adequate than good. This could be addressed by a

concentrated effort by professors to help new graduates find employment. The only area in this

specialization that was rated below adequate was information gained on addictions. There is

now a requirement of taking an addictions course that has not always existed based off of written

feedback from respondents. This may be an issue that has already been addressed, but the low

score and

School Counseling Specialization:

The data suggests that graduates of the school counseling track perceive the program as

preparing them near adequately in most areas. The average perception was only rated as better

than “good” when it comes to handling the social and emotional issues of students. This is

reflected in some of the comments left in the open ended questions:

“I felt completely unprepared for the rest of what we do (the other 85%) what

ASCA would consider non-counseling duties.”

“School counseling program needs to be much stronger. With school counseling

there needs to be more of an emphasis working with children and in schools,

and not so much of a focus on individual counseling.”

“I can't speak to the other tracks, but school counseling track needs better

classes that focus more of school counseling and working with children and in a

school system and not focus so much on the individual counseling and

research aspect.”

“As important as comprehensive school counseling is, the fact is that very few

school are actually able to have a fully comprehensive counselor and I felt

completely unprepared to do many parts of my job, one of which was academic

advising. I think that spending at least a little time on "real world counseling"

would be so beneficial to school counseling students, especially those that are

planning to work in the Southern Illinois area. Also, addressing attendance

support would be helpful and how to introduce a school to what a school

counselor is for as opposed to the traditional guidance model.”

The graduates of the school counseling track appear to have issues when it comes to

preparation outside of counseling duties, which in the real world they appear to perceive as

playing a smaller role than what is addressed in the curriculum. These comments are consistent

with the data collected. The areas that fell below the 3.50 threshold were the handling of student

academic issues, program development and evaluation, and coordination of programs that

considers all stakeholders in the school community. When it came to participating in IEP

meetings as well as administering standardized tests, the average rating was below adequate.

These are all areas that are outside of the “counseling” aspects of a school counselor’s job.

These low ratings could be the result of being taught by professors with experience in different

tracks outside of the specialization, as reflected in this comment:

“Hire professors with actual school counseling experience. ESPECIALLY at the

high school level. Invite us (practicing school counselors) or hire us as part time

teachers. Students must know what we actually do.”

Based upon the data, the recommendation would be to hire a professor with experience as

a school counselor, but that happened at the beginning of the 2012-2013 school years. The

effects of this recent change would not realistically be reflected through results of this survey.

This would not be the only recommendation. One comment in particular appeared to come from

someone who has experience working with our recent interns:

“It appears that quality of school counseling interns/practical students is low right

now. These individuals do not follow through with issues addressed, are often

unable to work without being told specifically what to do every step of the way,

and have to be coached on how to log hours etc. also appears that they are

lacking in foundational aspects of theories in counseling.”

This particular comment could be the result of one person’s experience with only one

student. Considering the feedback and overall lower ratings of the school counseling program in

comparison to the other specializations, the entire school counseling specialization would be an

area where the program delivery could be specifically improved. The recommendation would be

to have a professor with school counseling experience look at the program and brainstorm ways

to include more aspects of school counseling that reflect real world experience. Another possible

way to improve would be to host a focus group with graduates from the school counseling

program that may yield more concrete ideas about how the school counseling curriculum could

be improved to better match the experiences of school counselors in the real world. A focus

group with experienced graduates who are experienced school counselors would be invaluable in

generating recommendations for improvement in comparison to a survey.

Marriage, Family, and Couples Counseling Specialization:

Graduates of the Marriage, Family, and Couples Counseling (MFC) specialization

appeared to perceive the program as preparing them well for work within their specialization.

Eleven of the fifteen responses where on average rated as “good” or better, with historical

knowledge and philosophical premises of the specialization rated as closer to “very good” than

“good”. The data suggests that the program could benefit from covering the role of the

specialization in differing settings, particularly the school setting. The only recommendation for

improvement would be more detailed coverage of the roles of the specialization in community,

business, and school settings in more detail. This could be done in the “Legal and Ethical Issues

in Marriage, Family, and Couples Counseling” course, that serves as an introduction to the

specialization.

Although an adequate number of responses were received from graduates of this

program, the same cannot be said for their employers. It is our opinion that the impressions of

the graduates towards this program should be compared to the opinions of their employers;

which was our original intent. However due to the lack of responses of employers such a

comparison was not possible in this evaluation. With a longer time period in the future, more

employer surveys may be possible to receive. With the lack of employer responses it is difficult

to fully understand how employers rate the quality of education from graduates of the SIUC

Counselor Education program. The frequency of responses was higher on the low end, which

suggests that this may still be an issue that needs to be further addressed. Incorporating more

information on addictions into courses could be one way of improving in this regard, but

encouraging CMH students to take the CADC sequence in the Rehab department would probably

be the best way to fully prepare CMH students for working with addicted populations,

Program Evaluation Southern Illinois University Carbondale ... · Psychology at SIU Carbondale. Our...

Documents

Transcript of Program Evaluation Southern Illinois University Carbondale ... · Psychology at SIU Carbondale. Our...