Creating “Turning Points” in the Lives of Youth Residing in High

Risk Communities: Participation and Response to School-

Based Mentoring and Impact on Academic OutcomesMarcia Hughes

May 31, 2006

Purpose of the Study

Examine a youth program that was designed to help participants from high risk communities do well in school, and plan and prepare for post secondary education.

The study asks for whom was the program effective, in what ways, and under what circumstances.

The Intervention Project: GEAR UP

(Gaining Early Awareness and Readiness for Undergraduate Programs)

• Federally-funded: US Department of Education

• Collaborative effort: University of Connecticut & Hartford Public Schools

Acknowledgements

Principal Investigators: John C. Bennett, Sara Harkness, Kay A. Norlander-Case, Charles W. Case

Advising Committee: Sara Harkness, Preston Britner, Orv Karan, and Charlie Super

Undergraduate, Graduate Students, Participating Youth:

Many thanks to all the undergraduate and graduate students who have served on the project, & to all the participating youth.

Background: Evolution of Youth development field (Small & Memmo, 2004)

- Prevention: target a particular youth population deemed at risk of problem

- Fix-it approach: Runs counter to what is known about

human motivation

- Resilience: Risk and protective mechanisms exist together

- basis of mentoring programs

- assist with coping, problem solving

Youth Development Field: - Positive, asset building orientation - Builds on strengths rather than categorizing according to deficits - Get involved, develop competencies, experience success as intervention

Background: “Process is Product” (McLaughlin, 2000)

Developmental outcomes of interest:

- Learning to be productive; learning to connect; learning to navigate (Gambone, Klem, Connell, 2002)

- Constructive use of time, commitment to learn, positive values, & social competence (Small & Memmo, 2004)

- Confidence, character, connection, competence, contribution (Pittman, Irby, Tolman, Yohalem, & Ferber, 2003)

Getting “Buy-in” and Sense of Belonging:

- “Just opening the doors and getting youth involved marks a major accomplishment.” (McLaughlin, 2000)

- Belonging is of equal significance and utility in youth programs as social, behavioral, and academic outcomes (Barkdull, 2004)

Background: Broad Based Comprehensive Strategy

Contemporary models of youth development and problem prevention:

Incorporates all 3 paradigms, framed in a developmental-ecological model:

•Prevention

•Resilience

•Positive youth development (Catalano, Boglund, Ryan, Lonczak, & Hawkins, 1998; Gambone et al., 2002; Kerpelman, 2004; Small & Memmo,2004)

Promote bonding •Foster resilience •Promote social competence•Promote emotional competence•Promote cognitive competence•Promote behavioral competence•Promote moral competence•Foster self-determination•Foster spirituality•Foster self-efficacy•Foster clear and positive identity•Foster belief in the future•Provide recognition for positive behavior•Provide opportunities for prosocial involvement•Foster prosocial norms

“The State of the Field” (Catalano et al, 1998)

Evaluation of 25 National programs:

Background: Research and Program Evaluation

In order to advance the Youth Development Field, calling for:

- More meaningful measures (Gambone et al, 2002; McLaughlin, 2000)

- Assessing the “whole” child (Catalano et al, 1998; Riggs & Greenber, 2004)

- Linking the chain of effects (Catalano et al, 1998; Gambone et al, 2002)

- Getting inside the “Black Box”: Document/describe/understand

day-to-day realities and challenges of youth programs (Kalafat &

Illback, 1998; Koss-Chioino & Vargas, 1999; Small, 2005)

Research Model

- Social Support - Emotional bond/support- Academic Support- Cognitive Skills- Self-Awareness- Belonging - Sense of Competence- Sense of Control/ Coping Strategies- Improved relationships

PROGRAM PARTICIPATION & RESPONSIVENESS

PROBLEM BEHAVIORS RISK FACTORS & PROMOTING PROCESSES

Individual level

Family level

School & Community

level

- University or Community College- Technical School- Working- Retained- Drop out

Post-Secondary Outcomes

Last 2 years of

project participation

Research Questions

1) By what means, if at all, have the components of the project fostered processes that facilitated positive change for the youth?

2) How are youths’ level of participation and response to the intervention modified by individual adolescent needs and individual situations at home and at school? Specifically:

• What are the overlapping problem behaviors and support?

• What are the overlapping risk factors and promoting processes?

• In what ways did the project mitigate risk exposure?

• In what ways did the project promote positive processes?

3) How are differences in youth’s participation and responsiveness related to their post-secondary status (i.e., long-term goal of the project)?

GEAR UP: Current Status

• Longitudinal project, began in 1999:

- Two cohorts of students in the 6th & 7th grades (N=130)

- Have now completed the 11th and 12th grades (N=110)

• Indicators of Success:

- Continued high rate of participation (Approx. 82%)

- Low school dropout rate (Approx. 10%)

- High rate of students accepted for post secondary education (Approx. 50%)

Intervention Model: Program Description(Harkness, Hughes, Muller, & Super, 2004)

•Site-based program: Ongoing presence in the schools

•Continuity across contexts of home and school

•Transitional support: early adolescence - early adulthood

•One-to-one mentoring assignments

•Strong infrastructure: close monitoring and accountability

Intervention Model: Practicing Principles

(Harkness, Hughes, Muller, & Super, 2004)

•Intervene with youth in the context of their environments

•Not one-size-fits-all intervention/Environmental Fit

•Ethnographic process of inquiry, collaborator, not “expert”

•Focus: strengths, resources, & broad indices of development

•Process first, outcomes second

Methodology: Study Sample

Wide range of participants:

From the student who is excelling academically to the student who appears to be near drop-out.

Total # of participants/subjects in this study:• 45 from older cohort: 23 Male/20 Female

• 34 from younger cohort: 22 Male/12 Female• Total: 79 Subjects: 45 Male/32 Female

Ethnic Distribution: • 65 are Hispanic (all but 5 are Puerto Rican)• 12 are Black (all but 3 are African-American)• 2 are White

Methodology: Data Collection

Instruction on Data Collection • Service notes: Every 2 weeks with feedback • Summary reports: End of each semester

Data Collection (secondary data) • Documentation of process data for each participant • Collected by mentors/practitioners• In the form of a “service note” and “summary reports”

Analyzed data for the last 2 years of the project• Older Cohort: Fall semester 2003 - Summer 2005• Younger Cohort: Fall semester 2004 - Present (2006)

“Checklist” for organizing observations and impressions

Summary Reports: Different Children, Different Needs

Methods of Analysis

I Analysis of Qualitative Data: Coding system; analyzing service notes

1) Responsiveness 2) Problem behavior/risk factors 3) Promoting processes

II Factor Analysis: Reduced the large number of variables (i.e., codes) down to a few factors for each of the 3 subsets of variables -responsiveness, problem/risk factors, and promoting processes

III Cluster analysis: Students were “clustered” into groups based on similar profiles across the factor scales (i.e., on responsiveness, problem/risk factors, and promoting processes)

IV Chi-Square to test the effect of cluster group (each group with its own “profile” across responsiveness, problem/risk factors, and promoting processes ) on post secondary outcomes

Coding System for Analysis of Qualitative Data

Student participation & responsiveness

(R)

Problem behaviors and risk factors

(A, B, C)

Promoting processes

(E, F, G)

1. Social/instrumental support

2. Mentor bond/emotional support

3. Academic support

4. Cognitive skills

5. Self-awareness

6. Belonging and membership

7. Improved sense of competence

8. Improved sense of control

9. Improved relationships

1. Problem behaviors: motivational, behavioral, academic, interpersonal, resistance, stress

2. Risk: individual level

3. Risk: family level

4. Risk: school level

1. Individual

2. Family

3. School

Improved sense of competence/ability; positive school experience; looks for attention and approval from mentor

Goal oriented and goal setting; self-awareness including self evaluation and discussion of future self

Improved sense of control & improved sense of competence:

Hopeless and giving up but with sustained effort (versus avoidance), reduction of negative chain reactions and “corrective” experience

Results of Qualitative Analysis: Fine tuning and reliability of coding system

Coding system: fine tuning• For each service note, code was noted if present (0 or1)• Progress/report cards to confirm related mentor reports• Double-coded 18 cases

Reliability:

• Aggregated data by obtaining % across observations

• Calculated alphas: two sets of coded data for same youth

• Mean reliability = .92

Factor Analyses: Description

Data Screening • Aggregated data by obtaining % across observations • One outlier that was removed• Transformed variables where necessary (skewness or kurtosis)

Separate Analysis (responsiveness, problem behaviors/risk factors, promoting processes)

• Principal Components Analysis: Decide on number of factors• Factor Analysis: Varimax rotation to maximize the spread in loadings

Factors • Responsiveness to Project: 4 Factors/16 “observed” variables• Problem Behaviors/Risk Factors: 4 Factors/17 observed variables• Promoting Processes: 3 Factors/10 observed variables

4 Factors (59%) Observable variables: Student Responsiveness Loadings

Academic Engagement

(α=.82)

2.1 Accepts tutoring from mentor .62

2.2 Asks for tutoring assistance .49

7.1 Improved beliefs about ability, attitude/behavior .73

7.4 “Corrective” experience .75

7.5 Positive school experience .61

8.2 Engaged in problem solving, sustained effort .81

Goal Setting

(α=.66)

2.3 College application process .48

4.1 Increased self-awareness .49

4.2 Constructing and believing in possible selves .52

8.1 Goal setting, planning, and decision making .71

8.4 Follow through, tasks/next steps, short term plans .55

Relatedness

(α=.69)

1.1.1 Relate to one another, common interests, & ideas .60

5.2 Several good relationships within project .91

3.1 Engage in meaningful conversation/debate .50

Special Relationship

(α=.47)

1.1.3 Confides in mentor/asks for personal guidance .54

1.1.5 Excitement / inspired by mentor .47

4 Factors (60%) Observable variables: Problem Behavior/Risk Factors Loadings

Problem Behaviors

(α=.79)

A.7 Breaking rules: tardy, skipping class or school .53

B.2 Drug or alcohol abuse .60

B.6 Delinquency/crime .67

B.7 Alienation/rebelliousness .64

C.2 Home regulation: chaotic .69

C.5 Ineffective discipline .46

D.6 Negative peer influence .56

School Disruption

(α=.77)

A.8 Disruptive in class/behavioral issues .68

A.9 Not taking responsibility for behavior .62

A.16 School discipline: suspension .74

D.6 Negative peer influence .45

Stress and Strain

(.71)

A.13 Indicators of problems coping .73

B.9 Mental health difficulties, low self-esteem .74.

C.1 Family stress/strain .52

C.4 No/limited knowledge or support of academic status .54

Low Academic Motivation

(α=.54)

A.3 Inconsistent motivation, not following through, distracted .61

A.6 Avoidance/procrastination .52

A.11 Failing grade(s), difficulty with class .53

3 Factors (67.3%) Observed variables: Promoting Processes Loadings

Goal Oriented

(α=.83)

E.4 Self-regulated: goal-oriented & purposeful .63

E.6 Enduring set of values, balanced perspective .87

E.7 Good coping, problem-solving, & fortitude .78

E.8 Positive attitude toward school .58

High Academic Motivation

(α=.81)

E.3 Positive academic achievement .79

E.4 Self-regulated: goal-oriented & purposeful .56

E.9 Positive beliefs about self and future .70

Connected

(α=.62)

E.1 Positive attitude, responsive to others, pro-social .49

E.2 Attachment to others/intimate relationship .45

F.2 Parental warmth, encouragement, cohesion .65

F.6 Stable, integrated family routines, interdependency .59

Cluster Analyses: Responsiveness, Problem/Risk factors, Promotive Processes

Factor Scales on the following 11 factors: Student

Participation & Response to

Project

1. Academic Engagement

2. Goal Setting

3. Relatedness

4. Special Relationship

Problem Behavior & Risk Factors

5. Problem Behaviors

6. School Disruption

7. Stress and Strain

8. Low Academic Motivation

Promoting Processes

9. Goal Oriented

10. High Academic Motivation

11. Connected

Results of Cluster Analysis

CLUSTERS: 1 2 3 4

Sig # OF PARTICIPANTS:

26 12 27 9

1. Academic Engagement 4.9 7.6 10.6 8.4 .00

2. Goal Setting 15.8 21.4 21.1 18.3 .03

3. Relatedness 10.6 10.3 8.4 9.2 .31

4. Special Relationships 8.0 11.9 12.3 33.9 .00

5. Problem Behaviors .64 .12 .67 .77 .20

6. School Disruption .15 .21 .18 .31 .53

7. Stress and Strain 4.9 4.9 5.5 11.1 .00

8. Low Academic Motivation 7.9 6.4 15.5 8.5 .00

9. Goal Oriented 4.1 20.4 9.0 7.6 .00

10. High Academic Motivation 6.3 26.6 11.9 6.7 .00

11. Connected 4.6 8.1 9.3 10.0 .00

Chi-Square: Count/Percentage within Groups

(χ2 =.35, p<.00)Post Secondary

Status

Group 1 Group 2

Goal Setting

Group 3

Academic Engagement

Group 4

Special Relationship

Total

College 20%(5)

83.3%(10)

68%(17)

55.6%(5)

37

Technical School

16%(4)

0% 4%(1)

0% 5

Working 28%(7)

16.7%(2)

20%(5)

44.4%(4)

18

Retained 8%(2)

0% 1(4%)

0% 3

Drop Out 33.8%(7)

0% 4%(1)

0% 8

Total 25 12 25 9 71

Conclusions

• Using process data obtained from a comprehensive program that had noted success, was able to meet the challenges that are current in the youth development field:

• Used meaningful measures

• Assessed the whole child

• Demonstrated that developmental outcomes can be linked to long term academic outcomes

• Illuminated the day-to-day processes of the program

Conclusions

• For whom, in what ways, and under what circumstances?

• Was able to identify patterns of change among subgroups

• Meaningful overlap on student responsiveness, problem behaviors and risk factors, and promoting processes.

• Results support a developmental-ecological framework:

• In some cases, the project was able to mitigate exposure to risk.

• In other cases, the project was able to promote positive development.