Circle Program Presentation

The New Hampshire Circle Program Final Presentation

Derek Gower, Stacy Martinez, and Phillip Pless

2017 M.P.P. Candidates at The College of William & Mary

December 2016

“I met my second mom through the program. She forever changed my life. I owe so much to the camp.”

-Circle Program Alumna

Project Deliverables

Entrance and Exit Survey

InstrumentsLiterature Review Analysis of Survey

Results

SWOT Analysis Program

Recommendations

Catalano et al. Meta-Analysis of PYD Efficacy

Studies included review of programs of one, two, or two or more social domains

School

Family

Community

Experimental or quasi-experimental design studies

Improvements in school attendance, academic performance, and peer emotional support

Increased communication and cognitive competence

Program findings not generalizable

Successful Characteristics of PYD Programs

Include at least five PYD constructs

Measure positive and problematic outcomes

Structured curriculum

At least 9 months or greater in length

Dubois et al. (Meta-Analysis)

Meta-analysis reviewed 55 studies on the positive effects of mentoring programs on youth

Results of a meta-analysis provides support for effectiveness of youth mentoring programs

Results of a fixed-effects model analysis indicate an overall or modest positive effect for specific mentoring programs

Theory-based mentoring programs (i.e. those that provided ongoing training to mentors and structured activities with mentors and youth) tended to higher effect sizes than others

Having ongoing support for mentoring relationships is an important predictor for larger effect sizes

Intensity and quality of relationships is also an important influencer

Meta-Analysis Limitations

Findings do not necessarily reflect the causal effects of either mentoring or different moderator variables examined.

Methodologies of experimental design differed (some random assignment versus others)

Generalizability of findings are also a significant issue of concern

This study doesn’t include potential changes in intervention design that could impact the findings

Other studies (e.g. Dubois and Silverthorn) suggest that the overall effects of mentoring is small

Research Gaps

Some studies (4-H) under-represent minorities in sample

Effectiveness of PYD on substance use not definitive

No common mentoring theoretical framework may impact program evaluations

“The Circle Program forced me out of my comfort zone…it taught me about responsibility and reward, and my experiences there are now among my most cherished memories”


Analysis of Teen Survey Results

Demographics (11 Questions)

Program Involvement (4 Questions)

Positive Youth Development (PYD)

Confidence (6 Questions)

Connections (3 Questions)

Character (8 Questions)

Caring (3 Questions)

Competency (3 Questions)

Contribution (3 Questions)

Teen Survey: Demographics

Eleven questions developed measured demographic information

*n=25; Teen Survey Question #1

0

1

2

3

4

5

6

7

8

9

14 15 16 17

Nu

mb

er o

f R

esp

on

den

ts

What is your age?

0

1

2

3

4

5

6

7

8

9

10

Freshman (9thGrade)

Sophomore(10th Grade)

Junior (11thGrade)

Senior (12thGrade)

Nu

mb

er o

f R

esp

on

den

ts

What grade are you currently in?

Teen Survey: Competence

Three questions were developed to investigate enhancement in social, emotional, cognitive, and vocational ability.

91.7%

8.3%

Since Circle Program: Do you feel that you think more about what you'd like

to do after High School?

Yes

No

*n=24; Teen Survey Question #11b

64.0%

54.2%

36.0%

45.8%

0%

10%

20%

30%

40%

50%

60%

70%

Before Circle Program After Circle Program

Perc

ent

of

Resp

ondents

Did you generally feel that you had trouble focusing or concentrating?

Yes No

Teen Survey: Confidence

Six questions were developed to measure improvements in self-esteem, self-confidence, and self-efficacy.

40%

92%

60%

8%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Before Circle Program Since Circle Program

Pe

rce

nt

of

Re

spo

nd

en

ts

Before & Since Circle Program: Do you generally feel confident?

Yes No

75%

96%

25%

4%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Pe

rce

nt

of

Re

spo

nd

en

ts

Before & Since Cicle Program: Do you generally feel proud of your accomplishments?

Yes No

*n=25; Teen Survey Question #8a+14a *n=25; Teen Survey Question #8b+15b

Teen Survey: Connections

Six questions were developed that aim to measure the building or strengthening in connections with people and institutions.

79%

100%

21%

0%0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Pe

rce

nt

of

Re

spo

nd

en

ts

Before & Since Circle Program: Did you generally feel that you had people you could

talk to about your problems?

Yes No

*n=24; Teen Survey Question #9a+15a

Teen Survey: Character

Eight questions were developed to measure enhancement in character development through increased self-control, and increased respect for society rules.

96%

4%

Since Circle Program: Did you generally feel like you have a better understanding of right

and wrong?Yes

No

55%

29%

45%

71%

0%

10%

20%

30%

40%

50%

60%

70%

80%


Pe

rce

nt

of

Re

spo

nd

en

ts

Before & Since Circle Program: Did you generally feel that you didn't want to go to school?

Yes No

*n=24; Teen Survey Question #9d+15d *n=25; Teen Survey Question #11e

Teen Survey: Caring

Three questions assess an increase capacity in girls for caring through empathy and identification with others.

92%100%

8%0%

0%

20%

40%

60%

80%

100%


Pe

rce

nt

of

Re

spo

nd

en

ts

Before & Since Circle Program: Did you generally feel that you listened to and cared

about other people?

Yes

No

92%

8%

0%

20%

40%

60%

80%

100%

Yes No

Pe

rce

nt

of

Re

spo

nd

en

ts

Since the Circle Program: Do you feel like you care more about your friends

in general?

*n=25; Teen Survey Question #9b&15b *n=25; Teen Survey Question #11d

Teen Survey: Contribution

33%

8%

58%

0%

10%

20%

30%

40%

50%

60%

70%

Yes No No, but I would liketo in the future

Pe

rce

nt

of

Re

spo

nd

en

ts

Have you served as a peer-mentor for other girls in summer camp?

0

1

2

3

4

5

6

7

8

9

10

None 1 2 3 4 5

Nu

mb

er

of

Re

spo

nd

en

ts

Number of Extra-Curricular Activities

How many extra-curricular activities are girls involved in outside of school?

*n=24; Teen Survey Question #18 *n=25; Teen Survey Question #4

Teen Survey: Program Involvement

Four questions aimed to measure program involvement

25%

21%

38%

13%

4%

0%

5%

10%

15%

20%

25%

30%

35%

40%

More thanonce aweek

Once aweek

Two ormore times

a month

Once amonth

Less thanonce permonth

Pe

rce

nt

of

Re

spo

nd

en

ts

How often are you in contact with your Circle Program mentor?

87.5%

12.5%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Yes No

Pe

rce

nt

of

Re

spo

nd

en

ts

Are you still in contact with friends you made at camp during the school

year?

*n=25; Teen Survey Question #19 *n=25; Teen Survey Question #17

Entrance and Exit Survey:

Lessons Learned from Teen Survey Assessment

More Likert Scale Questions

Simplify wording (avoid double negatives)

Need for open-ended questions

“The Circle Program gave me a sense of home when I did not feel like I had one.”


Survey Design- Alumni PYD

Research and Teen Survey

Build off Previous

Work

What do we want to know?

Tailor Survey to

Adults

Audience

Readability

Simplicity

Recall Bias

Original Program

Questions

6 C’s

Personal and Financial Security

Anecdotal Free Response

Demographic

Survey Design- Alumni

Total of 35 Questions

11 Demographic

14 relating to 6 C’s

3 Financial Stability

3 Personal Stability

4 Free Response

Survey Analysis- Alumni

“The lessons learned in Circle have enabled me to go through day to day experiences and identify the positives from each day. Circle has taught me to trust others, to create friendships, to work hard, and to appreciate everything in life. Thank you.”

– Circle Alumna


6

5

7

6

0

1

2

3

4

5

6

7

8

17-20 21-25 26-30 31-35

Ages of Alumni Surveyed


0

2

4

6

8

10

12

4th Grade 5th Grade 6th Grade

# o

f Resp

ondents

What Grade were you in when you entered the Circle Program?

0

2

4

6

8

10

12

14

6th Grade 7th Grade 8th Grade 9th Grade 10th Grade 11th Grade 12th Grade

# o

f Resp

ondents

What grade were you in when you left the Circle Program?


50% Yes

4

3 3

2

33% without children want them someday

0

2

4

6

8

10

12

14

Do you havechildren?

1 Child 2 Chidren 3 Children 4 Children If none now,in the future?

Children

0

2

4

6

8

10

12

14

Strongly Agree Agree Neutral Disagree StronglyDisagree

# o

f Resp

ondents

Generally, I would say I am happy and healthy.


0

1

2

3

4

5

6

7

More than oncea week

Once a week Two more timesa month

Once a month Less than once amonth

# o

f Resp

ondents

Fequency of Contact

Connection - 45% of Aumni surveyed are still in contact with a mentor

0

1

2

3

4

5

6

7

8

9

More than oncea week

Once a week Two more timesa month

Once a month Less than once amonth

# o

f Resp

ondents

Fequency of Contact

Connection - 83% of Aumni surveyed are still in contact with friends made at camp


75%

42%

0

2

4

6

8

10

12

14

16

18

20

Do you see yourself as a mentor to someoneelse?

Do you currently have a mentor (in or out ofthe Circle Program)?

# o

f YES R

esp

onse

s

Contribution and Connection

0 2 4 6 8 10 12 14 16 18 20

Doing what I believe is right, even if myfriends make fun of me.

Standing up for what I believe, even when it isunpopular for me to do so.

Telling the truth, even when it's not easy.

Accepting responsibility for my actions when Imake a mistake or get into trouble.

Doing my best, even when I have a job I do notlike.

Lerner's Character EvaluationHow important are the following to you?

Extremely Important Very Important Somewhat Important Not Important


0

1

2

3

4

5

6

7

8

9

10

Not at all Slightly Moderately Quite a bit Completely

# o

f R

esp

on

den

ts

Competence and Connection - Do you feel that you developed skills and relationships of value from the

Circle Program?

92%

62%67%

87%

62%

0

5

10

15

20

25

Working with otherpeople

Expressing myfeelings

Understandingother people's

feelings

Being confident in agroup of people

Being able tomanage everyday

life stressors better

Competence, Confidence, Caring - Did Circle Participation help you feel more capable of the

following?


0

2

4

6

8

10

12

14

16

Anxiety Stress Anger Uncertainty Sadness Frustration

Currently do you feel you cope well with the following emotions?

Yes Somewhat No

0

2

4

6

8

10

12

14

16

Just me One other adult Two other adults Three other adults Four or more otheradults

How many adults live in your home?


0

1

2

3

4

5

6

7

8

9

10

Some highschool

Highschool

graduateor GED

SomeCollege

Currentlyenrolled

in college

Collegedegree

Tradeschool or

vocationaltraining

AttendingGraduate

School

# o

f R

esp

on

den

ts

Highest Level of Education

0

2

4

6

8

10

12

14

16

Full-time Part-time Seasonal Work fromhome

Unemployed Other

Number of responses more than number of individuals surveyed. Question worded as "Check all that apply".

Employment Status of Alumni Surveyed


25%

50%

12.5% 12.5%

00

2

4

6

8

10

12

14

Completely Quite a bit Moderately Slightly Not at all

# o

f R

esp

on

den

ts

Do you feel participation in Circle Program aided in your personal and/or social achievements?

50%

25%

4%

21%

0

2

4

6

8

10

12

14

Never 1-2 times a year 3-6 times a year Almost every month

# o

f R

esp

on

den

ts

Frequency

How often do you worry about paying your electric or gas bill?


Positive Results, 6 C’s appear to have positively influenced Alumni

Recall Bias

Self Reporting

Response Bias

SWOT Analysis:

Strength:

Combination of year long mentoring and residential summer camp

Survey analysis suggests program effectiveness

Weaknesses:

Lack of resource diversification

Anecdotal evidence for program effectiveness

Opportunity:

Entrance/Exit Survey Circle Program’s impact

Program expansion opportunities

Threats:

Downturn in economy

Recommendations:

Administer Entrance and Exit Survey Electronically

Measure improvements in school and/or reductions in risky behavior

Assess survey results and methodology annually

Alumni are interested in participating

Consider developing a mentor survey

“Being a middle child of five children, the Circle Program helped me to establish my own personal identity through the many experiences that the program offered to me.”


Thank you! Questions?

Sources

“Mission and Philosophy Quote.” The Circle Program’s Mission and Philosophy, 01 Jan. 2016. Web. 09 Sept. 2016. http://www.circleprogram.org/mission1

“Circle History.” The Circle Program’s History, 01 Jan. 2016. Web. 09 Sept. 2016. http://www.circleprogram.org/about1-c1xmq

“Annual Newsletter.” The Circle Program’s Newsletter, 01 Jan. 2016. Web. 09 Sept. 2016. http://www.circleprogram.org/newsletters

Ramer, H. 28 April 2014.”New Hampshire ranks 8th in high school graduation rates.” http://www.washingtontimes.com/news/2014/apr/28/new-hampshire-ranks-8th-in-high-school-graduation/ Last accessed 28 Nov 2016

http://www.circleprogram.org/mission1

http://www.circleprogram.org/about1-c1xmq

http://www.circleprogram.org/newsletters

http://www.washingtontimes.com/news/2014/apr/28/new-hampshire-ranks-8th-in-high-school-graduation/

Circle Program Presentation

Documents

Transcript of Circle Program Presentation