Instructor: Bess A. Rose

Photograph by Quentin Bacon Gourmet Magazine, December 1999 http://www.gourmet.com/recipes/1990s/1999/12/minibrownies

Brownies are outcomes! Or maybe out-

YUMS!

Given information from sources such as program designers’ web site, program curriculum materials, experts, and research-based materials, participants will analyze information and identify outcomes as designed by program creator and experts and theorized in research with 100% accuracy.

Logic Model Template

Inputs – resources (funds, staff, materials), activities, and strategies

as designed by program creator and experts and theorized in research (using credible, accurate, reasonable and supported sources)

based on implementation o using a method for

identifying, recording, maintaining and updating information on all program sites in Maryland

o using a tool for conducting site visits of programs in Maryland and observing and recording program activities in a manner that is objective and reliable

Outcomes – short-term, intermediate, and long-term changes in knowledge, attitudes, skills, and behaviors

as designed by program creator and experts and theorized in research

expected by program implementers (using interviews)

expected by key stakeholders (using surveys)

stated in grant applications

Relationship – a logical explanation of how the inputs lead to the outcomes

is there a direct cause-and-effect relationship?

what assumptions are implied?

will this relationship be true in any context?

what factors might confound this relationship?

Program Name and Information

Today we shift over to OUTCOMES

Definition: Short-term, intermediate, and long-term changes in knowledge, attitudes, skills, and behaviors

The measurable effects that a program is intended to produce.

as designed by program creator and experts and theorized in research

expected by program implementers (using interviews)

expected by key stakeholders (using surveys)

stated in grant applications

After-school tutors provide one-on-one assistance in reading that follows research-based best practices, is aligned with classroom instruction, and is tailored to specific, identified student needs based on diagnostic testing.

“This program supports the creation of community learning centers that provide academic enrichment opportunities during non-school hours for children, particularly students who attend high-poverty and low-performing schools. The program helps students meet state and local student standards in core academic subjects, such as reading and math; offers students a broad array of enrichment activities that can complement their regular academic programs; and offers literacy and other educational services to the families of participating children.”

What is achieved?

NOT what is provided – not services, activities, instruction – these are inputs

Watch out for confusing language – they might word inputs as outcomes and vice versa

Attitudes, abilities, performance, knowledge, skills

“This program supports the creation of community learning centers that provide academic enrichment opportunities during non-school hours for children, particularly students who attend high-poverty and low-performing schools. The program helps students meet state and local student standards in core academic subjects, such as reading and math; offers students a broad array of enrichment activities that can complement their regular academic programs; and offers literacy and other educational services to the families of participating children.”

Students who participate in after-school tutoring sessions will:

improve on the standardized assessment

receive good classroom grade in reading

Each student who participates in at least 80% of assigned after-school tutoring sessions by the end of the first year will:

improve by 25 points on the standardized assessment

receive classroom grade of at least a B in reading

Look for:

Who – exactly (all?)

What – measurable?

When – timeline?

How much? Growth vs. standard?

ESEA: 21st Century Community Learning Centers (OESE)

FY 2007 Program Performance Report (System Print Out) Strategic Goal 1 Formula ESEA, Title IV, Part B Document Year 2008 Appropriation: $1,081,166 CFDA 84.287: Twenty-First Century Community Learning Centers

Program Goal: To establish community learning centers that help students in

high-poverty, low-performing schools meet academic achievement standards; to offer a broad array of additional services designed to complement the regular academic program; and to offer families of students opportunities for educational development.

Objective 1 of 3: Participants in 21st Century Community Learning Center programs will

demonstrate educational and social benefits and exhibit positive behavioral changes.

Measure 1.1 of 14: The percentage of elementary 21st Century regular program participants whose mathematics grades improved from fall to spring. (Desired direction: increase) 1032

Year Target Actual (or date expected) Status

2000 43 Measure not in place 2001 43 Measure not in place 2002 45 41 Did Not Meet Target 2003 45 43 Made Progress From Prior Year 2004 45 43 Did Not Meet Target 2005 45 39.65 Did Not Meet Target 2006 46 19.24 Did Not Meet Target 2007 47 41.76 Made Progress From Prior Year 2008 47.5 (October 2009) Pending 2009 48 (October 2010) Pending 2010 48.5 (October 2011) Pending

Source. U.S. Department of Education, OESE, 21st Century Community Learning Centers, grantee performance reports.

Frequency of Data Collection. Annual

Data Quality. Data supplied by grantees

Explanation.

In the reporting of grades data, some centers (particularly community-based and faith-based organizations) serve students that attend schools that do not provide grades to the sub-grantee to be reported in the APR reporting process. Some afterschool centers that operate in the summer do not provide grades or teacher survey information. Approximately 84% of the centers report grades.

Currently, program staff members are working with States to explore ways that data can be provided in a consistent way.

GPRA measure 1.1: The percentage of elementary 21st Century regular program participants whose mathematics grades improved from fall to spring.

Use the sources for your program you used for inputs, such as program designers’ web site, program curriculum materials, experts, and research-based materials (electronic versions make searching easier!)

Analyze text to identify outcomes - Short-term, intermediate, and long-term changes in knowledge, attitudes, skills, and behaviors; the measurable effects that a program is intended to produce.

Start to capture the outcomes in your logic model diagram

(1) Continue class activity with further resources on your list and post the results to Kazoo or Facebook.

(2) In your journal, reflect on the extent of agreement or disagreement about outcomes across the resources and share your reflections in the discussion on Kazoo or Facebook.

Lesson 8: Identifying Expected Outcomes from Program Implementers

Monday, August 10 2:00 – 3:00 p.m. 8th floor CR4

Objective: Given notes from interviews with program implementers, participants will analyze findings and identify expected outcomes with 100% accuracy.

After Monday’s class, conduct at least one phone or face-to-face interview using questions and techniques developed in class.