Evaluation BasicsDefining Evaluation

Evaluation is an important strategy in achieving program quality and impact

Evaluation means weighing the value of…actions that go into youth programs

(processes)results of youth programs (outcomes)

Evaluation BasicsPutting Evaluation in Perspective

Evaluation brings together many streams or processes and is often most visible at the end of a program. But the truth is that evaluation, understanding the value of a program…

Begins before a program ever beginsSpeaks to youth as much as to evaluatorsRelates to marketing and recruitment as much

as to accountability and program improvement

Evaluation BasicsProcesses in Perspective

Processes related to evaluation include…

Evaluation logic, describing valued outcomes and strategies to reach them

Best practices and conditions for quality programsProgram development and organizational mission,

framing program priorities Program evaluation process from planning through

reporting results

That’s a lot of ideas, but we will consider them one at a time!

Evaluation BasicsProcesses of Evaluation

Evaluation answers questions about…

actions, processes, or quality of programsIs the program running as designed?What elements make the most difference?

results, outcomes, or impacts of programsHow much difference did the program make?What audience benefits the most?

Evaluation BasicsEvaluation Logic

…and that quality and impact doesn’t happen by chance…

Positive outcomes result from quality programs

Evaluation BasicsEvaluation Logic

For example, when a 4-H robotics club sets individual and club goals, Club leaders and youth ask…

What do we need to know or do?

(the impact question)

What’s the best way to do it?(the quality question)

Evaluation BasicsEvaluation Logic

A 4-H robotics club might value outcomes related to…

science and technologyteamwork, goal-setting, problem solvingserving their community

Evaluation BasicsProcesses of Evaluation

Science and technology outcomes involved…building a robotprogramming a robot to perform

Teamwork outcomes involved…working together in different roles

Research outcomes included…investigating a topic developing a creative presentationusing learning to serve the community

Evaluation BasicsProcesses of Evaluation

Outcomes, in the short term, describe results of a program, including changes in…

Knowledge—describe or explain an idea, object

Attitudes—thinking or feeling about events

Skills—abilities to do somethingAspirations--plans to do something

Evaluation BasicsProcesses of Evaluation

Outcomes beyond the short-term describe results of a program, including changes in…

Applications—using knowledge or attitudes in new situations, to learn more, or teach others

Practices—using skills, often with new attitudes, to perform at a higher level

Evaluation BasicsProcesses of Evaluation

Outcomes for the long-term describe hoped-for changes in the quality of a community life…

Social—relationships within the communityEconomic—work and business lifeEnvironmental—physical and natural

settingCultural—shared life and culture

Evaluation BasicsEvaluation Logic

If they put the resources and strategies, practices and conditions, short and long-term outcomes in order….

…they would create a Logic Model, or description of the logical sequence of actions and results…

Inputs Outputs

Short-termOutcomes

Mid-termOutcomes

Long-termOutcomes

Robot kitSkilled leaderShop to work

TrainingPractice

Skills in building, programming

Problem solvingCritical thinking

Sci-tech savvy workers

Evaluation BasicsBest Practices and Conditions

The value of the club’s work is not just in the results, but in the process, or how they work, learn, and play together, including…

hands-on learning through projects and trips

youth leadership with adult supportrecreation and fun

Evaluation BasicsBest Practices and Conditions

Process Evaluation looks at best practices that define quality in youth programs such as…

Hands-on learning through projects and trips-Ex: When is hands-on learning most effective?-Ex: What was learned/used from a field trip?

Youth leadership with adult support-Ex: Did young leaders have adequate

support?-Ex: How did leadership roles enhance

performance in a competition?

Evaluation BasicsBest Practices and Conditions

Process Evaluation looks at best practices that define quality in youth programs such as…

Teamwork in learning and working together-Ex: How do members fulfill/mesh roles?-Ex: (How) does the team work as one unit?

Creative Presentation-Ex: Is delivery audible, clear, rehearsed?-Ex: Does team show enthusiasm,

creativity?

Evaluation BasicsBest Practices and Conditions

Process Evaluation also looks at best practice conditions that promote quality programs, such as…

Safety, physical and emotionalSupport from peers and adultsSkill building opportunitiesSelf-confidence-buildingSocial norms promoting responsibility and

serviceSocial inclusionStructure balancing discipline and independenceSynergy with family and community influences

Evaluation BasicsProcesses of Evaluation

…more processes to follow…