1 of 19Evaluating an Information ProjectDefining Content

© FAO 2005

IMARK Investing in Information for Development

Evaluating an Information Project

Defining Content

2 of 19Evaluating an Information ProjectDefining Content

Learning ObjectivesLearning Objectives

At the end of this lesson you should be able to:

use a Logic Model to identify the if-then causal relationships in your project;

understand how this Logic Model can be used to decide what you want to measure.

3 of 19Evaluating an Information ProjectDefining Content

IntroductionIntroduction

In order to examine the issues involved in the planning of an evaluation, we are following a scenario: Dr Kumar, the Director of publications at SMAU University, must evaluate a newsletter.

Dr Kumar and his colleagues have held the first Evaluation Management Committee meeting, to reach an agreement on the focus of the evaluation.

The EMC conducted a preliminary analysis of the RICE NEWS project, examining Needs, Beneficiaries, Activities, Outcomes.

Also, a SWOT analysis identified potential risks and benefits of the evaluation.

The next step will be to define what to measure, and how.

4 of 19Evaluating an Information ProjectDefining Content

Deciding what to measureDeciding what to measure

During the EMC meeting, we agreed that an impact of RICE NEWS is “more farmer knowledge about fertilizer applications”.

How can we measure that? Should we ask how many farmers have read RICE NEWS?

Or how many farmers actually knew more after reading it?

5 of 19Evaluating an Information ProjectDefining Content

Deciding what to measureDeciding what to measure

It is important to decide first what to measure, and then how. Therefore:

1. agree on what issues to cover in the evaluation (e.g. farmer knowledge)

2. agree on what data to collect and how to do it.

...we can use something called a “Logic Model”..

6 of 19Evaluating an Information ProjectDefining Content

What is a Logic ModelWhat is a Logic Model

A Logic Model is a tool that helps understand the if-then relationships in our project.

We can use it to understand “what leads to what”...

7 of 19Evaluating an Information ProjectDefining Content

What is a Logic ModelWhat is a Logic Model

What inputs in the project have led to what activities?

What activities have led to what outputs?

What outputs have led to what impacts (or outcomes)?

8 of 19Evaluating an Information ProjectDefining Content

The Usefulness of a Logic ModelThe Usefulness of a Logic Model

These are some of the questions that a Logic Model can answer:

Are we clear as what the project was supposed to achieve?

What do we really want to measure? Are we focusing on the right things?

Are we likely to be able to collect good data about the issues we want to include in the evaluation?

9 of 19Evaluating an Information ProjectDefining Content

Analyzing the if-then relationships in the projectAnalyzing the if-then relationships in the project

To build a Logic Model, start by preparing some worksheets:

•the first one deals with the relationship between inputs and activities;

•the second with activities and outputs; and

•the third with outputs and outcomes.

10 of 19Evaluating an Information ProjectDefining Content

Analyzing the if-then relationships in the projectAnalyzing the if-then relationships in the project

Let’s take the first worksheet: inputs and activities.

What resources have been used for RICE NEWS?

What activity has each input contributed to?

11 of 19Evaluating an Information ProjectDefining Content

Creating your own logic modelCreating your own logic model

Let’s think about our resources, or inputs, one at the time. Let’s start

with funds coming from the Ministry of Agriculture...

...we used that money to buy some equipment for the production of RICE NEWS

12 of 19Evaluating an Information ProjectDefining Content

Creating your own logic modelCreating your own logic model

This is the first table: inputs and activities.

SOURCE TYPE OF INPUTS CONTRIBUTION TO WHICH ACTIVITIES?

Ministry of Agriculture Funds Purchase of publishing equipment

Mikuni State Agricultural University

Publication professionals

The Newsletter production process

Mikuni State Agricultural University

Scientists Research

The extension department

Extension agents The Newsletter distribution

13 of 19Evaluating an Information ProjectDefining Content

Creating your own logic modelCreating your own logic model

This first table is useful, to:

evaluate if the RICE NEWS project has been organized efficiently (a “performance” evaluation);

focus more on “impact”, and see how inputs and activities have contributed to outcomes. It will be necessary to use this first table, as well as the next two.

14 of 19Evaluating an Information ProjectDefining Content

Creating your own logic modelCreating your own logic model

Now the second table.

For each activity noted in the first table, let’s ask:

What has been the main output?

Who have been the main beneficiaries of that output?

15 of 19Evaluating an Information ProjectDefining Content

Creating your own logic modelCreating your own logic model

Here's the second table: activities and outputs.

ACTIVITIES OUTPUTS BENEFICIARIES

Research. Articles submitted to RICE NEWS.

Articles accepted.

The UniversityScientists

The Newsletter production process.

Number of trained staff.Participating students.

The Publication Department.Students.

Commercial Firms.

Purchase of publishing equipment.

Number of PCs, printers, presses etc.

The University.The Publication Department.

The Newsletter distribution.

Number of copies distributed.

Extension agents.Farmers.

Agribusiness.

16 of 19Evaluating an Information ProjectDefining Content

Creating your own logic modelCreating your own logic model

And finally, the third table. Outputs and Intended Impacts (or outcomes):

OUTPUTS INTENDED IMPACTS

Articles submitted to RICE NEWS.

Articles accepted.

More relevant research.More scientists promotions.

Number of trained staff.Participating students.

More farmer knowledge of new developments in rice

production practices.

Number of PCs, printers, presses etc.

More cost-efficiency in the production of RICE NEWS

Number of copies distributed

Increased farmer commitment to put

knowledge into practice.

17 of 19Evaluating an Information ProjectDefining Content

Creating your own logic modelCreating your own logic model

A Logic Model for planning can help separate activities, from outputs, from impacts (or outcomes). By doing so, you can decide whether the main focus of your evaluation will be on performance, or impact, or both. 

18 of 19Evaluating an Information ProjectDefining Content

Creating your own logic modelCreating your own logic model

 You have your Logic Model. You are ready to use this Logic Model as a basis for formulating your detailed evaluation questions.

19 of 19Evaluating an Information ProjectDefining Content

Summary

To agree on the detailed content of an evaluation, it is useful to build a “Logic Model”.

A Logic Model is a tool that helps understand the if-then relationships in a project.

A Logic Model can therefore help focus on what you really want to measure.

The major benefit of using a Logic Model for planning your evaluation is to separate activities, from outputs, from impacts (or outcomes).

It can also help evaluate if the objectives of each of these areas have been met.