Introduction to Developing, Monitoring and Evaluating Projects

Noah Keuzenkamp

TGEU Capacity Building Officer

noah@tgeu.org

Overview

Project development Monitoring & Evaluation Putting both together: LogFrames Group Exercise Presentations A few points to keep in mind Questions

Baseline

Who has developed a project before?

Who has written a fundraising application with a monitoring & evaluation plan before?

Who has monitored a project before?

Who has evaluated a project before?

Who knows what a LogFrame is?

Thinking about project development

with a logic model

Key terms

Goal (Impact)

Outcomes

Outputs

Activities

• The outcome at the end of the project – often the expected benefits to the target group(s)

• Key words: “increased, improved, etc.”• Example: Participants have increased knowledge of M&E

• The broad impact to which the project contributes – e.g. at the national level or over the long-term. Your project may not achieve this immediately/by itself.

• Example: The trans* community obtains more funding OR: To build the capacity of the trans* community to obtain funding

• The direct/tangible results (goods &services) that the project delivers (mainly under your control)

• Key words: “delivered, produced, conducted, etc.”• Example: M&E workshop delivered to 20 participants

• The main tasks carried out to eventually deliver the outputs• Key words: “prepare, design, develop, research, etc.”• Example: Develop a workshop on M&E

Key terms: another example

GoalThe broad impact to which the project contributes:Trans* concerns are better represented in politics

Outcomes Expected outcome/benefit at the end of the project:Trans* rights were mentioned 5 times in parliament

OutputsDirect results delivered:Briefings delivered to 30 parliamentarians

ActivitiesTasks carried out: Identify friendly parliamentarians and organise meetings P

LA

NN

ED

INT

EN

DE

D

Thinking about project development

with a logic model

How do we know we are approaching our goal?

Monitoring:

the systematic and routine collection of information during the implementation of a project to assess whether the project is progressing as planned

Evaluation:

a systematic and ‘objective’ assessment of a completed project, especially its outcomes and impact, in order to inform future strategic decisions and projects

Development, Monitoring & Evaluation in the Project Cycle

Why conduct M&E?

In practice often:

Because it is a donor condition

To show that you are doing what you are funded for

To obtain more funding in the future

But, it can also:

Help you learn and improve

Help you understand how change works (or doesn‘t)

Help you create, preserve and disseminate knowledge

Make you accountable to your communitie(s)

Logical Framework Matrix (LogFrame)

Helps you systematically develop, plan, implement, monitor and evaluate a project

Presents clear and organised connections between impact, outcome, outputs and activities

Many donors use/require (a kind of) LogFrame

Using a LogFrame

Project Summary

Indicators Means of Verification (MoV)

Assumptions

Goal

Outcomes

Outputs

Activities

If...then

Monitoring & Evaluation

Horizontal logic

Project Summary

Indicators Means of Verification (MoV)

Assumptions

Goal Variables/ information used to measure change in a process or phenomenon

The source of the indicators and the means of obtaining them.

= M&E method for obtaining information

Assumptions that need to be fulfilled to eventually achieve your goal

Outcome

Output

Activity

Indicators: examples

„More positive public attitude towards trans* people“

Increase in number of positive news items Decrease in number of hostile statements by politicans Increase in positive attitude of individuals (survey) Gender identity is included in anti-discrimination laws ... Etc.

Indicators: examples

„Trans* people are more visible in society“

... ... ...

Indicators: SMART

S Specific E.g. nr of people, % increase

M Measurable Can it be measured in general? Can you measure it?

A Achievable Is this realistic?

R Relevant Does it measure the ‘concept’? Or: reliable

T Time-bound By when or from when to when

Not SMART: People will know more about M&E

SMART: By the end of the workshop, 70% of people who didn‘t know what a LogFrame is report that they know „fairly well“ how to fill in a LogFrame

Means of verification (Mov)

Method for collecting indicators to conduct monitoring and evaluation, e.g.

Surveys / Questionnaires Focus groups Observation Official statistics News reports Photos Testimonies / stories Etc...

Project Summary

Indicators Means of Verification (MoV)

Assumptions

GoalThe trans* activist community is well funded

Income of participating organisations rises by 20% by 2016

Online survey - M&E plans are key in grant writing-Participants will write grants

Outcomes 70% of participants have increased knowledge of M&E

Nr. of participants who report having gained knowledge

Questionnaire handed out after session (self-report)

LogFrames are not too complex

Outputs Workshop on M&E delivered to 20 people

Nr. of participants

Sign-in sheet People are interested in M&E

Activities Develop a workshop on M&E

InputsStaff, time, technology, etc.

CostWork time, travel, accommodation etc. = ?

PreconditionsTGEU has capacity to deliver

This workshop: full example

Noah‘s ArkProject Summary

Indicators Means of Verification (MoV)

Assumptions

Goal Earth remains populated by people and animals

Census count 100 years after the flood

Census survey - Plant life returns- All animal pairs can/want to reproduce

Outcomes All people and animals have survived the flood on the Ark

Census count after landing

Checklist upon unloading

- No one will die from illness- Food will not run out

Outputs Ark is loaded with people and pairs of all animals

Census count on the Ark

Checklist upon boarding

Animals can be found and will happily get on the Ark

Activities

Build ark

InputWoodNoah’s time and skills

CostBarter

PreconditionsCan find a safe plot of land to build on

Sometimes it‘s better not to be Noah

When developing a new project, resist the temptation to (always) start with the Ark, i.e.(specific) activities:

„We should run a training, because that‘s what we usually do“

Starting with the goal or the problem and then working from top to bottom is better for productive brainstorming and creative problem solving

Working from top to bottom also helps identifying the best target groups or stakeholders

Use the LogFrame as a tool for logical and creative thinking

Group Exercise

Brainstorm about what the main problems are Your project only needs to contribute to the goal Complete 1-2 outcomes/outputs/activities first, then add

more if you have time Resist the temptation to start with the activities Write your LogFrame on flipchart paper Agree on presenter(s)

Time for preparation: 30 mins

A few points to keep in mind

When writing a grant proposal Language matters (concise and precise) Terminology choice matters (what does the donor

use?) Describing activities: usually active verbs and present

tense, e.g. „compile a factsheet“, „perform a play“ Describing outputs and outcomes: usually past tense,

e.g. „confidence has increased“, „question has been raised in parliament“

A few points to keep in mind

Don’t let the language of donors confuse you

e.g. outcome = purpose / (intermediate) objective / result

The same outcomes can be measured with different indicators: chose the one(s) best suited to your goals and resources (money, time, skills, technology, …)

Evaluation does not always have to be quantitative: qualitative data, such as personal stories, can be used, too

M&E is political: who determines what the outcomes should be? who is asked for their opinion? who conducts the evaluation? ( feminist evaluation)

Questions?

Noah Keuzenkamp

noah@tgeu.org

What should be SMART?

SMART outcome with a variable as indicator:

Outcome: 80% of participants have increased their knowledge to a „good“ level by the end of the project

Indicator (variable): Number of participants with „good“ knowledge

Outcome with a SMART indicator:

Outcome: Participants‘ knowledge is increased

Indicator: 80% have a „good“ level of knowledge by the end of the project

Concept vs Indicators

Trans* people are SAFER in our city

1: Trans* people‘s report of how safe they feel

2: Nr. of attacks on

trans* people

reported to police

3: The nr. of anti-Pride protestors

4: Nr. of trans* customers at LGBT bars

Indicators

Sometimes they are direct:

Planned outcome: A trans* person gets elected into parliament

Indicator: Did a trans* person get elected into parliament?

But usually they are indirect:

Especially when talking about social change, attitudes, knowledge, awareness, skills, etc.

Example: this workshop

I could monitor: Baseline: what was people‘s knowledge before? Status: how many people are in the room? How many

people are listening? Trend: are people leaving? Switching off? Getting

more engaged? Implementation: is the workshop being implemented?

Are there interruptions? (Ongoing) effectiveness: are people learning

something right now?

Example: this workshop

I could evaluate: Efficiency: was a workshop the best/cheapest method

for people to learn about M&E? Effectiveness: did people learn about M&E? Outcome/Impact: do people feel better prepared to

complete an M&E plan? will they use the knowledge when writing funding applications? will they get more funding?

Sustainability: will people still remember this in a year? will they pass the knowledge on?

What is a project?

„An individual or collaborative enterprise that is carefully planned to achieve a particular aim” (Oxford Dictionary)

Has a start and end date Has constraints (time, money, resources,...) Aims to create change

Introduction

“I would like to have money for a project to…”