Monitoring & Evaluation

36
World Change Starts with Educated Children. ® Monitoring & Evaluation

Transcript of Monitoring & Evaluation

World Change Startswith Educated Children. ®

Monitoring & Evaluation

World Change Startswith Educated Children. ®

Presentation Flow

• Glimpse on M&E • M&E at RtR • GI & CI • GSD• Data Trail for Quality Assurance • Ground Rule – There is no Ground Rule is the

Rule Every One is encouraged to

A(lert) S(eek) K(now)

World Change Startswith Educated Children. ®

• When or Where is the Last Time you have used an M&E Mechanism in Your

LIFE Other than Programme

World Change Startswith Educated Children. ®

TERMINUS

Where Does It Stand Now ?

World Change Startswith Educated Children. ®

Authorized Service Centre ?

Passengers ?Driver ?

Time-keeper at the

terminus ?

Whose Concern ???

World Change Startswith Educated Children. ®

Driver

The car is in good running condition !!

Road is good !! Running at good

speed !!Passengers

Authorized Service Centre

Time-keeper at

the terminus ?

No Jerking !! Feeling Comfortable !!

Driver is not drunk !Is the car good enough

for this kind of a

road ??

Need to redesign some

of the parts ??

How

far

is the

car ??

How

long

will

it tak

e ??

Who are

they ?

What Concerns

World Change Startswith Educated Children. ®

DriverPassengers Authorized

Service Centre

Time-keeper at

the terminus ?

Who monitors ? Who evaluates ?

Is monitoring and evaluation same ?

World Change Startswith Educated Children. ®

Monitoring & Evaluation (As defined by World Bank)

Monitoring Evaluation

A continuing function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing development intervention with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds.

Thus monitoring embodies the regular tracking of inputs, activities, outputs, outcomes and impacts of development activities at the project, program, sector and national levels. This includes the monitoring of a country’s progress against the millennium development goals (MDGs), or other national measures of development success.

the process of determining the worth or significance of a development activity, policy or program ….. to determine the relevance of objectives, the efficacy of design and implementation, the efficiency or resource use, and the sustainability of results. An evaluation should (enable) the incorporation of lessons learned into the decision-making process of both partner and dono

World Change Startswith Educated Children. ®

Indicator& DataIndicator

A quantitative or qualitative variable that provides a valid and reliable way to

measure achievement, assess performance, or reflect changes , Progress

connected to an intervention.

An indicator provides a sign or a signal that something exists or is true. It is

used to show the presence or state of a situation or condition. In the context of

monitoring and evaluation, an indicator is a quantitative metric that provides

information to monitor performance, measure achievement and determine

accountability.

Data

specific quantitative and qualitative information or facts that are collected and

analyzed thru indicators .

World Change Startswith Educated Children. ®Indicator Designing Flow Chart

World Change Startswith Educated Children. ®

Types of IndicatorsMost of the indicators are classified as Quantitative or Qualitative Indicators which are used as per the programme needs . There are different school of thoughts on what should be nature of an Indicator

SMART SPICED

S- Specific S-Subjective

M-Measurable P-Participatory

A- Achievable I-Interpreted and Communicable

R- Realistic/ Relevant C-Cross Checked and Comparable

T- Time Bound / Timely E-Empowering

D-Diverse and Disaggregated

Source : http://www.toolkitsportdevelopment.org/html/resources/40/408CC56F-509A-40D8-BE46-D7EEB4261F97/10%20Indicators.pdf

World Change Startswith Educated Children. ®

Qualities of a Good Indicator

UsefulnScientifically Robust Valid Reliable Sensitive Specific

Understandable

Accessible

Ethical

Yep!!!!!!!!

My Indicators are GOOD Indicators

World Change Startswith Educated Children. ®

Input Process Output Outcome / Impact

Fund Flow

TrainingSupplies

StrategyService PackageActivitiesAdvocac

y

Management Targets

Quantitative &

Qualitative

Change in the

CommunityQuantitative & QualitativeOrganization/ National Goal

Monitoring Evaluation

System Box

World Change Startswith Educated Children. ®

1. Know : what you

are suppose

d to achieve

2. Know : what you are

supposed to do

3. Selection

of key indicators

4. Data Collection

5. Analysis and

Interpretation

6. Action7. Share

Steps

World Change Startswith Educated Children. ®

Data

Dat

a

Information

*&^+

Books / bOys/ Girls !@#$%

Books Related to RR

Girls Related to GEP

Raw Vegetable

s

Eatable Cooked

Food

Analysis

Presentation

In 2012 , 850 RRs have been established in 8

states.

3000 Girl Scholars are supported with

Academic Support

World Change Startswith Educated Children. ®

Whether things are in order

Whether desired changes take place in the community

Continuous Usually at one or two years interval

Internal Internal /External

Corrective Effectiveness of strategy

Management Decisions Lessons Learned

MIS Data /Checklists ~ Registers, Forms, reports

Population Based Sample Survey/Cohort Analysis/ Participatory Appraisal/ Civil Registration System

How Many ? What %age ?

Monitoring Evaluation

World Change Startswith Educated Children. ®

Monitoring and Evaluation Framework

Process data-4

Analyze data-5

Ensure data quality-3

Show results-6

Share results-7

Receive feedback from

users-8

Change policies and

perspectives-9

Change actions-10

Identfy issues for examination- and

Define Role as per program Demand- 1

Collect data-2

Ideal Flowof Information

in a Functioning PerformanceMonitoring

System

World Change Startswith Educated Children. ®

Role of RM&E in Program Cycle

Baseline/Research/Assessment

Planning

Implementation &

Monitoring

Evaluation

World Change Startswith Educated Children. ®Project Data Flow

AP&B Document

Targets as Forecast Status

General Ledger

GSD

MIU/Quarterly Report

Global Indicators Reports

Donor Report

Budget

Project Progress (standard & data quality) Reports

Finance Reports

World Change Startswith Educated Children. ®

Program Monitoring SystemRtR

RtR PM/PD/CD

Field Facilitator

Moblizer/LMC/SMC

• Partner NGO• District /PA

PO/PA

RtR- RO/GO

Feed Back

1

2

3

4

Reporting

5

World Change Startswith Educated Children. ®

Goals, Objectives, and Indicators

• All programs have global goals, objectives and indicators

• All countries are collecting global indicator data for all programs & projects since 2008

• This data gets into the Global Solutions Database (GSD) for GI reporting

• Data Quality reports to check errors, outliers, exceptions etc.

World Change Startswith Educated Children. ®

Summary of GIs

PROGRAM NO. OF GIS QTRLY

ANNUAL

RR 11 5 6

SR 9 2 7

GEP 11 2 9

LLP 6 2 4

ALL PROG 37 11 26

World Change Startswith Educated Children. ®

Summary of CIs

PROG

BD KH IN LA NP SA LK VN ZAM

RR 7 2 5 8 9 5 1    

LLP 6 2   3 3     3  

SR 2 2   3 3   2    

GEP 3+6 5+5 7+11 2+2 7+8   5+5 3+4 6+4

ALL PRO

G

24 16 23 18 30 5 13 10 10

World Change Startswith Educated Children. ®

RR GIs & CIs

Indicator Frequency

RR1 Number of libraries established Quarterly

RR2 Number of local language and/or English books provided to libraries Quarterly

RR3 Percentage of libraries where school personnel are trained by Room to Read and/or a Room to Read partner

Quarterly

RR4 Average number of days project sites receive training (REVISED GI) Quarterly

RR5 Number of books checked out by library users Quarterly

RR6 Number of targeted children enrolled in schools with a RtR library Annually

RR7 Number of targeted teachers at schools with a RtR library Annually

RR8a Average number of minutes per week each student/class can access the library during designated library/reading periods

Annually

RR8b Average number of minutes per week each student/class can access the library during non-designated library/reading time

Annually

RR9 Percentage of libraries with book classification systems in place Annually

RR10 Percent of libraries where books are classified according to reading level (NEW GI)

Annually

RR11 Percentage of libraries that have functional check-out systems Annually

World Change Startswith Educated Children. ®

•Percentage of RRs where the reading period is held regularly •Percentage of RRs where the CMC is trained •Percentage of RRs which have active CMCs •Percentage of RRs where the community has contributed in terms of events – organization and material support •Percentage of RRs where the students are checking out books as per the reading levels

RR CIs

World Change Startswith Educated Children. ®

GEP GIs & CIs

Indicator Frequency GEP1 Number of GEP participants in 2011 QuarterlyGEP2 Percentage of GEP participants who dropped the program in 2011 QuarterlyGEP3 Number of GEP participants who completed the final year of the program in

2011Annually

GEP4 Percentage of eligible GEP participants who advanced to the next grade AnnuallyGEP5 Percentage of eligible GEP participants who transitioned school levels AnnuallyGEP6 Number of GEP participants who received academic support AnnuallyGEP7 Number of GEP participants who attended life skills training AnnuallyGEP8 Percentage of GEP participants whose guardian(s) has participated in Girls

Education meeting(s)Annually

GEP9 Percentage of GEP participants who received mentoring and stayed in school each school year (NEW – for Enhanced approach only)

Annually

GEP10 Percentage of GEP participants who received material support and stayed in school each school year (NEW – for Enhanced approach only)

Annually

GEP11 Percentage of GEP participants in targeted grades who participated in life skills activities

Annually

World Change Startswith Educated Children. ®

GEP CIs

Percentage of girls transition from 5 to grade 6 (annual)Percentage of girls transition from 8 to grade 9 (annual) Percentage of girls transition from 10 to grade 11 (annual) Percentage of girls with less than 50% attendance (monthly) Number of girls participated in development camp (annual) Percentage of girls who went on exposure visit (annual)Percentage of field staff (SMs/PCs) trained on thematic areas (annual)

World Change Startswith Educated Children. ®Annual M&E Activities (Typical)

• Need Assessments to identify new project sites• Site visits by field staff to monitor progress & provide

hands-on support plus feedback to PO/PA & above• Covering all active project sites at least once a quarter• Monthly and Quarterly Reviews by RtR and NGO staff to

examine plans and progress/identify and overcome bottlenecks

• Pilot studies to test new program approaches• Country specific research/evaluations as needed• Annual/quarterly collection of GIs and entry into GSD• Measurement and sharing of CI progress with program

teams and initiate corrective measures• Verify GSD data quality during field visits with source

information on project progress

World Change Startswith Educated Children. ®

GSD purpose and processes

• Global Solutions Database (GSD) a Web based program management system of salesforce.com

• All program staff expected to use GSD are indicated by HR and given password by Administrator

• GSD power user in a country is the first point of contact with support from Database Administrator at RO

• Each GSD user is expected to:– Keep login active– Use reports for review and action– Update information as necessary– Check data quality reports– Share /discuss results with team members

World Change Startswith Educated Children. ®M&E and the GSD

• Complementary roles in program management

• M&E caters to local CO needs while supporting GSD to meet global needs of information gathering and reporting

• Field information through M&E System flows into GSD

• Both mutually contribute to program quality improvement

World Change Startswith Educated Children. ®Data Quality

“It is the process of ensuring that the Data flow from source or the first point of generation to the final source is same, consistent , accurate and assured Corrective in data”

World Change Startswith Educated Children. ®Example

World Change Startswith Educated Children. ®Report Sent by NGO - SO

ALARIPPU, HaridwarProject year- 2009 RR (Phulwari) Usage of books Month :- October - December 2011 Block- Bhagwanpur

S.N. Name

of the schools

Check in and Check Out DetailBy Children By Teachers By Community TOTAL

Books Readers Books Readers Books Readers Books ReadersDec

Bhudhwa Shaheed

646 59 28 2 216 23 890 84

Nov

Bhudhwa Shaheed

555 61 0 0 175 17 730 78

Oct

Bhudhwa Shaheed

311 56 0 0 52 9 263 65

1512 176 28 2 443 49 1883 227

World Change Startswith Educated Children. ®

World Change Startswith Educated Children. ®

Any Questions????

World Change Startswith Educated Children. ®

Thank Q………