Evaluation Technical and Financial Plan Revised

Evaluation Plan Impact of KLIP activities on governance and policy making in Kisumu

Date: 21st July 2014

Trim Ref #: 09/

Evaluation Manager:

Contact Person: Name:

Position title:

Contact Phone:

Business Unit:

Table of Contents

Contents

1. Background ......................................................................................................................... 1

1.1 Project/Program/Initiative Definition ........................................................................................... 1

2. Evaluation Governance ........................................................................................................ 1

2.1 Purpose of the Evaluation .......................................................................................................... 1

2.2 Core Evaluation Questions ......................................................................................................... 2

2.3 Evaluation Stakeholders ............................................................................................................. 2

2.4 Evaluation Management Structure ............................................................................................. 3

2.5 Evaluation Governance Group Membership .............................................................................. 3

2.6 Evaluation Governance Group Terms of Reference .................................................................. 4

2.7 Reporting Requirements ............................................................................................................. 4

3. Evaluation Design ................................................................................................................ 5

3.1 Evaluation Framework ................................................................................................................ 5

3.2 Information and Collection .......................................................................................................... 5

4. Evaluation Resources and Timeframe ................................................................................. 5

4.1 Budget and Staff Resources ....................................................................................................... 5

4.2 Evaluation Staff Roles and Responsibilities ............................................................................... 7

4.3 Time Schedule ............................................................................................................................ 7

Appendix 1 Evaluation Plan Checklist ........................................................................................... 11

Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900

1

1. Background

1.1 Project/Program/Initiative Definition

KLIP is part of a consortium of organizations funded by Swedish International

Development Cooperation Agency (Sida) and Mistra Urban Futures (M-UF) with the sole

aim of promoting sustainable urban development. KLIP therefore strives to achieve the

set out goals of Fair, Green and Dense city of Kisumu by incorporating two thematic

pillars of: a) Eco-Tourism and b) Market Places. The two themes aims at addressing

urban challenges like food insecurity, climate change, poverty, urban insecurity among

others within Kisumu city and its environs. The partnership between Kisumu LIP and

participating stakeholders seeks to overcome the threats to food security, urban

insecurity, and skewed distribution of factors of production in a changing environment;

exploring new ways of helping vulnerable urban communities adjust to global changes

and explore other factors of production other than fishing along the adjacent beaches.

This consultancy exercise is a result of the collaboration between KLIP Sida and Mistra-

Urban Futures and its main purpose will be to evaluate the flagship programs of Eco-

tourism and Market places under the perspective of the various stakeholders, who

include KLIP Trustees, Kisumu Local Government Leaders, Various Private Sector

organizations and the local residents. The whole process will establish the impact of the

activities since their inception and also model future activities based on the results

obtained.

2. Evaluation Governance

2.1 Purpose of the Evaluation

The purpose is to identify the societal impact that has been achieved so far, or has the

potential to be achieved, through the support of KLIP project and capacity building

activities


2

2.2 Core Evaluation Questions

In fulfilling the purposes of the evaluation, the following core questions were addressed:

How has the activities carried out by KLIP, influenced governance and policies of

collaborating public and private sector organizations?

How has KLIP impacted in the society so far, and what can it achieve in the future?

How can the programs in KLIP be further improved for achieving the organization’s vision and

mission?

2.3 Evaluation Stakeholders

KLIP TRUSTEES

Under the banner of SIDA and M-UF, KLIP Trustees formulated programs that were targeted at:

1) Supporting Ecotourism and Market Places as the flagship projects

2) Strengthening of the Secretariat through participation in the strategic development of Mistra

Urban Futures; and facilitating the establishment of the KLIP Trust.

3) Promoting Knowledge and Technology Transfer as well as Capacity Building for County

Governments via the.

In this evaluation process, we will try to establish the reasons behind formulation of the programs

by the trustees, their methods of implementation, possible success and failure stories, as well as

the future scope of the projects from the trustees’ point of view

Kisumu Local Government Leaders

For any project to succeed, participation of local leaders is paramount. And so KLIP partnered

with various local leaders holding various positions, in the implementation of the various projects,

with special focus in Eco-Tourism and Market place.

As a means of evaluation, we will establish the impact of the programs on the governance in

Kisumu. We will find out, how the partnership was carried. Possible success and failures

throughout the partnership and future modifications that will foster better partnership in the

various projects.

Kisumu Private Sector Organizations


3

One of the goals of KLIP is to bridge the gap between the private sector and the local

government. Through the various programs, we will evaluate the collaborations created in the

process between the private sector and the local government. We will find out, how these ties

have improved the region’s economy. In addition, the evaluation will highlight possible

drawbacks encountered through the partnerships created, and how these can be improved for

future collaborations

Kisumu Residents

KLIP always puts the needs of the locals in the programs and projects it formulates. The goal of

KLIP is cogeneration of knowledge, practical application and dissemination for the purposes of

sustainable urban development. Through the various activities of Eco-tourism and Market

places, the evaluation will establish the following aspects:

Acceptance and attitude of the locals to the various projects

Training involved in the various projects

Impact of the projects on the locals lifestyles

Future proposals for the projects and programs

2.4 Evaluation Management Structure

Position Name

Assistant-Director General Dr.

Director Dr.

Project/Program/Initiative Manager Mr. Owuor

2.5 Evaluation Governance Group Membership

Position Name

Research Assistant Mr.


4

Data Analyst Mr.

2.6 Evaluation Governance Group Terms of Reference

The roles and responsibilities of the Evaluation Governance Group include:

Roles and responsibilities

Project Consultant – Overall in charge. Supervises the project.

Project Manager- Supervises day to day activities in the evaluation cycle. Reports to consultant

Research Assistant – In charge of data collection

Data Analyst – Analyses the data

Field Guide – Assists in data collection

Field Support staff – provides as need service to the team

2.7 Reporting Requirements

Date Report type Writer/s Audience

08/08/2014 Progress Report Project Manager Evaluation Steering Committee

10/08/2014 Draft Report Project Consultant Evaluation Steering Committee

12/08/2014 Quality Assurance Review

Contracted Consultant Evaluation Steering Committee

13/08/2014 Final Report Project Consultant KLIP Director


5

3. Evaluation Design

3.1 Evaluation Framework

The Guide below shows the framework to be applied during the evaluation process

3.2 Information and Collection

Core evaluation

question to

address

Information

required

Information

sources

Data collection and

analysis

What are the reasons behind setting up the various programs and activities in KLIP?

KLIP Programs and Activities

Literature available since 2010

Literature review

Interviews

What is the impact of the KLIP programs on governance in Kisumu?

Influence of KLIP activities on Governance

Key stakeholder – Local Leaders Interviews

Questionnaire

What is the impact of KLIP activities in fostering private sector – government collaborations

Influence of KLIP on fostering private sector – local government relations

Key stakeholder – Private organization partnered in the activities - Interviews

Questionnaire

How has the activities impacted on impact local residents lifestyle

Economy elevation of the locals

Key stakeholder Interviews

Literature reviews

Questionnaire

4. Evaluation Resources and Timeframe

4.1 Budget and Staff Resources


6

Budget for the evaluation study

Internal DET staff

resources

Position Classification

level

Approximate time

and / costs required

Dr. Oketch Team Leader Top level 80 hours – Kshs

150, 000

Project Manager Middle level 50 hours – Kshs 75,000

Research Assistant Middle level 50 hours – Kshs 75,000

Data Analyst Middle level 50 hours – Kshs 50,000

External human resources Approximate time and cost required

Data Entry Technician 40 hours – Kshs 35,000

Field Guide 80 hours – Kshs 20,000

Field Support Staff 20 hours – Kshs 5,000

Other resources required

Cost estimates

Program Logic Modelling session

Data tools generation Kshs 5,000

Questionnaires booklets

Kshs 2,000

Interview recording devices (4)

Kshs 15,000

Video recording devices (1)

Kshs 23,000

Stationary Kshs 5,000

Totals Kshs 45,000

Accommodation & Travel Committee Members

5

Bus Return Ticket Kshs 5,000

Kshs25,000

3 Day Hotel bed & breakfast

Kshs 4,000

Kshs20,000

Total Kshs 45,000


7

Allowances for Evaluation Committee Food Allowances Kshs 10,000

Airtime Allowances Kshs 10,000

Transport Allowances Kshs 10,000

Miscellaneous Kshs 15,000

Total allowances Kshs 45,000

4.2 Evaluation Staff Roles and Responsibilities

Internal DET

evaluation team

members

Position Roles and responsibilities

Dr. Oketch Project Manager Overseeing project evaluation process

Mr. Owuor Research Assistant Supervision of data collection

Mr. Oloo Data Analyst Data cleaning and analysis

External

consultant(s)

Position Roles and responsibilities of the

consultant(s)

Dr. Okecth Project Consultant Consultation on project activities

4.3 Time Schedule

Time/Activities 8:00-10:00AM 11:00-12:00AM 2:00-4:00PM 5:00-7:00PM

31/07/14 Evaluation

Commencement

Approval

process for the

evaluation

Project

Committee 1st

meeting

Project goals

reviewed

01/08/14 Program Logic Model generated Specific Objectives for team


8

members formulated

02/08/14 HOLIDAY

03/08/14 HOLIDAY

04/08/14 Data Tools Generated Data tools reviewed and revised

05/08/14 Data Collection (interviews) Data Collection (interviews)

06/08/14 Data Collection (Questionnaires) Data Collection (Questionnaires)

07/08/14 Data Entry Draft Project progress Report

08/08/14 Draft Project Report Presented to

Evaluation Governance Team

Feedback from Evaluation

Governance Team

09/08/14 Data Cleaning and Analysis Quality Assurance consultant

contracted

10/08/14 Data Analysis Complete Progress Report to evaluation

steering committee

11/08/14 Draft Project Report Presented to

Evaluation Governance Team

Report compilation

12/08/14 Review/Endorsement of report by

the Evaluation Governance Team

Report revision

13/08/14 Final Report approved by

evaluation steering committee

Presentation of final report to KLIP

trustees


9


10

Major milestones Due date

Consultation with Evaluation Services regarding support and approval process for the evaluation

31/07/14

Program Logic Model (or evaluation framework) generated 01/08/14

Evaluation Governance Group established with endorsed Terms of Reference

04/08/14

Draft Evaluation Proposal presented to Evaluation Governance Group for feedback

05/08/14

Evaluation Proposal endorsed by Evaluation Governance Group 06/08/14

Evaluation Proposal approved by Evaluation Steering Committee 07/08/14

Consultant contracted (if applicable) or Evaluation Team appointed

08/08/14

Progress Report to Evaluation Steering Committee 08/08/14

Data gathering and analysis complete 10/08/14

Progress Report to Evaluation Steering Committee 11/08/14

Draft Evaluation Report presented to Evaluation Governance Group for feedback

12/08/14

Draft Evaluation Report endorsed by Evaluation Governance Group

12/08/14

Final Evaluation Report approved by Evaluation Steering Committee

13/08/14

Brief with any recommendations to the Policy and Performance Committee with any recommendations

13/08/14


11

Appendix 1 Evaluation Plan Checklist

This checklist has been included to assist in the development of the Evaluation Plan.

Socio-political Factors The team should identify and effectively address affected/concerned groups

Involvement Whose sanction and support is required?

How will it be secured?

Internal communication How will key audience needs for information on the evaluation’s progress be determined and met?

How will communication be maintained between the program team, the evaluation team and the Evaluation Steering Committee (ESC)?

External credibility Will the evaluation be free of bias?

Realistic expectations How will the evaluation team make it clear to stakeholders that realistically only a subset of their information needs will be addressed?

Security What provisions will assure security of the data?

Protocol What communication channels will be honoured and employed?

Public relations How will stakeholders be consulted and kept informed about the intents and results of the program evaluation?

Stakeholder confidence What checks will be made to ensure that the evaluation plan and the composition of the evaluation team are responsive and acceptable to the key stakeholders?

Contractual/Legal Arrangements The evaluation team and participants should establish clear working agreements to ensure efficient collaboration and protect involved parties’ rights

ESC, evaluation team, and other roles

Who is the sponsor

Who is on the evaluation team

Who are the other audiences

How are they related to the program?

Realistic commitments What clarifications assure that the evaluation can proceed while making reasonable efforts to serve a broad audience but not becoming bogged down in over identifying and consulting with stakeholders?


12

Contractual/Legal Arrangements The evaluation team and participants should establish clear working agreements to ensure efficient collaboration and protect involved parties’ rights

Delivery schedule What is the schedule of evaluation services and products?

Editing reports Who has authority for editing evaluation reports?

Access to data What existing data may the evaluation team use

What new data may they obtain?

Access to stakeholders Are there sufficient safeguards to assure that the evaluation team may contact involved stakeholders?

Pre-release reviews Will the ESC be provided appropriate opportunities to review the draft report for clarity and fairness prior to their finalisation and release?

Release of reports Who will release the reports

What audiences may receive them?

Responsibility and authority

Have the system personnel and the evaluation team agreed on what persons and groups have both the responsibility and authority to perform the various evaluation tasks?

Finances What is the schedule of payments for the evaluation

Who will provide the funds?

Technical Design The evaluation team should convert a general evaluation plan to a detailed, yet flexible technical plan

Objectives What is the program intended to achieve/produce

In what terms should it be evaluated?

Variables What classes of information will be collected, e.g., context, inputs, processes, outcomes?

Program description Will the program be described sufficiently, so that stakeholders will understand its nature?

Investigatory framework Under what conditions will the data be gathered, e.g., experimental design, case study, survey, site review, examination, etc.?

Instrumentation What data gathering instruments and techniques will be employed

How will the evaluator assure that they address the key evaluation questions?

Sampling What samples will be drawn

How will they be drawn


13


Will they meet both utility and technical requirements?

Data gathering How will the data gathering plan be implemented

Who will gather the data?

Have multiple modes of data gathering been planned to ensure triangulation?

Data storage and retrieval

What format, procedures, and facilities will be used to store and retrieve the data?

What strategies are in place to ensure the data complies with ethics and codes of conduct?

Data analysis How will the data be analysed?

Sources of interpretation Who is charged to interpret findings, e.g., the evaluators, various stakeholders, a regulatory body, etc.?

Bases for interpretation What bases will be used to interpret findings, e.g., objectives, assessed needs, contractual specifications, laws and regulations, democratic ideals, social norms, performance by a comparison group, technical standards or criteria, polls, judgments by reference groups, etc.?

Methods of interpretation What methods will be used to assign value meaning to findings, e.g., focus groups, a Delphi study, advocacy and adversary reports, etc.?

Reports What reports will be used to disseminate the evaluation findings?

Reporting media Considering the preferences of the audiences, what are the most appropriate means of reporting findings, e.g., detailed technical reports, summaries, press conferences, study sessions, memos and letters, video presentations, web assessable documentation, etc.?

Reporting language Will reports need to be presented in different languages: technical or non-technical, simple English or other language(s) to meet the needs of different audiences?

Reporting format Will reports be carefully formatted to enhance their readability, e.g., use of white space?

Responsive design What ongoing evaluation planning process and resource plan will assure flexibility for adding to or otherwise revising the evaluation questions and obtaining unanticipated, pertinent information?

Delimited design Is there a clear delimitation of the design, including the


14


purpose of the evaluation and the questions that will be answered?

Attention to trade offs How will the evaluation address trade-offs between comprehensiveness and selectivity at each stage of the evaluation: planning; budgeting; and collecting, organizing, analysing, interpreting, and reporting information?

Technical adequacy What are assurances that the findings will be reliable, valid, and objective?

Moral/Ethical Imperatives The evaluation team and clients/stakeholders should clarify and confirm the evaluation’s role in ethically serving some socially valuable purpose

Philosophical stance Will the evaluation be value based, value plural, or value free?

Evaluation team’s values Will the evaluation team’s technical standards and values conflict with the client system’s and/or sponsor’s values; will the team face any conflict of interest problems; what will be done about possible conflicts?

Judgments Will the evaluation team judge the program; or obtain, analyse, and report the judgments of various reference groups?

Objectivity How will the evaluation team avoid being co-opted and maintain their objectivity?

Equity How will the evaluation team make sure to address and honour the needs and rights of all stakeholders equitably, taking appropriate account of their gender, ethnicity, and language backgrounds?

Cost effectiveness Compared to its potential payoff, will the evaluation be carried out at a reasonable cost?

NB the above listed prices and costs are quoted in their net values.

Evaluation Technical and Financial Plan Revised

Documents

Transcript of Evaluation Technical and Financial Plan Revised