Monitoring and Evaluation Framework - mwcsd.gov.ws WEBSITE/2014... · The Monitoring and Evaluation...

Monitoring and Evaluation Framework

Ministry of Women, Community and Social

Development

Government of Samoa

2 Ministry of Women, Community and Social Development DFRPPIP, 2013

TABLE OF CONTENTS

1. INTRODUCTION 3

2. PURPOSE 3

3. OBJECTIVES 4

4. SCOPE 4

5. M+E FRAMEWORK SET UP 6

6. M+E FRAMEWORK TOOLS 12

7. IMPLEMENTATION SCHEDULE 18

8. REFERENCES 20

9. ANNEXES 21


1. INTRODUCTION

The Monitoring and Evaluation Framework (M+E Framework) is a tool the Ministry of

Women Community and Social Development (MWCSD) will use for monitoring and

evaluation of its activities from a ‘whole of system’ approach, with a focus on how the

services of the Ministry contribute directly to community and social development.

Strengthening the feedback system of the Ministry acknowledges its advisory and

advocacy function in leading knowledge on community issues, needs and priorities. This

allows the Ministry to address community concerns as communicated directly to them by

community members, partners and stakeholders, critically aligning the MWCSD with

international trends in public sector management and accountability.

By creating a more integrated view on community outcomes, the M+E Framework hopes

to inform broader government strategy for community development in Samoa, as well as

the strategic direction of the MWCSD. On the ground, the M+E Framework will allow

progress to be tracked consistently over time across the range of Objectives and

Outcomes that the Ministry has invested in across the Community Sector. This will in

turn clarify vulnerable populations and target groups in order to focus and prioritize

Ministry interventions. An improved M+E Framework will also create better information

sharing between staff across all MWCSD Divisions, harmonizing the cycle of information

between research, policy, planning and implementation. It will allow for development of

systematic and focused research on issues affecting our communities, resulting in sharper

organizational governance, strategy and policy development. Formalizing meaningful

community feedback into the M+E Framework, along with throughput from data capture

and analysis, is a means of emphasizing accountability as well as demonstrating results to

all stakeholders; ultimately creating an enhanced Community Sector and strengthened

communities.

2. PURPOSE

The M+E Framework constitutes an essential tool which will facilitate monitoring and

evaluation of the entirety of the Ministry’s programs and activities, including those

collaborations with partners across the Community Sector. It addresses the need for a

culture change at the Ministry both in terms of operational structures and work attitudes;

signaling an important transformation away from simply completing activities toward

critically evaluating their impacts on the individual and the community. The M+E

Framework will ensure MWCSD remains proactive in addressing challenges facing the

Samoan Community Sector by ensuring improved monitoring and evaluation is reflected

in the strategic planning and working mentality of MWCSD. Monitoring progress and


impact using defined targets and indicators highlights the successes and failures of the

Ministry’s efforts, critically redirecting attitudes and resources toward improved

community and social development.

Through consistent management of data collected from Divisions and unified procedures

for reporting and analysis, the M+E Framework will strengthen communication and

analytical capacities of all Divisions across MWCSD. Supporting DFRPPIP’s role as the

analysis hub and creating centralized processes will ensure that monitoring and

evaluation data is utilized in evidence-based planning and policy formulation in a regular

and formalized way. The basic purpose of the M+E Framework is thus to measure the

outcomes and perceived impacts of all Ministry activities, in order to assess their benefit

to the individual and/or the community. By prioritizing community consultation and

data/feedback analysis, the M+E Framework aims to capture the trends, concerns and

experiences of individuals and groups in order to improve decision making.

3. OBJECTIVES

Consistent and centralized tracking of progress on our activities to improve

analysis, planning and program development mechanisms, in order to facilitate

implementation of the Strategic Plan 2013-2017 and the Community

Development Sector Program 2013 - 2017;

Promote a culture of forward-planning at MWCSD based on the feedback cycle,

emphasizing increased communication, information sharing and collaboration;

Improve community consultation, links with partners, Sectoral policy advice,

and accountability to all stakeholders.

4. SCOPE

The M+E Framework design draws on the special policy report- Outcomes Framework for

the Social and Economic Development of Samoa’s Communities, developed for MWCSD in

2010 to capture the key community development outcomes against which Sector

performance could be measured. DFRPPIP utilized the Outcomes Framework blueprint

to develop an M+E Framework that governs the work of the MWCSD.

The MWCSD M+E Framework aligns with the two Core Outcome Areas of the

Community Development Sector Plan 2013-2016 (also developed from the Outcomes

Framework), and will both inform and complement the M+E Framework developed for

the Community Sector Plan. Linking closely with and supporting the strategic framework

of the newly developed Strategic Plan 2013-2017, all core internal functions of the


Ministry, including printing, institutional capacity and service delivery, were integrated

under an additional Key Priority Area: Institutional Strengthening. The MWCSD M+E

Framework also aligns under Priority Area II: Social Sector of the Strategy for the

Development of Samoa 2012-1016, ‘Social Policies’, as per Table 1 below:

Table 1

This approach ensures that monitoring and evaluation processes at MWCSD provide

performance tracking against community development outcomes not only at the

Ministerial level, but also at the Sector and national levels. Furthermore, in recognition

that the Outcomes Framework (2010) was designed as an overarching concept for the

entire Community Sector, the above areas of the Strategic Plan 2013-2017 are designed to

guide strategic planning, monitoring, and evaluation of the initiatives of MWCSD, inter-

government collaboration, joint partnerships, and relevant stakeholders.

Whilst the strategic scope of the M+E Framework is limited to analysis of the activities

and programs of MWCSD, Sector-wide performance evaluation is undertaken where the

Strategic Plan incorporates initiatives or activities with Sector partners. This is in

recognition that ‘causal contribution’1 from both collaborative and partner-led initiatives

also impacts upon the outcomes of Ministry projects, as well as linked environmental

factors. The M+E Framework will critically link with the Community Development Sector

Plan’s M+E Framework, based on their shared use of key priorities and outcome areas,

whilst their mandates rest at the Ministerial and Sector levels respectively. 1 Better Evaluation, 2013.


In addition, progress tracking, data collection and analysis of impacts against indicators

under the MWCSD M+E Framework will be fed into the Community Sector Plan’s

monitoring and evaluation processes, in order to inform and improve the way the

Ministry collaborates with its partners to achieve social development results. The

relationship between these two Frameworks creates exciting opportunities for key

collaborations across participatory evaluation and feedback techniques, toward the

common goal of community ownership for sustainable social development. This

approach aims to ‘save resources that may otherwise be spent in inefficient programs or

overlapping activities supported by different partners’2, and also signifies critical

alignment between the Strategic Plan 2013-2017 and the Community Development Sector

Program 2013-2017.

5. M+E FRAMEWORK SETUP

The core elements of the M+E Framework design reflect the directives of the MWCSD

Strategic Plan 2013-2017 under the umbrella of the Samoan Community Development

Sector Program 2013-2017:

a) categorization of outcomes into economic, social and cultural dimensions,

b) clear connections between outcomes, reflecting the interconnected nature of social

development initiatives,

c) focus on indicators and outcomes that the Ministry can control and are within its

resources and capacity and,

d) a platform for reporting impacts at the community level.

Therefore the M+E Framework will be categorized by Outcome levels supported by

contextualized indicator sets. The M+E Framework has also been designed to fit with and

improve on existing Ministry processes for monitoring and evaluation, to emphasize

effectivity, usability and limit duplication of efforts or processes.

The M+E Framework aligns with the structure of the MWCSD Strategic Plan 2013-2017 as

laid out below and in Table 2:

3 Key Priority Areas, the main indicator areas the Ministry aims to contribute

towards in the 5 year Strategic Plan period, which match the two Core Outcome

areas of the Community Development Sector Program.

3 Objectives, broad strategic goals which detail the change MWCSD would like to

see in social and community development across Samoan society.

2 The Global Fund, 2013.


7 Outcomes, are of a thematic and continuing nature, governing not only Ministry

programs for service delivery but also its technical and institutional structures.

Both Objectives and Outcomes are often addressed in a cross-sectoral fashion, and

remain influenced by interrelated social, environmental and economic factors.

10 Outcome Indicators, to link Ministry program areas and activities to the above

Objectives and against which MWCSD efforts can be evaluated. Outcome

Indicators measure the direct means by which the activity is able to contribute to

the wellbeing of the community, for example access to services, reduced disputes

or increased ownership. The M+E Framework further delineates activities under

each Outcome Indicator by 2 sub indicator sets:

o Process Indicators (lower level)

o Results Indicators (higher level)

It is acknowledged that there is significant cross-over of impacts under the Outcomes, as

indicative of the interrelated nature of community development work. Other broader,

collaborative and more cross cutting outcomes will be addressed at the Sector level, as

had been explained in the ‘Scope’ section. There will be a degree of intersection at the

indicator level between the MWCSD and Community Sector Plan M+E Frameworks,

whereupon information sharing and strategic results management for reporting will

occur. Such cooperation will promote evidence-based planning for improved program

development and delivery across the entire Community Sector.


Table 2

N.B. The activities carried out by the Ministry toward these Objectives have been

grouped in the Strategic Plan into 8 STRATEGIES or modes of implementation:

Strengthening policy and legal frameworks

Research and evaluation

Community and social development programs

Communications

Corporate governance

Cross-sectoral collaboration

Village governance

Gender mainstreaming


M+E Matrix Indicators

Outcome Indicators are supported by 2 sub indicators sets as set out below, and shown in

Table 3. Up to two indicators have been chosen for each activity, based on measurability,

relevancy and Ministry capacity/coverage, in order to streamline the monitoring and

evaluation process. The indicators chosen have been developed to provide a snapshot of

whether the particular program or activity is making progress toward the linked

objective and intended results. Indicators have also been chosen to allow DFRPPIP to

identify whether the direction of implementation is in line with Ministry

recommendations and resolutions recorded from the previous year(s).

a) Process/Lower Level Indicators

Process indicators contain the following characteristics:

Measure the outputs, or various financial and physical resources dedicated to a

goal; e.g. public expenditures on trainings and facilitators

Measure the actual goods and services that are produced by the inputs at the

activity level; e.g. number of training sessions, number of people trained

b) Results/Higher level Indicators

Results Indicators measure the positive impacts of an activity on the people or group

immediately involved, and toward the desired objective. These indicators are more

qualitative, measuring broader social, economic or cultural impacts; e.g. measures of

knowledge, understandings or practices in community, or long-term effects of program

initiatives in terms of social change. Often, monitoring against Results Indicators will

display progress made by MWCSD toward broader Community Sector objectives,

supporting analysis of trends and strategic governance toward Sector-wide priorities.

An additional Indicator set is that of the Performance Budget Framework of the Ministry

of Finance, indicated by yellow highlight. The MWCSD currently utilizes the MOF

Performance Budget Framework to evaluate progress against budgeting allocations and

performance requirements. As well as budget indicators, these indicators can also include

reference to organizational or Sector performance measures for relevant regional and

international programs and agreements. Finally, indicators which align with those

developed under the SDS 2012-2016 are indicated by an *.

The above sub indicators have been incorporated within the existing guiding frameworks

of the MWCSD, namely the MOF Performance Framework, and the Annual Management

Plan and 6 months Plans, along with individual Divisional Strategic Plans and Policies.

This will ensure fit and continuity of the M+E Framework across the diverse workplans

of individual MWCSD Divisions.


Indicators will be informally appraised during the implementation phase, ensuring fit

with the Ministry’s current workplans, and formally reviewed annually in order to

guarantee they remain relevant and timely. The review process will include a formal

stock-take of indicator and process alignment with the Community Development Sector

Program’s M+E Framework, to ensure the continuation of shared performance

measurement and management. This review process also ensures that indicators can

accurately reflect change and capture social developments across the community.

NB. Gender disaggregation of relevant indicators will be prioritized under the Gender

Management System and MWCSD’s focus on gender mainstreaming. Disaggregation

allows the Ministry to check for change in the indicator for separate groups- namely

women and men. This is important to program design and implementation in terms of

‘assessing changes in equality and discrimination over time’3 to understand who is being

included/excluded.

Activities Activities used within the M+E Framework are taken directly from the MWCSD Strategic Plan 2013-2017. The M+E Framework sub Indicators are aligned to these activities, and will ensure a maximum knowledge and reporting base for the achievement of the activities, as well as the extent to which they are being realized. Activities within the Strategic Plan 2013-2017 will also be revised annually.

Indicator clarification

The green columns in the 3 Indicator Matrix tabs in the Central M+E Matrix offer

comprehensive contextualization for each indicator (see Table 3). The Evidence column

lists the means of verification of each indicator, ensuring that indicators are measurable

and that the data collection methodology has been grounded in the reality of the

particular setting. The MWCSD M+E Framework sources of evidence include but aren’t

limited to:

Divisional databases and records

Meeting Minutes

Research studies and reports

Population based surveys (target group/customer feedback surveys; Family

Health & Safety; NYS; Social Impact Assessments)

Training, Monitoring or Evaluation reports

Village representative Performance Management Systems (PMS)

Tailored monitoring/tracking databases

Internal reviews and partner/stakeholder reports

3 Save the Children UK, 2008, p.10.


Evidence sources can be used to demonstrate how the activities of the Ministry contribute

to community development, outlining measures of change around each objective area.

Establishing and/or identifying cost-effective evidence sources for indicators is an

ongoing process, dependant on existing Ministry mandates, capacity and resources.

Whilst some systems exist others may be developed in collaboration with Sector partners,

particularly under the directive of the Implementation Framework for the Community

Development Sector Program.

The Frequency column refers to how often data collection occurs, as relevant to the activity

and indicator. Description allows for definition of key terms; collaborations relevant to the

progress on the particular indicator, for example Sectoral partnerships; identification of

risks that may influence progress on the activity; and space for assumptions- for example

existing levels of support or success of linked initiatives. The final column defines who is

responsible for data collection/management for the indicator measurement.

Table 3

Participatory consultation

The M+E Framework will work under Objective 3 to increase participatory evaluation

initiatives. A range of options for participatory community consultation measures are to

be workshopped with Divisions and management throughout the implementation

process for the M+E Framework. Strategic planning and cooperation with the

Community Development Sector Unit will further contribute toward efficient

implementation of participatory monitoring and evaluation methods for all MWCSD and


Sector activities. These methods will work to put the Ministry’s quantitative data

collection into context, and will help to ‘capture environmental factors influencing the

project outcome, that have not been taken into account quantitatively.’4 This will in turn

inform analysis of program effectivity, and feed into the Ministry goal of improved

village level ownership and empowerment for community development.

6. M+E FRAMEWORK TOOLS

The following are the design and process tools that will support the implementation of

the M+E Framework: these have either been updated from existing forms, or developed

to complement the new M+E Framework design. The Ministry, through the M+E

Committee and its representatives, will be collectively involved in the development and

application of these tools so that monitoring and reporting capacities will be strengthened

and a culture of continuous evaluation of initiatives promoted.

Strategic Framework

Table 2, above, displays the visual framework of the Strategic Plan 2013-2017. This

provides a graphical overview of the linkages between the Ministry’s Key Priority Areas,

the Overarching Outcomes and the Objectives, and will be used largely for ease of

reference and public information.

Feedback Cycle at MWCSD (see Annex 1)

Annex 1 provides a visual overview of the Feedback Cycle at MWCSD, and its linkage with

the M+E Matrix housed on the CIS as the central Ministry server. The Feedback Cycle

details the social and community development process at MWCSD, from program

implementation, monitoring and feedback, reporting and analysis: - and the continuation

of this circle of information and communication under the M+E Framework.

Central Information System (see Annex 2)

The recently implemented Central Information System (CIS) of the Ministry represents an

information system capable of storing data required to enhance monitoring and

evaluation of results. This database will house the Gender Management System and the

DevInfo information system for collating and storing monitoring data of the Ministry’s

program Divisions. Together these systems will provide the baseline data, gathered by

the program divisions from the field, and converted by the DFRPPIP for analytical and

policy purposes. The CIS will also house the new Central M+E Matrix, along with

Ministry publications, Committee reports and minutes, Sector documents, and evaluation 4 Grun, 2006.


reports and questionnaires. In addition, a centralized M+E Framework folder containing

all monitoring and evaluation material will be easily accessed by staff. The functioning of

the CIS in conjunction with the Central M+E Matrix will work to promote wider access to

and application of monitoring data by staff across the Ministry.

DevInfo Monitoring System

The DevInfo information system will be used to collect and compile quantitative data on

ongoing Divisional activities, as outlined in the Calendar of Events (see below).

Monitoring data will be collected by Divisions during their regular monitoring visits,

surveys and discussion groups. DevInfo will capture information including:

- monitoring visit data: number of programs conducted/projects implemented etc.

- trainings and information sessions

- representatives meetings/reports/ PMS

- partnership meetings

- events conducted

This data will then be input into DevInfo by DFRPPIP according to defined indicators

making up contextualized Divisional datasets. Divisions will submit quarterly

quantitative reports to DFRPPIP via Proformas based off these datasets. Relevant data

will then be extracted from DevInfo for regular input into the Central M+E Matrix,

according to high level indicator sets, and used for analysis and reporting. Such data

collection can measure alignment of implementation with planning, and when translated

into the Central M+E Matrix, will allow the Ministry to identify ‘snapshot’ performance

outcomes across our community development initiatives.

Central M+E Matrix

The Central M+E Matrix (Matrix) is the substantive center of the M+E Framework,

developed to allow capture, display and analysis of both raw data and descriptive

feedback. The Matrix is a central database which visually portrays the logic chain

between Ministry program implementation, monitoring, reporting and planning,

presenting the links between Ministry planned objectives and outcomes (via Activities

within the Strategic Plan 2013-2017) and indicators of progress. This new database is

designed as the hub of the M+E Framework, the key performance management tool

responsible for tracking and collating all relevant high level M+E data and reports. This

tool will allow input of required streamlined data against agreed cross-divisional

indicators, from which DFRPPIP will conduct analysis of outcomes and progress. The

Matrix functions as a visible reminder to all staff of the strategic nature of the data in

terms of collating outcomes and analyzing results.


The Central M+E Matrix consists of the following elements:

Indicator Matrix 1-3; delineated according to the 3 Strategic Plan Key Priority

Areas, including evidence source, frequency and definitions/risks/assumptions.

Performance Tracking Table; outlining overall progress against targets and

highlighting areas to be addressed in research, planning and implementation. Data

on indicators is stored in this tab, including baseline data and targets where

applicable. The PTT will provide a visual warning when a program or activity is

off schedule or underperforming, and allow for proactive adjustments.

Reporting schedule; highlighting the timeframe, format and requirements of

reporting for each outcome area. Linkages with the Community Development

Sector Plan and its M+E Framework will be outlined here. This tab will also

increase impetus for Divisions to submit timely reports, showing reports

submitted and those awaiting submission.

Calendar of Events; a monitoring tablature based on the 6 Month Plan, aligned to

the Annual Management Plan. It allocates the activities in the Plan to specific

months, dates and locations that can be monitored over the 6 months. It is able to

break down major activities, such as the National Youth Awards, into detailed sub

activities for ease of monitoring. More importantly, it is able to list the dates for

which monitoring is to take place, thereby making monitoring a more visible part

of Divisional workplans.

Partners/Stakeholders; a comprehensive list of MWCSD partners and

stakeholders, detailing areas of collaboration, timeframe for cooperation, and

terms of engagement for each.

Activity/Indicator Register; the visual layout displaying linkages between

Activities and their Indicators for success. For all staff ease of reference when

following up progress against indicators.

In order to delineate the work being undertaken under each Outcome and Objective, a

colour code has been assigned to each Division, as reflected in the Calendar of Events and

Activities Register.

- Office of the Chief Executive Officer (Community Sector Unit) – Green

- Division for Youth – Dark Blue

- Division for Women – Light Blue

- Division for Internal Affairs – Red

- Division for Corporate Services – Pink

- Division for Research, Policy Planning and Information Processing - Purple

- Division for Printing - Orange

NB. Activities with cross-Divisional responsibility have been coded according to the main

Divisional mandate, and an * placed to demonstrate additional support.


Gender Checklist (see Annex 3)

The Gender Checklist will become a formal element of ongoing Divisional monitoring, as

a means of promoting gender mainstreaming across MWCSD. Individual Gender

Checklists will be developed for each Division in order for them to assess progress

against their work in terms of promoting gender equality and gender aware goals and

action. The Gender Checklist will be incorporated into Divisional datasets in DevInfo,

outlining the specific Gender Indicators for that Division. Gender Checklists will be

reported against by each Division via 6 monthly Evaluation Presentations, and DFRPPIP

will analyze progress via the 6 monthly Analysis and Annual M+E Reports. DFRPPIP

will also provide analysis on annual progress in terms of gender mainstreaming across

MWCSD in the Annual M+E Report. Gender Checklists will be reviewed and updated

annually to ensure relevancy. Please see Annex 3 for a generic example of the Gender

Checklist format.

Reporting and Analysis Schedule

Focused analysis and reporting on data is a crucial step in measuring successes, identify

areas for improvement, and capturing lessons learnt. It also allows the Ministry to isolate

issues with implementation or planning, and identify contextualized ways to revise

programs. Monitoring and evaluation reports are a central element of any sustainable

community development cycle, improving accountability of the Ministry by

demonstrating impacts against planning and justifying interventions. Accurate M+E

reporting shows that MWCSD remains responsive to community feedback and priorities,

and can maintain transparent communication and collaboration with partners and

stakeholders. See Table 4 below for a visual of the M+E Framework Operational

processes, including reporting timeframes.

The M+E Framework lays out quarterly, six-monthly and annual reporting and analysis

processes. Two Impact Evaluations will be conducted during the 5 year Strategic Plan

period, with a Ministry-wide focus to analyze our community development and broad

programmatic impact. The initial Impact Evaluation will be in 2015 and the final in 2017.

The M+E Framework will use standard reporting and analysis templates to centralize and

ensure ease of reporting.

Templates cover:

- Quarterly Divisional Reporting (to complement DevInfo proformas)

- Quarterly DFRPPIP Analysis; quarterly reporting/analysis is designed to promote

improved internal communication and teamwork at MWCSD, based on constructive

feedback and dialogue, and relevant follow-up support


- 6 Monthly Divisional Presentations; cover two Divisional quarterly reporting

periods, and provide a progress report for development of best practice and program

improvement

- 6 Monthly DFRPPIP Analysis Reports; provide a summative critical examination of

Quarterly Divisional Reports submitted to DFRPPIP, along with additional M+E data

and feedback across the 6 month period. 6 monthly templates are designed to provide

a comprehensive quantitative and qualitative picture of the Ministry’s work in terms of

program/project impacts at the community level, analysis of cross-sectoral

collaboration, and alignment with our strategic priorities as a Ministry.

- Annual M+E Report; presented at the end of financial year July Evaluation Process.

Content is taken from quarterly and 6 monthly reporting and analysis, along with the

Performance Tracking Table in the M+E Matrix. This Report captures progress against

the Ministry’s higher level strategic priorities as per the Strategic Plan.

- Impact Evaluation Report (see above)

- Qualitative Evaluation (general template for Divisional use across qualitative

methods)

In addition to the above schedule, MWCSD undertakes various reporting processes

across all program levels. Such additional structures will be retained and fed into the

M+E Framework to complement high level reporting, outlined in the Reporting Schedule.

Reporting structures include:

Training reports

Community liaison officer Performance Management Systems (PMS)

Financial and Donor reports, in alignment with Sector initiatives

MWCSD Annual Reports and Village Profiles

Performance reviews

From service providers, NGOs, and local/regional/international stakeholders

The Reporting Schedule tab will clearly outline the format, timeframe and intended user

for each report, along with the indicator to which it is linked. The CIS will allow these

reports to be accessible by all Ministry staff, effectively contributing to increased internal

accountability and knowledge circulation. Under the M+E Framework, improved

community consultation techniques to be incorporated will increase the levels of

participatory feedback to be communicated to those who plan and develop policy for the

community. Here, there will be a focus on building staff capacity in collecting correct data

to effectively portray community opinion and needs.


Targeted reporting enables the Ministry to provide the ‘specialist knowledge’5 to the

Community Sector for positive development outcomes. The above tools of the M+E

Framework create a cycle of information and data to all levels; from the grassroots up to

the Minister (see Annex 1). Aligning reporting with the key strategic areas of the

Community Sector Program, such as family and community strength and income and living

standards,6 contributes toward Ministry plans that are ‘strengths based, collaborative, and

promote the management of resources’7 at the community and village level.

Table 4 below outlines the different processes, schedules and relevant documents which

inform the M+E Framework.

Table 4

5 Hon. Tolofuaivalelei Falemoe Leiataua, Minister for Women, Community and Social Development, MWCSD, 2013, p. 3. 6 Implementation Framework - Community Development Sector, MWCSD, 2013, p.3. 7 Leituala Kuiniselani Toelupe Tago Elisara, Chief Executive Officer, MWCSD, 2013, p. 5.


M+E Committee

The Monitoring and Evaluation Committee (M+E Committee) will provide critical

support to the design, development and implementation of the M+E Framework, to

ensure that it is employed effectively across all Divisions of MWCSD. With full Divisional

representation, members will be tasked with leading improved monitoring and

evaluation processes internally within Divisions, including monitoring and assessment

procedures, data management, evaluation techniques, and analysis and reporting. Each

M+E Committee member will become a critical part of promoting unity and

centralization, encouraging increased data sharing and collaboration across Divisions

within MWCSD, to facilitate cross-divisional access to community statistics, feedback,

and concerns.

7. IMPLEMENTATION SCHEDULE

Table 5 details the planned implementation schedule.

Table 5

N.B. The Implementation Schedule will be revised during this phase as necessary.


Training will be conducted for all MWCSD staff, and also for the M+E Committee

specifically. Training will comprise contextualized M+E theory, MWCSD M+E

Framework design and objectives, required technical processes, and specific staff roles.

The training program will be expanded to include a session on alignment with the

Community Development Sector Program M+E Framework in 2014 as necessary.

Review process

MWCSD will conduct a review process of the M+E Framework from April 2014. The

review purpose is to allow for stakeholder (i.e. primarily MWCSD staff) experiential

feedback regarding the M+E Framework design, the implementation process, and

perceived results of the initiative. An analysis report on the M+E Framework, detailing

areas for revision and proposed alterations, will be generated for Management. This

review process will also include examination of scope for increasing performance

evaluation of Sector-wide initiatives and programs.

Resources and Technical Capacity

Implementation of the M+E Framework will require certain dedicated resources, as

follows:

IT and material resources will be allocated to support implementation. IT support

will be required to ensure databases are completed and functioning, and housed

on the CIS in an intuitive and accessible way.

Material and human resources will be put toward developing additional

participatory evaluation techniques. M+E Committee members will be critical in

identifying needs, worshopping ideas, and updating Divisional workplans to

include these methods.

M+E members will dedicate time to execute monitoring and reporting on activities

and programs within their Division, as per the M+E Committee Roles and

Responsibilities. Duties will include monitoring indicators, targets and data

collection methods, ensuring that data is input into relevant databases, and

overseeing the completion of templates and reports. A culture of teamwork is to be

encouraged from staff at all levels of the feedback cycle, and representatives will

lead this culture change for their Division.

These initiatives will ensure that all Divisions play a critical and integrated role in the

functioning of the M+E Framework. Furthermore, the Ministry as a whole must work to

develop ownership of the M+E Framework and associated practices, through buy-in from

staff, leadership from Management, and an emphasis on best practice and quality control.


References

Better Evaluation. 2013. Accessed June 2013.

< http://betterevaluation.org/plan/understandcauses >

Government of Samoa. 2008. Strategy for the Development of Samoa 2008-2012. Apia, Samoa.

Grun, R. 2006. Monitoring and Evaluating Projects: A step-by-step Primer on Monitoring,

Benchmarking, and Impact Evaluation. HNP Discussion Paper, The International Bank for

Reconstruction and Development/The World Bank. Washington, DC.

Ministry of Finance. 2013. Performance Budget Framework. Government of Samoa, Apia,

Samoa.

Ministry of Health. 2011. Monitoring and Evaluation Operational Manual. Health

Sector/System Performance. Government of Samoa, Apia, Samoa.

Ministry for Women, Community and Social Development. 2013. Implementation

Framework - Community Development Sector. A proposed framework option for the

implementation of the Community Development Sector Program 2013 – 2017. Government of

Samoa, Apia, Samoa.

Ministry for Women, Community and Social Development. 2013. Strategic Plan 2013-2017.

Government of Samoa, Apia, Samoa.

Save the Children UK. 2008. Menu of Outcome Indicators. Guidance Note.

The Global Fund. 2013. The Global Fund; To Fight AIDS, Tuberculosis and Malaria. Accessed

July 2013. <http://www.theglobalfund.org/en/activities/monitoringevaluation/ >

http://betterevaluation.org/plan/understandcauses

http://www.theglobalfund.org/en/activities/monitoringevaluation/


Annex 1: Feedback Cycle at MWCSD


Annex 2: MWCSD Centralized Information System

The Division for Research, Policy Planning and Information Processing has developed

and implemented the Centralized Information System, as directed under the Corporate

Plan 2008 – 2012 (Objective 2, Strategy 5). The CIS objectives are to provide ease of

access to information and policy advice; support the development of other central

systems at MWCSD such as the Gender Management System, M+E Framework, and the

Interagency Response Systems; and to ensure staff are informed of programs at

MWCSD, including those outside of their own Divisions.

Current database systems operating within the Ministry:

1. DFW - Child protection information system, developed under PCP (0405). This is a Sectoral tool that utilizes the DFW’s community access and gathers unique child related data, such as birth registration records.

2. DFW - Performance Management System for coordination of duties of Village Women Representatives (0405)

3. DFW – Women’s Information System for measuring progress in line with CEDAW indicators (0506)

4. DFW – Database for Aiga Ma Nuu Manuia (0405) 5. DFIA - Performance Management System, coordinating duties of Pulenuu (0405) 6. DFY – Youth Information System (1011) – data collection for the National Youth

Policy 2011 – 2015, and other relevant national youth-based data. 7. DFRPPIP / DCS – Management of salary payments of liaison officers (1011) 8. DCS – Personal file index for liaison officers (1011)

10. GENDER MANAGEMENT

(DFRPPIP)

9.

Central M+E Matrix

(DFRPPIP)

1. CPIS

(DFW)

2. GWR PMS (DFW)

3. WIS

(DFW)

4. ANM

(DFW)

5. VR PMS

(DFIA)

8. Youth Information

System (DFY)

7. Salaries of liaison officers

(DCS)

6. Personal file index of liaison

officers

(DCS)

(DCS)

SECTOR COMMITTEE

REPORTS

MWCSD COMMITTEE

REPORTS

QUESTIONNAIRES MWCSD PUBLICATI0NS

GENERAL STATS


7 new systems and files proposed: 9. M+E Framework Folder, containing the Central M+E Matrix for all Ministry staff

access. Along with the M+E Matrix, this folder will house the DevInfo database, M+E Committee documents, and M+E reports.

10. The Gender Management System will be incorporated into all MWCSD systems to promote gender mainstreaming across all Ministry policies, program and activities. Gender Indicators drafted under the GMS will be reflected in the Central M+E Matrix to allow tracking and reporting on gender by each Division.

11. File for minutes/reports from Sectoral meetings Ministry staff attend 12. File for minutes/reports from meetings of committees and councils housed

under the Ministry 13. File for information questionnaires completed for partners 14. File for all Ministry publications:

Annual reports

Plans (Corporate Plans, Annual Management, Calendar of Events)

Profiles (Directories, Village Profiles, Glossaries, etc.)

Newsletter issues

Ministerial briefs

Policies and Policy papers

Legal mandates of the Ministry, as well as other supporting legislations 15. General Stats – this contains relevant resources from other Sectors, including

policies and reports

All databases mentioned above have been developed and/or transferred onto the CIS.

The DFRPPIP is tasked with the following in relation to management of the CIS:

Monitoring and ensuring that databases are updated (Senior Information Officer). The duration in between updates of each database is determined by respective divisions.

Ensuring that databases are protected to avoid people other than the relevant officer(s) from the Division to which the database pertains, from altering data. (Senior Information Officer). This requires the relevant Permissions and Access to be attributed to specific MWCSD staff.


Annex 3: Gender Checklist example format

GENDER CHECKLIST Division for <__________>

Gender mainstreaming is now a globally recognized tool used to promote and achieve gender equality.

It comprises analyzing all work and initiatives from a gender perspective, to ‘ensure that women and men have equitable access to, and benefit from, society’s resources and opportunities.’8 In recognition of the often disadvantaged position of women and girls across all societies, gender mainstreaming at MWCSD aims to fully address the needs and priorities of all groups and members of the community.

MWCSD has developed Gender Checklists as an integral part of the M+E Framework, designed to analyse specific gender issues as relevant for the work focus of each Division. Individual Divisional Gender Checklists represent the specific gender indicators for the M+E Framework, designed to measure and assess whether individual Divisions, and the Ministry collectively, is working toward its gender mainstreaming goals. Questions under the Checklist are designed as simple and straightforward measures of each Division’s progress toward gender equality within their work, programs and guiding policies and plans. Gender Checklist information is monitored by the Division and recorded in 6 Monthly Evaluation Presentation reports, with progress evaluation and reporting presented bi-annually at the MWCSD 6 months Evaluation Process.

DIVISION

Date Completed

Signed Off (Divisional contact)

Review Date

Division Info Main legislations/policy/programs governing the work of the Division: Main Objectives of the Division: Target beneficiaries or impacted groups: - direct: - indirect:

Program Design

- Gender information

Has a gender analysis been conducted by the Division? If so, list which target groups have been assessed, and which programs?

Outline the core gender issues impacting on the context- are particular groups of women/men disadvantaged based on their age, religion, sexual orientation, disability, family responsibilities?

What are the perceived areas of inequality for the target/impacted groups? Please provide responses in terms of the potential or perceived differences in:

1. Rights 2. Roles and Responsibilities 3. Resources 4. Participation 5. Values and norms

8 International Trade Centre. 2009, p. 1.


- Consultation

Have both women and men been consulted in developing programs and activities?

What, if any, national/regional/international conventions or treaties on gender equality do specific programs or activities comply with?

- Needs and Priorities

Have specific needs of women and men been identified and integrated in designing programs?

Was awareness of gender roles in community taken into account when developing the logistics of the program/project- i.e. domestic responsibilities. Please explain in terms of:

1. Time 2. Space used and Transport 3. Technology 4. Money

Describe how specific activities are designed to address the following Gender Priority Areas9: 1. Leadership/politics 2. Gender based violence 3. Economic development 4. Health and Education

How do your programs and activities address observed gender differences and inequalities as identified above? Please provide specific examples from relevant programs

- Resources

Have resources been allocated to addressing the needs of women?

Has budgeting allocation taken into account gender sensitivities? Explain how

- Impacts

How will socio-cultural gender norms and stereotypes impact implementation and target group?

Describe how the program aims to promote the elimination of discrimination against women

How will women or men be differently impacted by your work? Both positive / negative impacts

Program Implementation

- General

Have your Division’s programs/opportunities been promoted in a gender sensitive manner? Provide responses in terms of:

- Print materials - Website - Radio - Television - Newsletter - Talk / seminar / workshop - Other_______________

Do activities have processes in place to be able to identify and report on any gender gaps?

- Inclusion/ Access

Do men and women have equal access to participate in training sessions run by the Division?

Do your program materials: -reflect current trends in gender equality and equity - show both males/females in their imagery and content

Are men and women equally able to access the outcomes and decision-making opportunities created by your work? Please explain why/why not

9 See the MWCSD National Policy for Women, which outlines the 4 Gender Priorities Areas to guide the Ministry’s work toward

promoting and improving gender equality and equity across Samoa.


Program Monitoring and Improvement Has sex disaggregated data or information been taken into consideration during the

implementation or evaluation?

If no sex-disaggregated data is available, what steps have been taken to collect it?

How does collation and analysis of M+E information contribute to gender equitable program development and improvement?

What mechanisms allow stakeholders/public to give feedback regarding the Division’s work?

Describe any program revisions you have undertaken in response to feedback or directives from your regional/international partners and collaborations

What review steps are being undertaken to advance gender equality within the Division’s work?

Institutional Strengthening

- Policy/legislation and decision-making

Identify any Divisional policies, legislation or practices which address and/or promote gender awareness

Do you encourage both your male and female staff to take time out for raising children?

- Recruitment and training

Please provide information on the gender disaggregated breakdown of staff within the Division

Have staff been provided the opportunity to attend gender awareness training? If not, what planning is in place to address training needs for relevant staff?

Do you encourage female staff to take part in continuing education and staff training measures?

- General

Do you make sure that all publications and materials disseminated from your Division are written in a language that is gender sensitive?

Are there any identified issues with staff gender sensitivity? Please explain, and provide strategies to address these

Are there procedures in place to address any workplace issues regarding sexual harassment, equal opportunity, recruitment practices or other gender related employment issues

Identify any areas and/or strategies to improve the enabling environment for gender equality and equity within your Division

What percentage of your current work across the Division would you advise is dedicated to MWCSD’s gender mainstreaming objectives?

Please provide any additional comments or feedback regarding the Division’s gender mainstreaming efforts

References - International Trade Centre. 2009. “Gender Mainstreaming: The Role that Entrepreneurs can play and why.” Gender Mainstreaming Checklist for Legislation, Public Policies and Programmes. <http://www.intracen.org/uploadedFiles/.../gender_mainstreaming.pdf> - National Commission for the Promotion of Equality (NCPE). 2010. “Gender Mainstreaming – in Practice”. Checklist for Gender Mainstreaming of Government Action. http://ec.europa.eu/progress - World Health Organization. 2010. “Gender mainstreaming for health managers: a practical approach.” WHO Gender Analysis Tools. - World Health Organization. 2010. “Checklist for assessing the gender responsiveness of sexual and reproductive health policies.” Pilot document for adaptation to national contexts, p. 1-6.

http://ec.europa.eu/progress

Monitoring and Evaluation Framework - mwcsd.gov.ws WEBSITE/2014... · The Monitoring and Evaluation...

Documents

Transcript of Monitoring and Evaluation Framework - mwcsd.gov.ws WEBSITE/2014... · The Monitoring and Evaluation...