The World Bank · 2016-07-08 · SIGOB Sistema de Programación y Gestión por Objetivos y...

Document of The World Bank

Report No: ICR 3449

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-76200 TF-57066)

ON A

LOAN

IN THE AMOUNT OF US$ 8.5 MILLION

TO THE

REPUBLIC OF COLOMBIA

FOR A

STRENGTHENING PUBLIC INFORMAITON, MONITORING, EVALUATION FOR RESULTS MANAGEMENT PROJECT

UNDER THE STATCAP (APL) PROGRAM

June 24, 2015

Poverty Global Practice Colombia and Mexico Country Management Unit Latin America and the Caribbean Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS (Exchange Rate Effective April 3, 2015)

Currency Unit = CO$ CO$ 1.00 = US$ 0.00039 US$ 1.00 = CO$ 2,574.09

COLOMBIA - Government Fiscal Year

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

ADP Accelerated Data Program APL Adaptable Program Loan BPIN Banco de Proyectos de Inversión Nacional

National Investment Projects Bank CAS Country Assistance Strategy CPS Country Partnership Strategy COINFO Comisión Intersectorial de Políticas y Gestión de Información para la Administración

Pública Inter-Sectoral Committee for Information Policy and Management

CONPES Consejo Nacional de Política Económica y Social National Council for Economic and Social Policy

DANE Departamento Administrativo Nacional de Estadística National Statistical Office

DDTS Dirección de Desarrollo Territorial Sostenible Directorate for Sustainable Territorial Development

DEPP Dirección de Seguimiento y Evaluación de Políticas Públicas Directorate for the Evaluation of Public Policy

DIFP Dirección de Inversiones y Finanzas Públicas Directorate for Investment and Public Finance

DNP Departamento Nacional de Planeación Department of National Planning

DP Development Partner DSEPP Dirección de Seguimiento y Evaluación de Políticas Públicas

Directorate for the Evaluation of Public Policy FARC Fuerzas Armadas Revolucionarias de Colombia

Revolutionary Armed Forces of Colombia FINDETER Financiera Desarrollo Territorial

Territorial Development and Fiduciary Agency FM Financial Management GDP Gross Domestic Product GoC Government of Colombia GTZ Deutsche Gesellschaft für Technische Zusammenarbeit

German Agency for Technical Cooperation IADB Inter-American Development Bank ICT Information and Communications Technology IEC Inter-Sectoral Committee for Evaluation and Management for Results

IEG Independent Evaluation Group IIIE IMF

International Initiative for Impact Evaluation International Monetary Fund

IT Information Technologies JICA Japan International Cooperation KPI Key Performance Indicator LA Loan Agreement M&E Monitoring and Evaluation MDP Municipal Development Plan MHCP Ministerio de Hacienda y Crédito Público

Ministry of Finance and Public Credit MINTIC Ministerio de Tecnologías de la Información y las Comunicaciones

Ministry of Information Technologies and Communications MS Moderately Satisfactory MGR Matríz de Gestión de Recomendaciones

Recommendation Management Matrix MTEF Medium-Term Expenditure Framework NGO Non-government organization NDP National Development Plan PAD Project Appraisal Document PDO Project Development Objective PENDES Plan Estratégico Nacional de Estadísticas

National Strategic Plan for Statistics PHRD S

Policy and Human Resources Development Satisfactory

OECD Organization for Economic Co-operation and Development PRF Project Result Framework RAS Red de Apoyo Social/Red de Acción Social

Social Safety Net/Social Action Network SIGOB Sistema de Programación y Gestión por Objetivos y Resultados

System of Programming and Management by Objectives and Results SIIF Sistema Integral de Información Financiera

Integrated Financial Management Information System SINERGIA Sistema Nacional de Evaluación de Resultados de la Gestión Pública

National System for Evaluation of Public Sector Performance STATCAP Adaptable Lending Program for Statistical Development SUIFP Sistema Unificado de Inversiones y Finanzas

Public Finances and Investments Unified System TA Technical Assistance UN United Nations UNDP United Nations Development Programme USAID United States Agency for International Development

Vice President: Jorge Familiar Calderón Country Director: Gerardo M. Corrochano

Senior Global Practice Director: Ana L. Revenga Practice Manager: Louise J. Cord

Task Team Leader: Carlos Rodríguez Castelán ICR Team Leader: Carlos Rodríguez Castelán

REPUBLIC OF COLOMBIA Strengthening Public Information, Monitoring, Evaluation for Results Management

Project under the STATCAP (APL) Program

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 5 3. Assessment of Outcomes .......................................................................................... 11 4. Assessment of Risk to Development Outcome ......................................................... 23 5. Assessment of Bank and Borrower Performance ..................................................... 23 6. Lessons Learned ....................................................................................................... 25 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 26 Annex 1. Project Costs and Financing .......................................................................... 27 Annex 2. Outputs by Component ................................................................................. 28 Annex 3. Economic and Financial Analysis ................................................................. 32 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 34 Annex 5. Beneficiary Survey Results ........................................................................... 36 Annex 6. Stakeholder Workshop Report and Results ................................................... 37 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 38 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 53 Annex 9. List of Supporting Documents ...................................................................... 54

MAP

A. Basic Information

Country: Colombia Project Name:

Strengthening Public Information, Monitoring, Evaluation for Results Management in Colombia

Project ID: P099139 L/C/TF Number(s): IBRD-76200,TF-57066 ICR Date: 06/23/2015 ICR Type: Core ICR

Lending Instrument: SIL Borrower: GOVERNMENT OF COLOMBIA

Original Total Commitment:

USD 8.50M Disbursed Amount: USD 8.09M

Revised Amount: USD 8.09M Environmental Category: C Implementing Agencies: Departamento Nacional de Planeacion Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 04/18/2006 Effectiveness: 01/21/2009 01/21/2009

Appraisal: 12/10/2007 Restructuring(s): 09/09/2010 07/16/2012 11/26/2013

Approval: 12/12/2008 Mid-term Review: 09/12/2011 12/13/2011 Closing: 03/15/2014 12/31/2014 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Low or Negligible Bank Performance: Moderately Satisfactory Borrower Performance: Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Satisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies: Satisfactory

Overall Bank Performance: Moderately Satisfactory Overall Borrower

Performance: Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance Indicators QAG Assessments

(if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

No Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes Original Actual

Sector Code (as % of total Bank financing) Central government administration 96 96 Sub-national government administration 4 4

Theme Code (as % of total Bank financing) Managing for development results 40 40 Other public sector governance 20 20 Poverty strategy, analysis and monitoring 20 20 Public expenditure, financial management and procurement

20 20

E. Bank Staff Positions At ICR At Approval

Vice President: Jorge Familiar Calderon Pamela Cox Country Director: Gerardo M. Corrochano Axel van Trotsenburg Practice Manager/Manager:

Louise J. Cord Jaime Saavedra Chanduvi

Project Team Leader: Carlos Rodriguez Castelan Gladys C. Lopez-Acevedo ICR Team Leader: Carlos Rodriguez Castelan

ICR Primary Author: Maria Cecilia Zanetta F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project development objective is to assist the Government in strengthening its monitoring and evaluation system to ensure the availability and production of quality of information pertinent to improving the Borrower's: (i) program and policy design by the Inter-Sectoral Committee for Evaluation and Management for Results (IEC) and government entities; (ii) social accountability by the Office of the President; (iii) investment planning by DNP; and (iv) monitoring and evaluation capabilities, at the local and regional levels (PAD, p. ii, 8; Loan Agreement, Schedule 1, p. 5). Revised Project Development Objectives (as approved by original approving authority) "The objective of the Project is to assist the Borrower in strengthening its monitoring and evaluation system to ensure the availability and production of quality of information pertinent to improving the Borrower's: (i) program and policy design by the National Council for Economic and Social Policy (CONPES) and government entities; (ii) social accountability by the Office of the President; (iii) investment planning by DNP; and (iv) monitoring and evaluation capabilities, at the local and regional levels" (Project Restructuring Document, p. 2). (a) PDO Indicator(s)

Indicator Baseline Value

Original Target

Values (from approval

documents)

Formally Revised Target Values

Actual Value Achieved at Completion or Target

Years

Pre-restructuring Period1

Indicator 1 DNP collects information following strict protocols for data quality, reliability and timeliness on at least 60% of national spending made in regions by Year 3 of the project.

Value (quantitative or qualitative)

58% 60% Dropped 68.8%

Date achieved June 10, 2008 June 31, 2011 July 2012 December 31, 2011

Comments (incl. % achievement)

Exceeded (119% as of December 2011): The target was exceeded at Year 3. At restructuring, this indicator was dropped because it had been achieved and considered to be too similar to the newly added Intermediate Indicator 4, which raised the original target for the post-restructuring period and was also exceeded.

Indicator 2 IEC members as high level representatives of their Ministries have increased the number of programs evaluated by 50% over the project lifetime, with respect to the inventory of programs collected in the first year

1 The pre-restructuring PDO indicators were dropped under the 2012 Level 1 restructuring.


24% 50% Dropped 291%

Date achieved Dec. 17, 2008 March 15, 2014 July 2012 December 31, 2011


Exceeded (582% as of December 2011): The number of programs evaluated increased from 24 to 70 between 2008 and 2011. The indicator was dropped at restructuring since it was already achieved and surpassed. A total of 108 programs were evaluated by December 31, 2014.

Indicator 3

Presidencia, Hacienda and DNP report satisfaction, based on structured independent interviews (Years 3 and 5) and audits of the provision of quality information provided by SINERGIA (SIGOB and Evaluations) that will feed into performance based budgeting. A baseline audit will be performed at the beginning of the project


n.a. 70% Dropped

84% (SINERGIA Monitoring) and 94%

(SINERGIA Evaluation) of respondents considered information to be good, very good or excellent

Date achieved December 31, 2009 March 15, 2014 July 2012 December 31, 2012


Undetermined: A baseline was being constructed in 2011 but measurement was discontinued as the indicator was dropped due to the difficulties in measuring it bi-annually. However, the indicator is considered achieved, as the impact evaluation showed substantial improvement in satisfaction among institutional users.

Indicator 4 Leading NGO’s and civil society members report higher satisfaction levels in terms of access and quality of information on goals monitoring.


n.a. 28% Dropped

84% (SINERGIA Monitoring) and 94%

(SINERGIA Evaluation) of respondents considered information to be good, very good or excellent



Undetermined: A baseline was being constructed in 2011 but measurement was discontinued when the indicator was dropped during the 2012 restructuring due to the difficulties of conducting biannual reviews. However, the indicator is considered achieved, as the impact evaluation showed substantial improvement in satisfaction among both internal and external users.

Indicator 5

40% of recommendations of evaluations completed during project lifetime have been implemented by corresponding authorities at the end of the project. This will be verified by annual reviews implemented by the technical secretariat of IEC.

Value n.a. 40% Dropped 59.5%

(quantitative or qualitative)

Date achieved June 14, 2010 March 15, 2014 July 2012 December 31, 2011


Undetermined: A baseline was being constructed in 2011 but systematic measurement was discontinued as the indicator was dropped due to the difficulties of conducting biannual reviews. Analysis of four programs showed an average of 59.5 percent recommendation being adopted.

Post-restructuring Period2 Indicator 6 Percentage of increase in visits from citizens to SINERGIA.


0 13%

Added under Level

1 restructurin

g

724%

Date achieved July 31, 2012 December 31, 2014 July 2012 December 31, 2014


Exceeded: Usage of the SINERGIA public portal—measured by SINERGIA users that do not have an access code—has exceeded expectations, increasing from 44,932 visits in January 2012 to 325,276 in Dec. 2014.

Indicator 7 Percentage of increase in visits of the high government to control panels.


0 6%

Added under Level

1 restructurin

g

592%

Date achieved July 31, 2012 December 31, 2014 July 2012 December 31, 2014


Exceeded: Usage of the SINERGIA control panels by government officials—measured by SINERGIA users with access code—has exceeded expectations, increasing from 13,627 visits in January 2012 to 80,687 in Dec. 2014.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Component One

Indicator 1 DNP has developed and established the needed protocols, procedures, manuals and applications to support the inclusion of regional information in investment project formulation by Year 2.

2 The post-restructuring PDO indicators were added under the 2012 Level 1 restructuring.

Value quantitative or qualitative)

0% 100% Dropped 100%



Achieved (100% as of December 2011): A regionalized scheme is available and included in the formulation module (BPIN's website). Procedures, protocols and applications and manuals were developed and are now operational. The indicator was dropped as it had been achieved and considered no longer applicable after Year 2.

Indicator 2 DNP has implemented the systematic collection of regional data on investment projects by Year 2


0% 100% Dropped 100%



Achieved (100%): Public agencies were able to present data on budgetary execution for 2011 at the regional level, and to do budgetary planning at regional level for the 2012 budget and thereafter. The indicator was dropped as it had been achieved and considered no longer applicable after Year 2.

Indicator 3 DNP issues annually a regionalized budget. Value quantitative or qualitative)

0 100% Dropped 100%



Achieved (100%): Not only are annual budgets are now regionalized, but the Multi-Year Investment Plan is also regionalized under the National Development Plan 2014-2018, which constitutes a significant step toward regionalization. This indicator was replaced by Intermediate Indicator 4, which provides a more clear operational definition,

Indicator 4 Percentage of the total investment budget for which disbursements can be regionally tracked.


68.7 69.5 Added under

Level 1 restructuring

81.2

Date achieved July 16, 2012 December 31, 2014 July 2012 December 31,

2014


Exceeded (117%): It should be noted that the level of achievement is even greater when considering that 94.3% of the total investment budget that is subject to being regionalized could be regionally tracked in 2014. (13.5% of the 2014 investment budget was not subject regionalization). Note: This indicator was added under the 2012 restructuring.

Component Two

Indicator 5 IEC members as high-level representatives of their Ministries have increased the number of programs evaluated by 50% over the project lifetime, with respect to the inventory of programs collected in the first year


n.a. 50% Dropped More than 50%



Undetermined. The indicator was dropped during restructuring as it was already achieved and surpassed. Moreover, the IEC was not longer operational. However, the increase in the number of evaluations has exceeded expectations (see KPI 2).

Indicator 6 Levels of satisfaction with performance of SINERGIA of IEC members are 80% the end of the project. This will be included in the independent reviews in Years 3 and 5 of the Project.


n.a. 80% Dropped n.a.



Undetermined. A baseline was being constructed in 2011 but measurement was discontinued as the indicator was dropped due to difficulties in conducting bi-annual reviews. An impact evaluation of SINERGIA showed substantial improvement in satisfaction among institutional users between 2010 and 2012.

Indicator 7 100% of evaluation recommendations are monitored and followed up in a systematic way by DEPP by Year 5 of the project


n.a. 100% Dropped 100%

Date achieved June 2010 March 31, 2014 July 2012 December 31, 2014


Achieved (100%). This indicator was dropped during restructuring, as the evaluation process had been redefined in 2010 and this indicator was not considered useful in the new context. A new mechanism is being implemented by DEPP to systematically monitor recommendations identified as priorities by government units responsible for implementation of programs that were evaluated.

Indicator 8 40% of evaluation recommendations have been implemented by corresponding authorities at the end of the project. This will be verified by annual reviews implemented by IEC.


n.a. 40 Dropped 59.5%


2014


Achieved (149%): This indicator was dropped at restructuring due to the difficulties of conducting bi-annual reviews. However, analysis of four programs with a Recommendation Management Matrix shows that, on average, 59.5% of recommendations were adopted. Note: This indicator was added under the 2012 restructuring.

Indicator 9 Percentage of total evaluations in the approved agenda that reach the design phase.


60 80 Added under


88.5


2014


Achieved (111%): Implementation of the evaluation agenda has exceeded expectations, with 23 evaluations reaching the design stage in 2014. Note: This indicator was added under the 2012 restructuring.

Component Three

Indicator 10 Medium-term expenditure framework is presented to the municipal council yearly in the context of budget discussions in Medellin (Years 1 and 2).


0 100 Dropped 100

Date achieved June 30, 2009 Years 1, 2 July 2012 December 31, 2011


Achieved (100% as of Dec. 2011): Indicator was dropped as it was already achieved at restructuring.

Indicator 11 One municipal pilot and one department have designed and implemented 100% of the evaluation agenda.


0 100 Dropped 100



Achieved (100% as of Dec. 2011): By December 2011, two departments (i.e., Atlántico and Santander) and at least four municipalities had designed and implemented an evaluation agenda. This indicator was dropped as it was already achieved at restructuring.

Indicator 12 PBB annex is presented to the municipal council yearly with the budget in Medellin and Pasto (Years 1 and 2)


PBB annex was not presented PBB annex is

presented annually

Dropped PBB annex presented

Date achieved June 30, 2009 Years 1,2 July 2012 December 31, 2011


Achieved (100% as of Dec. 2011): Indicator was dropped as it was already achieved at restructuring and the component was restructured.

Indicator 13 Technological systems that integrate financial and physical monitoring of budget are implemented and functioning by the year 3 of project in Medellin and Pasto


No technological systems in use

Technological systems in use Dropped Technological

systems in use



Achieved (100% as of Dec. 2011): Indicator was dropped as it was already achieved at restructuring and the component was restructured.

Indicator 14 Index of survey results about usage and knowledge of PBB annex by municipal councilors has increased by 10% by Year 2 of Project.


n.a. n.a. Dropped n.a.

Date achieved June 30, 2009 Year 2 July 2012 December 31, 2011


Undetermined. Indicator was dropped as it was already achieved at restructuring and the component was restructured.

Indicator 15 DDTS and DEPP have jointly prepared one annual publication with technical guidelines on M&E practices at the territorial level (Years 1 and 2)


No document available Annual publication Dropped Document

published

Date achieved June 30, 2009 Years 1, 2 July 2012 December 31, 2011


Achieved (100% as of Dec. 2011): Document published “Experiencia de Pasto, Medellin y Tocancipa en la implementación del presupuesto por resultados.”

Indicator 16 DDTS has incorporated at least 50% of recommendations from the PHRD analysis to improve the methodologies for measurement of municipal performance by year 2 of the project


n.a. 50 Dropped n.a.



Undetermined. Indicator was dropped as it was already achieved at restructuring and the component was restructured.

Indicator 17

DDTS and DEPP have disseminated and trained local staff in the M&E methodologies in at least 2 municipalities and one department by Year 2 of the project (as verified by signed lists of attendance and number of training sessions).


0 7 Dropped 7



Achieved (100% as of Dec. 2011): Seven capacity building events were delivered by December 2011. Substantial capacity building was built among the 82 territorial units that participated in “SINERGIA Territorial,” including training workshops

and weekly sessions of on-site technical support. Indicator was dropped as it was already achieved at restructuring and the component was restructured.

Indicator 18 Territorial Entities with signed Memorandum of Understanding (MoUs). Value quantitative or qualitative)

0 56 Added under


86


2014


Exceeded (154%): Although the decision was made not to subscribe formal MoUs, a total of 86 territorial entities participated in SINERGIA Territorial, exceeding the target. Note: This indicator was added under the 2012 restructuring.

Component Four

Indicator 19 Strengthening managerial capacity of at least 50% of CIOs at the ministerial and sector level (as verified by number of training sessions provided and results summary)


None 50 Dropped 81



Achieved (162% as of Dec. 2011): Indicator was dropped at restructuring since DANE continued these activities in the framework of the National Strategic Statistics Plan (PENDES) outside the Project.

Indicator 20 100% of standards for data collection, production and use have been published and disseminated.


0 100 Dropped 100

Date achieved June 30, 2009 March 15, 2014 July 2012 December 31,

2011


Achieved (100% as of Dec. 2011): By 2011, 496 out of 1,167 statistical operations identified under PENDES (43%) had undergone a quality assessment by 2011. Indicator was dropped at restructuring since these activities were continued outside the Project.

Indicator 21 50% of selected agencies and sectors have implemented the corrective actions to improve their information flow processes by end of project.


6 50 Dropped n.a.



Undetermined. Indicator was dropped at restructuring since DANE continued these activities in the framework of the National Strategic Statistics Plan (PENDES) outside the Project.

Indicator 22 Number of Surveys with evaluations with PAD / Total surveys for evaluations (annual)


0 100 Added under


100

Date achieved

July 16, 2012 December 31, 2014 July 2012 December 31,

2014


Achieved (100%): Data collected under 31 evaluations conducted under the Project since 2011 have been documented according to the PAD standards. This is now a requirement for all SINERGIA Evaluations. Note: This indicator was added under the 2012 restructuring.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 02/27/2009 Satisfactory Satisfactory 0.00 2 11/06/2009 Satisfactory Satisfactory 0.91 3 05/24/2010 Satisfactory Satisfactory 1.87 4 02/15/2011 Satisfactory Satisfactory 2.98 5 08/26/2011 Satisfactory Satisfactory 3.94 6 03/13/2012 Satisfactory Satisfactory 5.38 7 11/17/2012 Satisfactory Satisfactory 5.69 8 06/26/2013 Satisfactory Satisfactory 6.05 9 03/12/2014 Satisfactory Satisfactory 7.46

10 10/17/2014 Satisfactory Satisfactory 7.91 11 01/13/2015 Satisfactory Satisfactory 8.22

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made DO IP

09/09/2010 N S S 2.11

A Level 2 Restructuring revised the funding allocations of Components 1, 3 and 4 to accommodate additional need for a significant number of training events, dissemination activities, and editing and publication of materials. The

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made DO IP

definition of “training” was revised accordingly in the LA.

07/16/2012 Y S S 5.56

A Level 1 Restructuring introduced: i) a slight change in the PDO to reflect change in institutional responsibilities regarding the approval of the evaluation agenda; ii) an overhaul of the Results Framework, including Key Performance Indicators to simplify it, enhance consistency between Project documents, and reflect changes being introduced at restructuring; iii) changes to individual components to reflect new strategic priorities; and iv) changes to the disbursement table.

11/26/2013 S S 6.45

A Level 2 Restructuring extended the closing date from March 14, 2014 until December 31, 2014, primarily to allow the completion of program evaluations that were important for the design of the NDP 2010-2014. It also reallocated resources between disbursement categories and components to support an expanded evaluation agenda.

If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Satisfactory Against Formally Revised PDO/Targets Satisfactory Overall (weighted) rating Satisfactory

I. Disbursement Profile

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal The Government of Colombia (GoC) had achieved considerable success in the creation and strengthening of its whole-of-government monitoring and evaluation (M&E) system: the National System of Evaluation of Public Sector Performance (Sistema Nacional de Evaluación de Resultados de la Gestión Pública - SINERGIA). The system, which was formally created in 1994, had its roots in seminal initiatives that went back to the early 1990s, such as the 1991 revision to the constitution mandating program evaluations. Like many M&E systems, SINERGIA has had its ups and downs. However, in the early 2000s, SINERGIA was brought back to the center of the government’s agenda as an indispensable tool for enhancing social accountability and the generation of quality information to inform government policy-making.

Support for the continuous development of SINERGIA went beyond the Office of the President (Presidencia), with other key stakeholders such as the National Department of Planning (DNP) and the Ministry of Finance (Ministerio de Hacienda y Crédito Público – MHCP) also exhibiting strong commitment toward strengthening the country’s M&E system. Concomitantly, the National Statistical Office (DANE) was pursuing a similar agenda under its National Strategic Plan for Statistics (PENDES), which included a variety of potent tools for strengthening the quality of information, including the certification of ministries and agencies for producing quality data.

In 2003, the Inter-Sectoral Committee for Information Policy and Management (COINFO) was created with the strategic objective to improve access to information by civil society and introduce strategies for the improvement of the quality of information by enhancing the coordination of IT policies throughout the government. The enthusiasm for quality M&E had also trickled down to local governments, as illustrated by the efforts of the municipal governments of Pasto and Medellin to build local M&E systems at the time of preparation.

The collective efforts of the various government entities over the previous five years had made SINERGIA one of the strongest M&E systems in Latin America, in terms of the range of M&E activities it covered and its utilization. However, significant challenges still remained, including: (i) lack of a single, clear conceptual framework; (ii) the need to clarify the roles and responsibilities of the ministries and agencies which support the government’s M&E system; (iii) absence of clear links between planning, budgeting and evaluation; and (iv) severe problems with the availability, frequency, and quality of data.

The Bank was well positioned to assist the GoC in its objective of further strengthening and institutionalizing SINERGIA. A Policy and Human Resources Development Fund (PHRD) grant that had been in place since early 2007 to support the preparation of this Project, had funded significant capacity building and consensus building among key stakeholders, giving the Bank thorough knowledge of Colombia’s M&E system and high credibility with the GoC. The Bank had also supported successful M&E activities within other lending operations.3 In addition, the Bank’s

3 The Bank supported several M&E activities under various lending operations, including the Programmatic Labor Reform and Social Structural Adjustment Loans (Loans No. 7193-CO; 7259-CO; 7400-CO); the Public Financial Management Projects (Loans No. 3669-CO; 7049-CO); the Social Safety Net Project (Loan No. 7337-CO); and the support for improved evaluation and quality of information in selected ministries through the Technical Assistance Loan (Loan No. 7260-CO).

1

Independent Evaluation Group (IEG) conducted a thorough diagnosis of Colombia’s central M&E system in 2006.

The Bank’s efforts to support the GoC in the strengthening of its M&E system lent itself to the utilization of a newly available lending vehicle (i.e., the STATCAP). The STATCAP was aimed to improve the statistical capacity in several countries through a sector-wide approach, through a Statistical Master Plan prepared and owned by the country. The GoC met all STATCAP eligibility conditions: (i) preparation of a national statistical development strategy and statistical master plan; (ii) strong national commitment and leadership; (iii) willingness to comply with good statistical practice; (iv) participation in global development and monitoring activities; and (v) use of sustainable financial and institutional arrangements. The key element of STATCAP was the government’s commitment to the implementation of a national statistical development strategy and approbation of the Statistical Master Plan, which, in the case of Colombia, was reflected in the development and adoption of PENDES. In this way, the Bank was able to support the CoG’s M&E efforts to consolidate, strengthen and expand SINERGIA as part of a wider multi-country program.

1.2 Original Project Development Objectives (PDO) and Key Indicators The PDO was to assist the Government in strengthening its M&E system to ensure the availability and production of quality information pertinent to improving the Government’s: (i) program and policy design by Inter-Sectoral Committee for Evaluation and Management for Results (IEC) and government entities; (ii) social accountability by the Office of the President; (iii) investment planning by DNP; and (iv) monitoring and evaluation capabilities, at the local and regional levels (PAD, p. 8; Loan Agreement – LA, p. 5). The Project’s Key Performance Indicators (KPIs) were defined as shown on Table 1.

Table 1: Original Key Performance Indicators KPI 1 - DNP collects information following strict protocols for data quality, reliability and timeliness on at least 80% of national spending in regions by end of project KPI 2 - IEC members as high level representatives of their Ministries have increased the number of programs evaluated by 50% over the project lifetime, with respect to the inventory of programs collected in the first year KPI 3 – 40 percent of recommendations of evaluations completed during project lifetime have been implemented by corresponding authorities at the end of the project. KPI 4 - Presidencia, Hacienda and DNP report higher satisfaction, based on structured independent interviews (years 3 and 5) and audits of the provision of quality information provided by SINERGIA (SIGOB and Evaluations) that will feed into performance based budgeting. KPI 5 - Leading NGO’s and civil society members, report higher satisfaction levels in terms of access and quality of information on public spending based on independent surveys (years 3 and 5 within SINERGIA reviews).

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The Project underwent a Level 1 restructuring in 2012, which introduced several changes, including introduced a slight modification of the PDO to reflect the GoC’s decision to replace the IEC with the Economic and Social Council (CONPES) as the government entity responsible for approving the evaluation agenda. The revised PDO read as follows:

“The objective of the Project was to assist the Borrower in strengthening its monitoring and evaluation system to ensure the availability and production of quality of information pertinent to

2

improving the Borrower’s: (i) program and policy design by CONPES and government entities; (ii) social accountability by the Office of the President; (iii) investment planning by DNP; and (iv) monitoring and evaluation capabilities at the local and regional levels.

At the GoC request, the restructuring also introduced comprehensive revision of the PDO and intermediate outcome results indicators to ensure consistency within the PAD and between the PAD and Loan Agreement, to simplify the results framework and to reflect the modifications proposed with this restructuring. The five original KPIs were dropped and replaced with two new KPIs, as shown on Table 2. Changes to Intermediate Outcome Indicators are indicated in the Datasheet (Section F(b)).

Table 2: Revised Key Performance Indicators KPI 6 - Percentage of increase in visits from citizens to SINERGIA. KPI 7 - Percentage of increase in visits of the High Government to Control Panels

1.4 Main Beneficiaries While the PAD does not explicitly identify the Project’s intended beneficiaries, the overall Project description points to, among others: i) the President, who would benefit from having at his/her disposal the information needed to discuss with the regions and line ministries the advances in terms of project and financial execution; ii) national authorities, which would be better equipped to support policy making with evidence-based data, including funding allocations among programs and regions; iii) government agencies, which would be able to use results from program evaluations in program design and redesign; and iii) civil society at large, as it would be able to rely on government information when engaging in dialogue on government priorities and performance.

1.5 Original Components 1 - Institutional strengthening for M&E of public investment projects at the national and

sub national level. This component provided support to the GoC in improving and strengthening the institutional capacity for the evaluation and monitoring of public investment. Activities were grouped in two sub-components, which focused on: 1.1 Institutional strengthening at the central level for the collection and analysis of national and

regional expenditure information; and 1.2 Information on regional indicators in existing regional module of public investment

information systems. 2 - Consolidate and expand the use of the national M&E system This component provided support to the GOC to institutionalize the central government M&E system and promote a greater use of monitoring information and evaluation findings for improving budget planning and management. Activities were grouped in two sub-components, which focused on: 2.1 Strengthening institutional arrangements for the government’s M&E system—the IEC and its

Technical Secretariat, the Directorate for the Evaluation of Public Policy (DEPP) in particular—and defining a strategy to better face increasing demands to support sector capacity and strategically focus its efforts; and

2.2 Increasing the utilization of monitoring and evaluation findings in the planning and budget processes. 3 - Support the development of local and regional M&E systems

3

This component provided support to the GoC to develop and strengthen local and regional M&E capacity. Activities were grouped in three sub-components, which focused on: 3.1 Developing and consolidating the M&E municipal pilot systems in Medellin and Pasto and

use them for planning, budgeting, accountability, and social control; and identify conditions for their expansion to other municipalities;

3.2 Strengthening the management capacity of the departmental government of Choco in the areas of planning and budgeting oriented by results; and

3.3 Promoting the harmonization of technical M&E concepts between the nation (central sectors, ministries and agencies), and the municipal M&E systems for increased coordination and utilization of M&E information.

4 - Establish the mechanisms to improve quality and relevance of public information This component provided support to the measures taken to improve the quality of the public sector information flow process―generation, use, distribution, and storage―by strengthening institutional, legal and regulatory frameworks, and technical areas. Activities were grouped in two sub-components, which focused on: 4.1 Modernization and strengthening of the information-flow process; and 4.2 Modernization and integration of the information-flow’s technical aspects, systems, and

delivery channels, by setting common standards in the information process of SINERGIA. 5 – Project Administration This component provided support to manage the implementation of the different components and activities in the loan.

1.6 Revised Components 1 - Institutional strengthening for M&E of public investment projects at the national and

sub national level. There were no changes made to this component. 2 - Consolidate and expand the use of the national M&E system 2.1 Activities related to the improvement of the functioning of the IEC and its technical

secretariat, the DEPP, were eliminated since the IEC had not been operational up to this point of the implementation of the loan, and instead CONPES had been responsible for approving the evaluation agenda.

2.2 No change. 2.3 A third sub-component was added to support the development and piloting of an open data

platform with four databases of the DNP. 3 - Support the development of local and regional M&E systems 3.1 No change. 3.2 No change. 3.3 No change. 3.4 A new subcomponent (3.1d) was added aimed at supporting the development of SINERGIA

Territorial for the expansion and mainstreaming of M&E to departments and municipalities. 4 - Establish the mechanisms to improve quality and relevance of public information 4.1 Activities in this component were limited to supporting the modernization and integration of

the information-flow’s technical aspects, systems, and delivery channels, by setting common standards in the information process of SINERGIA. Support for other areas was terminated, including: DANE, as the CoG considered that this agency needed a much larger, integral, stand-alone project to support it; the development of a new system of access to public information, as such system had been developed by the DNP and the Ministry of Technologies and Communications, with support to public information issues will be carried out under a new subcomponent (sub-component 4.1c below)

4

4.2 No change. 4.3 A new subcomponent (4.1.c) was added to complement the evaluation of public information

management and use its findings to prepare a National Policy on Open Data to be approved by CONPES.

5 - Project Administration This component provided support to manage the implementation of the different components and activities in the loan.

1.7 Other significant changes The Loan Agreement was signed on December 19, 2008 and declared effective on January 21, 2009. It was amended three times during the Project’s lifetime, as follows: • A Level 2 Restructuring was approved on September 9, 2010 in response to the GoC’s request

to revise the funding allocations of Components 1, 3 and 4 in order to accommodate the additional need for a significant number of training events, dissemination activities, and editing and publication of materials. In addition, the definition of “training” was revised accordingly in the Loan Agreement.

• In addition to the change in the wording of the PDO and the modifications to the Results Framework, the Level 1 Restructuring that was approved on July 17, 2012 introduced other changes aimed at providing more flexibility in use of project funds and reflecting new Government priorities. Project components were modified, with new activities being incorporated (e.g., activities related to Open Data and “SINERGIA Territorial”) and activities no longer critical to the achievement of the PDO being removed (see Section 1.6). The restructuring also simplified the disbursement table included in the Loan Agreement to provide greater flexibility in the administration and management of Project funds.

• A Level 2 Restructuring that was approved on November 26, 2013 reallocated resources between disbursement categories and components to support an expanded evaluation agenda. It also extended the closing date from March 14, 2014 until December 31, 2014 to allow for the completion of the envisioned evaluations.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry a. Soundness of the background analysis Although lengthy, the preparation stage was particularly important in the context of this operation, as it not only provided needed background analysis to support the operation’s design but also helped build substantial capacity and consensus among the main stakeholders within Colombia. With support from a Trust Fund (TF) from Japan’s Policy and Human Resources Development (PHRD) Fund, several activities were conducted as part of Project preparation that helped government officials gain a better understanding of the challenges ahead, including study tours and high-level seminars focusing on performance-based budgeting, municipal performance-based participatory budgeting, and results-based management as well as training provided in advanced techniques for impact evaluation. These activities also helped align funding coming from various DPs, including the Inter-American Development Bank (IADB), and the United States Agency for International Development (USAID), and the German Agency for Technical Cooperation (GTZ) towards the objectives of developing a whole-of-government M&E system.4 In addition to the

4 The IADB funded several individual impact evaluations and provided technical assistance in the context of the PRODESEV initiative; the GTZ focused on providing technical assistance to strengthen institutional

5

diagnostic studies conducted under the PHRD grant, the Project’s design benefited from two different studies that had provided exhaustive diagnosis of the GoC’s M&E system (CONPES, 2004; IEG, 2007). b. Assessment of the Project design Project design exhibited both strengths and weaknesses. In particular: Strengths • The sector-wide approach required under the STATCAP program was well suited for

Colombia, as the development of a whole-of-country M&E system went beyond the orbit of single government agency. The development of such an M&E system also had the support of multiple donors in crucial aspects of its M&E, which enabled the GOC to draw on diverse sources of M&E experience.

• The operation’s design reflected a holistic vision of an M&E system that went beyond the technical perspective to include also political economy aspects that would ultimate determine the acceptance and utilization of the system. In this regard, the engagement of key stakeholders, including the Office of the President, as part of project preparation was crucial in building government-wide support. The emphasis on the utilization of the M&E system was also reflected in the Project Result Framework.

• The operation’s design effectively capitalized on two of the important areas of Bank expertise. On the one hand, it built upon the experience of public sector operations (in particular, the Public Financial Management Project II – PFMP II),5 bringing a systemic perspective to the task of developing management tools of-government M&E system. On the other hand, it effectively capitalized on the in-depth expertise in impact evaluation within the Bank’s poverty network and the momentum generated by the evaluation of highly visible social programs, including the emblematic Familias en Acción Program.6

• The choice of instrument was appropriate given that the only options at time were lending instruments or analytical or non-lending technical assistance. It is important to note, however, that if the same Project were to be prepared today, Reimbursable Advisory Services (RAS) would be a more adequate instrument.7

Weaknesses • The operation was highly ambitious, particularly when considering the relatively small

amount of the loan. It had a wide scope and involved several large government entities at the central level (i.e., DANE and DNP) as well as departmental and municipal governments. Although this complexity certainly posed a significant risk to the Project’s overall

capacity in the most vulnerable municipalities in the country; and USAID provided technical assistance and funding in the design and development of selected M&E systems at the sub-national level. 5 Approved March 22, 2001, the PFMP II (Loan 7049-CO in the amount of US $35.47 million) and its predecessor (the PFMP I) supported reforms in revenue administration and public expenditure management at the national level, including the initial stages in the development of the SUIFP and, although in a very limited manner, the strengthening of SINERGIA. 6 The expansion of the Familias in Acción Program received joined support from both the IADB and the Bank (IBRD support was under the Social Safety Net Project (Loan 7337-CO in an amount of US$84.6 million). This program provided cash transfers to poor families conditional on their children’s enrollment in basic education and visits to health centers for basic care. In addition to the impact evaluation of the expansion of Familias en Acción by the DNP, the operation included a component to improve the monitoring and evaluation systems of the Ministry of Social Protection in order to better track and critically review the country’s social safety net portfolio. 7 Under RAS programs, the World Bank works with middle- and high income countries at their request, providing technical advice, analytical and convening services, and implementation support. The Bank is then reimbursed for the costs of delivering these advisory services.

6

effectiveness, the outstanding technical and institutional capacity of DANE, the DNP, and its directorates served to counteract this risk.

• Some of the areas of intervention, such as DANE’s efforts to improve the quality and relevance of public information across the government under PENDES were overly complex and may have merited dedicated support under a separate operation. Moreover, the original implementation arrangements in which the administration of all Project funds was under the DNP, required a high degree of coordination between DANE and the DNP.

• While some components were clearly defined in terms of both the specific objective as well as the activities to be supported under the Project (in particular, the strengthening of the M&E system for public investments supported under Component 1), others were less clearly delineated and took shape only during implementation.

• There were also weaknesses in the PRF, mainly a dependence of some indicators on results from an impact evaluation, ambiguous operational definitions, as well as an unduly large number of indicators, particularly in term of intermediate outcomes (see Section 2.3).

c - Relevance at Appraisal The assessment of the GoC’s commitment toward enhancing its M&E system was correct. The strong commitment from the Presidency and key government agencies, particularly DANE and DNP were a key element in the successful implementation of the Project. The Project design was highly relevant, as it addressed the main weaknesses that had been identified during preparation. It built upon the institutional framework in place at the time, providing support to the various institutional actors in relation to their assigned roles and responsibilities. Particular emphasis was given to the strengthening of the newly established IEC and DEPP, its technical secretariat. Likewise, by including relevant directorates within DNP, the operation contributed toward creating a single, clear conceptual framework for SINERGIA. Also, it addressed existing problems with the availability, frequency, and quality of data in the context of the SUIPF and SINERGIA systems as well as government-wide data under PENDES. The operation was also consistent with two of the three pillars outlined in the Bank’s Country Assistance Strategy (CAS) for Colombia, approved in December 2002: (i) building quality government; and (ii) sharing the fruits of growth. As a result of the full institutionalization of the government’s M&E system and improved and higher utilization of M&E information, the operation was expected to contribute to the CAS objectives by: (i) building quality government though a greater performance orientation, leading to more efficient and effective use of the country’s scarce resources and increased government accountability; and (ii) sharing the fruits of growth through a more egalitarian allocation of resources across geographical regions and population groups. d. Assessment of risks The Project was correctly identified as having Moderate risks at Appraisal. The main risks were: i) the underutilization of results of the M&E system by the government at both national and sub-national levels and civil society; and ii) a potential disruption during the change of administration in August 2010. None of these risks materialized as the Project was built on strong ownership on the part of the Borrower. The risk that the scope of the operation might be too broad was not identified.

2.2 Implementation Major Factors Affecting Implementation

Favorable economic and political implementation environment: Stable economic and political conditions rendered a favorable implementation environment. Colombia weathered the 2008-2009 financial crisis rather well, with GDP growth falling to less than 2 percent in 2009 but recovering thereafter. From 2010 onward, Colombia exhibited a robust economic performance with the support of a strong policy framework—anchored by an inflation-targeting

7

regime, a flexible exchange rate, a structural fiscal balance rule, and effective financial supervision and regulation—that has allowed GoC authorities to respond adequately to shocks and pursue effective demand management, with real GDP growing consistently at a pace above 4.0 percent during 2010-2014 (IMF, 2014, 2015). The presidential transition in 2010 was a smooth one, particularly in relation to this operation, as the incoming President Juan Manuel Santos also gave high priority to the continuous development of an effective M&E system. The re-election of President Santos to a second term in office has also provided stability in the country’s agenda and strategic priorities. Although the opposition increased its representation, the ruling coalition maintained a working majority in Congress in the congressional elections held in March 2014, Colombia held congressional elections. Lastly, the peace talks between the GoC and the main guerrilla group, the Revolutionary Armed Forces of Colombia (FARC), that began in November 2012 are proceeding and agreement has been reached on three of the five topics under discussion. Sector-wide approach: During implementation, DNP—as IEC, and later, CONPES Technical Secretariat—took over the coordination of the various efforts in strengthening Colombia’s SINERGIA that were being carried out under its various Directorates. Although it was ultimately decided that DANE’s program would be better managed outside this operation, coordination between DANE and DNP continued in areas of shared responsibility, such as the development of poverty measures. DNP—rather than the Bank—played a central role in coordinating sector-wide efforts, as it is indeed desirable, particularly in the case of a sophisticated Borrower such as Colombia. Great fluidity in the GoC’s M&E strengthening agenda: Despite broad support among key constituencies, the strategy toward strengthening Colombia’s M&E system underwent several changes during the operation’s lifetime, including the GoC’s decision to reassign responsibility for approving the evaluation agenda from IEC to CONPES. In addition, DNP and the Ministry of Technologies and Communications (MINTIC) developed a new system of access to public information, which rendered some of the activities initially formulated in the PAD no longer relevant. It is important to underscore that the changes did not impact the Project, as strong commitment toward enhancing M&E tools continued under the new institutional arrangements. Cancellation of DANE’s activities: During the pre-restructuring period, DANE was responsible for the implementation of activities under its orbit, although the administration of the corresponding loan proceeds was DNPs’ responsibility. As part of the restructuring, the DNP requested that the Project no longer provided support to DANE, indicating that this agency merited a much larger, integral, stand-alone project to support it. Moreover, the administration arrangements proved cumbersome for both entities, as DANE’s activities involved a large number of relatively small contracts. Other factors contributing to this decision were changes in DANE’s top administration, newly found fiscal space within DANE for the financing of PENDES activities, and the desire on the part of both DANE and DNP to avoid the cumbersome institutional coordination required to procure the numerous (albeit relative small) contracts related to DANE’s activities. As mentioned earlier, collaboration between DNP and DANE continued in areas of joint responsibility. Strong sense of direction of the part of DNP: The operation benefited not only from particularly strong ownership on the part of the key stakeholders in the Colombia’s public administration, but also from DNP’s clear vision of the future of SINERGIA. This vision, together with DNP’s strong technical and institutional capacity, were clearly the most important factors contributing to the achievement of the operation’s PDOs. Likewise, as the operation was implemented directly by DNP’s technical and administrative units, the operation has resulted in the strengthening of DNP, including FM and procurement functions. Substantial delays in the processing of the restructuring: Although the need to restructure the Project was identified as early as in 2010, the DNP did not initiate the formal process until

8

November 2011, when a formal request to process the restructuring was submitted to the MHCP. The Bank sent a proposed amendment to the GoC in September 2012, which was formally signed by Colombian authorities only in February 2013. These delays in processing the restructuring slowed down disbursements and may have contributed to the need to extend the original closing date for another nine months. However, it is important to note that the delay in the restructuring was not the only reason for the extension of the closing date. Primarily, it was a strategic decision to extend the closing date in order to make sure that results of the evaluations could feed into the NDP of President Santos’ second administration. Frequent changes in Project management within both the DNP and the Bank: There were a total of five Team Task Leaders responsible for the Project during the operation’s lifetime. Likewise, there were six different Project coordinators on DNP’s part. Despite everyone’s best efforts to ensure a smooth transition to his/her successor, the high level of rotation disrupted continuity in the implementation process. This was one of the contributing factors to the delays in the restructuring as well as the need to extend the operation’s closing date from March to December 2014. Changes within the DNP were also contributing factors to the need to extend the PHRD grant twice during preparation. The operation as leverage for additional financing and expertise: The operation served to secure financing for the strengthening of the M&E tools under SINERGIA, including US$15.8 million directly allocated to the Project (i.e., US$8.5 million in IBRD financing and US$7.3 in an earmarked budgetary allocation to DNP). In addition, the operation served as leverage to attract financing from other sources. The development of the SUIFP, for example, received additional financing from IADB, USAID, as well as US$1.2 million from the Bank’s Consolidation of National Public Management Information Systems Project. Likewise, the Project served to attract substantial additional financing for program evaluation, as a Memorandum of Understanding was signed with both IADB and the Bank to guarantee that all program evaluations financed by these institutions be conducted as part of SINERGIA Evaluation. Likewise, DNP authorities effectively cooperated with other government agencies interested in evaluating programs under their orbit by providing evaluation expertise and/or financing in exchange for including the program evaluation under SINERGIA Evaluation. This was a highly effective strategy on the part of the DNP for expanding the evaluation agenda and for acquiring evaluation expertise beyond social programs. The operation effectively used its resources to forge critical strategic alliances: DNP authorities were highly effective in providing technical support to key stakeholders, such as the Office of the President, to help them design and implement management tools to fully capitalize on the M&E information being generated by SINERGIA. In particular, different types of dashboards were developed for various users, including the Office of the President, line ministers, and the monitoring of cross-sector themes under the NDP. With Bank technical support, a strong priority was also given to the development of an effective website and user interfaces for SINERGIA. In addition, perception surveys were designed and implemented to identify differences between actual and citizens’ perceived achievement of the NDP’s targets. These tools, and most importantly the responsiveness of the DNP to the needs of SINERGIA’s final users both within and outside the government, were critical in expanding its utilization.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Monitoring and Evaluation (M&E) Design Project Result Framework (PRF): The Project had a weak PRF, both as originally designed and as revised during the 2012 restructuring. Specifically: Original PRF: The main weaknesses in the original PRF included: • There was not a full alignment between the PDO definition and the KPIs, as some of the

dimensions embedded in the PDO (i.e., data quality, pertinence) were not captured.

9

• The indicators did not have a clear operational definition—rather, they were loose definitions open to interpretation (i.e., KPIs 3 and 4 in particular).

• Some of the indicators were overly dependent on the results from the impact evaluation (i.e., KPI 3 and 4), including the prompt construction of the corresponding baseline.

• There was an excessive number of indicators, particularly in terms of Intermediate outcomes (IOI), with an excessive focus on outputs and activities that made reporting cumbersome without adding substance.

Revised PRF: The revised KPIs showed some improvement, most notable clearer operational definitions as well as easier to measure and fewer indicators. However, some of the weaknesses persisted. In particular, the two revised KPIs focused on access to SINERGIA information portals for government officials and the general public. Although usage of the information can be considered as a proxy for quality, availability, and pertinence of the information (in all likelihood, the use of the system would not have increased so much if those attributes were not present) it would have been desirable to include other indicators focusing directly on the various dimensions of the PDO. To mitigate this weakness, this ICR has relied on additional evidence to assess PDO achievement, much of which was obtained from the M&E tools developed under the Project. Impact evaluations: Two evaluations of SINERGIA were envisioned, one at mid-term (Year 3) and one in-depth evaluation at completion (Year 5). These evaluations were to include, among others, surveys of key stakeholders to assess the credibility, reliability and utilization of SINERGIA. The inclusion of impact evaluations as part of the M&E design was a strength, although a closer alignment with the evaluation’s design and the Project’s PDO would have been required. Monitoring and Evaluation (M&E) Implementation PRF monitoring: KPI reporting was weak during the pre-restructuring period, partially because the baselines for several of the indicators were still under construction. It improved significantly during the post-restructuring period. Impact evaluations: There were delays in the implementation of the impact evaluations, with the baseline and Mid-Term Impact Evaluation of SINERGIA were carried out in 2010 and 2013, respectively. Extensive interviews were conducted under the Mid-Term evaluation with a wide spectrum of actors within GoC’s public sector at different management levels (i.e., 266 interviews) and a sample of 24 external actors.8 The end-of-Project impact evaluation was not carried out. Monitoring and Evaluation (M&E) Utilization PRF utilization: As mentioned earlier, during the pre-restructuring period the PRF was not properly monitored, partly due to weaknesses in the PRF design. The PRF did not provide particularly useful information during the post-restructuring period, as the two KPIs were exceeded soon after restructuring. Impact evaluations: The baseline and Mid-Term evaluation proved critical to conducting the Project ex-post evaluation under this ICR, even though the evaluation did not fully capture the Project’s PDO. Having the final impact evaluation would have enhanced the measurement of the Project’s impact at completion.

8 The selection of respondents was based on: i) a census of all top-government officials, including planning heads of national departments, directors and sub-directors within DNP, managers of SINERGIA targets, and directors of programs that had been evaluated (a total of 266 respondents); and ii) sampling of DNP staff and external users (a total of 38 interviews for the assessment of SINERGIA Monitoring and Evaluation) and staff in 16 municipalities (a total of 154 interviews for the assessment of SINERGIA Territorial).

10

2.4 Safeguard and Fiduciary Compliance Environmental and social safeguards No environmental or social safeguards were triggered during the preparation of the Project, which was rated Category C. Financial Management (FM) The project Financial management (FM) risk was assessed as Moderate. Throughout implementation, FM performance was assessed as Satisfactory, except during the supervision carried out in February 2012 when the FM rating was downgraded to Moderately Satisfactory as a result of some possible ineligible expenditures, the need to update the operational manual and deficiencies in the monitoring of the project budget and reporting by project components. These issues were quickly and adequately resolved and the rating was upgraded to Satisfactory during the subsequent FM supervision. The project FM risk was assessed as Moderate. The audit reports were submitted within the contractual date. From 2009 to 2011 the project was audited by the Contraloria General de la Republica, and from 2012 through 2014 by private audit firms, and except for 2011, the auditors issued unqualified (clean) opinions. The auditors’ qualification on the 2011 project financial statements did not have an impact on the project financial situation of the project, and the issues qualifying the opinion were adequately documented by DNP. Furthermore, DNP implemented the recommendations of the audit reports on internal controls in a timely manner. Similarly, acceptable interim unaudited financial reports were submitted within the contractual date. Procurement As this was the first Bank operation to be implemented by DNP, procurement faced similar issues as financial management. The Bank provided capacity building on procurement procedures and policies throughout the project implementation, which ultimately strengthened the administrative capacity of DNP. In general, procurement and contract administration were performed in accordance with the agreed procedures and documents and the approved procurement plan. However, minor weaknesses in procurement procedures regarding the selection process persisted throughout the lifetime of the project, resulting in a Moderately Satisfactory rating during most of the implementation. Nonetheless, DNP was able to comply with agreed action plans in a timely manner to resolve any procurement issues and prevent them from impeding overall Project implementation. Legal Covenants There were no covenants applicable to Project implementation.

2.5 Post-completion Operation/Next Phase The GoC has already allocated budgetary resources to DNP for the continuous strengthening of SINERGIA at the national and regional levels. In particular, a total of US$20.4 million have been allocated to the DNP over the 2015-2018 period to continue with the development and consolidation of SINERGIA Monitoring and SINERGIA Evaluation. Activities focusing on the strengthening of M&E capacity at the subnational level under SINERGIA Territorial continue with financial support from the Subnational Institutional Strengthening Project (Loan 8320-CO), which was approved on December 2013 for an amount of US$70.0 million. DNP’s Directorate for Territorial Sustainable Development is responsible for its implementation with technical assistance from Colombia’s Territorial Development and Fiduciary Agency (FINDETER).

3. Assessment of Outcomes

11

3.1 Relevance of Objectives, Design and Implementation Relevance of Objectives: High (during both pre- and post-restructuring, since the PDO is essentially the same)

The operation’s PDOs are fully consistent with the 2014-2018 NDP – All for a New Country: Peace, Equity and Education.9 Specifically, by providing relevant, high-quality and timely information, the operation indirectly supported policymaking under the three pillars of the NDP—i.e., peace, equity and education. It was also directly aligned with the transversal strategy focusing on good governance. It is also important to underscore that the two main innovations within the current NDP underscore the importance of the PDOs supported under this operation: a stronger focus on results, and an operationalized territorial dimension. In terms of results focus, the 2014-2018 NDP is the first to have a built-in value chain linking products, intermediate outcomes, and main outcomes. In terms of the territorial dimension, this is the first NDP to have indicators and targets defined at the regional level (NDP, 2015).

The operation’s PDO also continues to be relevant in the context of the current World Bank’s Country Partnership Strategy (CPS) for Colombia (i.e., CPS FY2012-2016, Report No. 60620-CO, discussed by the Board of Executive Directors on July 21, 2011). Specifically, the operation’s PDO is fully aligned with one of the three areas of results—i.e., Improved Public Sector Management and Equity and Efficiency of Economic Policies—under one of the three main strategic themes—i.e., Inclusive Growth with Enhanced Productivity. As noted in the CPS, by helping the GoC to increase the use of information for policy design and accountability in a cost-effective way, this operation has addressed the challenges of low accountability and weak basis for managerial decisions. Moreover, the focus on M&E at both the national and subnational levels was particularly attuned to the priority being given to decentralization. Finally, the CAS stressed the Bank’s commitment to continue to provide technical support to the NDP in consolidating results-based public management and rigorous evaluation of public programs and policies (World Bank, 2011).

Relevance of Original Design: Substantial The operation’s design continues to be largely relevant. The focus on strengthening M&E in relation to public investments under SUIFP, progress toward the NDP under SINERGIA Monitoring, and the evaluation of government programs under SINERGIA Evaluation continues to be highly relevant. Likewise, the original strategy of providing support to a small number of reform-prone municipalities and departments reflected the conditions at the time of Project preparation. Strengthening the M&E capacity of subnational governments continues to be a highly relevant area of intervention, particularly in the context of ongoing decentralization. Finally, while further enhancing the quality of government statistics is still an important area of intervention, the GoC’s strategy has changed significantly, with the newly established MINTIC now playing a central role in this area. Relevance of Revised Design: High The operation’s revised design continues to be relevant. As mentioned above, the focus on M&E in the areas of public investments, monitoring of the NDP, program evaluation continues to be relevant areas of intervention at both the national and subnational levels. Moreover, other areas

9 The 2014-2018 NDP focuses on three pillars: i) peace, ii) equity, and iii) education. In turn, five transversal strategies contribute to the achievement of these pillars: i) competitiveness and strategic infrastructure; ii) social mobility; iii) transformation of the countryside; iv) security, justice and democracy for peace building; and v) good governance.

12

supported under the revised design, such as the open government data, are placing Colombia at the forefront of information management policies.

3.2 Achievement of Project Development Objectives PDO achievement is deemed to be Substantial to High, as the Project has effectively supported its objective of assisting the GoC in strengthening its monitoring and evaluation system. As characterized by OECD, SINERGIA Monitoring is, by Latin American and OECD standards, one of the most advanced whole-of-government performance-monitoring systems used in the world today. The system tracks policy performance government-wide using an elaborate series of performance-based indicators covering the entire spectrum of activity related to the implementation of the NDP (OECD, 2013, p. 171). Likewise, the Project effectively supported the implementation of other M&E tools, including SIUFP, SINERGIA Evaluation, and SINERGIA Territorial focusing on public investments, program evaluation, and the strengthening of M&E capacity among subnational entities.

For the purpose of this ICR, PDO achievement is assessed against the three different dimensions embedded in the operation’s overall PDO definition:

PDO 1 – Availability of public information PDO 2 – Quality of public information PDO 3 – Pertinence of public information for improving: a. Program and policy design by IEC/CONPES and government entities; b. Social accountability by the Office of the President; c. Investment planning by DNP; and d. Monitoring and evaluation capabilities at the local and regional levels.

The overall outcomes of the operation in relation to the various PDOs can be summarized as follows:

PDO 1 – Availability of public information (i.e., Project’s Intermediate Outcomes) High Achievement: With Project support, there have been significant improvements in terms of the amount, type, frequency, and ease of access to public information available under SINERGIA and the SUIFP. Specifically:

Public investments • The SUIFP compiles detailed information on public investment projects at all stages of

the project cycle, including project documentation, approval process, sources of funding and modifications in the allocations (i.e., national budget, royalties, local resources, etc.). Information on individual investment projects, such as implementation progress or modifications, is available through the Project Monitoring System (SIP) website.

• The number of investment projects incorporated each year into the SUIFP has grown significantly during the operation’s lifetime (2008-2014): i) from none to 1,277 under SUIFP with financing from the national budget; and ii) from none to 2,769 under SUIFP with financing from the Royalty System.

KPI 1 – DNP collects information following strict protocols for data quality, reliability and timeliness on at least 60 percent of national spending made in regions by Year 3 of the project. (144% level of achievement). • There has been substantial progress toward the regionalization of public investments,

with 81.2 percent of the total investment budget for which disbursements could be regionally tracked in 2014. This is equivalent to 94 percent of the investment budget that

13

is subject to being regionalized, as 13.5 percent of the total investment budget was not subject to regionalization.

SINERGIA Monitoring • Periodic reports are issued on the progress in the implementation of the NDP utilizing

quantitative and qualitative information derived from SINERGIA Monitoring dashboards, including the Annual Report to Congress and the President, Balance of Results, and Thematic Periodic Progress Reports.

• Progress toward the achievement of NDP targets is also publicly available in SINERGIA Monitoring’s website. In addition to more detailed information on individual programs presented in individual data sheets (I.e., program description, responsible entity, objectives, targets, quantitative and qualitative progress), information on progress under the various thematic areas, programs and individual targets is communicated effectively through color-coded dashboards. A total of 31 dashboards have been developed, including a dashboard for the President’s Office, 6 cross-sectoral dashboards, and 24 sectoral dashboards.

• Average composite scores of SINERGIA Monitoring’s website range from 3.4 to 3.7 in a scale of 1 (low) to 5 (high) in the Impact Evaluation (see Figure 1).

Figure 1. Users’ assessment of SINERGIA Monitoring’s website

Source: DNP (2014).

• The number of indicators under SINERGIA Monitoring has increase from 500 in 2007 to 1,100 in 2014.

• The frequency with which indicators are measured has also increased, with the percentage of indicators measured every six months or less increasing from 71 to 83 percent between the NDP 2006-2010 and the NDP 2010-2014 (see Table 3).

Table 3. Change in the frequency of measurement of DNP indicators

Frequency NDP 2006-2010

NDP 2010-2014

Annual 29% 17% Semi-annual 11% 16% Quarterly 27% 30% Monthly 33% 37%

Source: DNP (2015).

• Reporting on progress toward the achievement of DNP targets is now updated monthly as required under Presidential Directive 021 of 2011. This Directive requires all entities in the public administration to update progress toward NDP indicators in

14

SINERGIA Monitoring within the first ten days of each month. This Directive has contributed to enhancing the timeliness of the information under SINERGIA Monitoring.

• The 2014-2018 NDP is the first development plan to be regionalized, with performance targets being defined at monitored at the regional level. This constitutes a significant landmark in the development of Colombia’s M&E system, which, until this point, only had a national scope.

SINERGIA Evaluation KPI 2 - IEC members as high-level representatives of their Ministries have increased the number of programs evaluated by 50 percent over the project lifetime, with respect to the inventory of programs collected in the first year. (300 percent level of achievement.) • The number of programs evaluated increased from 24 to 108 between 2008 and 2014. • The scope of the evaluation agenda also increased significantly with Project support,

increasing from only two sectors to 16 sectors in 2014 (se Figure 2).

Figure 2. SINERGIA Evaluation – Number of evaluations by year and sector

Source: DNP (2014).

• The new “Radar” initiative developed under SINERGIA Evaluation serves as an

evaluations clearinghouse, systematically compiling evaluations performed by public entities. Currently there are 171 evaluations available for downloading at the SINERGIA Evaluation website and an additional 530 are being reviewed.10

• Users’ average composite scores of SINERGIA Evaluation’s website range from 3.6 to 3.9 in a scale of 1 (low) to 5 (high) (see Table 4).

Table 4. Users’ assessment of SINERGIA Evaluation’s website

10 https://sinergia.dnp.gov.co/SISDEVAL/paginas/Evaluaciones/RadarEvaluaciones.aspx

15

Source: DNP (2014).

SINERGIA Territorial • 63 of the 95 territorial entities that participated in SINERGIA Territorial now have

performance dashboards and 63 of them generate regular monitoring reports. Other results • The Project contributed to restoring long-term comparability in the measurement of

poverty after a disruption during 2006-2007 as a result of changes in data collection instruments. In addition, DANE has assumed responsibility for calculating and publishing the results of the official measurements of poverty in Colombia, including traditional measures of monetary poverty as well as multi-dimensional poverty. Multi-dimensional poverty measurements have been available since 2011.

• The impact of the Project with regards to the measurement of poverty went beyond methodological improvements, contributing as well to greater transparency in the governance of poverty measurement. In particular, it provided support to the establishment and early work of the Monetary Poverty and Multi-dimensional Expert Poverty Committees, including the financing of national technical experts to sit on the committees and support the development of the new methodologies.

• In 2013, the DSEPP began the documentation of databases according to international standards and best practices under the Accelerated Data Program (ADP), including ensuring the anonymity of micro-data to encourage open dissemination of information. 11 By 2014, there were 31 documented databases. In addition, the documentation of databases has been included as a requirement under all evaluations conducted under SINERGIA Evaluation.

• With Project support, the DNP is now routinely carrying out "Public Perception Surveys” aimed at capturing relevant information from citizens regarding their perception of government policies and programs with the ultimate objective of ensuring their alignment.

PDO 2 – Quality of public information (i.e., Project’s Outcomes) High Achievement:

Public investments • In addition to the stringent logic-framework methodology that is in place for the

identification, formulation and approval of investment projects defined jointly by the DNP and the Governing Committee of the National Royalty System (Comisión Rectora del Sistema General de Regalías),12 the DNP collects information following strict protocols for data quality, reliability and timeliness on all public investments with financial resources from the national budget under the SUIFP.

SINERGIA Monitoring • OECD describes SINERGIA as an elaborate and well-designed performance

evaluation tool of exceptional quality that has been developed to exacting standards (OECD, 2013, p. 29).

11 Since April 2006, the Accelerated Data Program has been implemented as a satellite program of the PARIS21 Secretariat at OECD with World Bank support. It aims at strengthening country capacity in producing statistical data relevant for policy design, monitoring and evaluation by: i) better documenting, preserving, and disseminating existing micro-data; ii) better exploiting existing datasets (quality assessments, further analysis); and iii) better strategizing and aligning survey programs and statistical outputs to priority data needs. 12 Metodología General Ajustada, Art. 23, Law 1.530 de 2012.

16

KPI 3 - Presidencia, Hacienda and DNP report satisfaction with the quality information provided by SINERGIA Monitoring and Evaluations. KPI 4 - Leading NGO’s and civil society members report higher satisfaction levels in terms of access and quality of information on goals monitoring. • Although data is not available for these operational definitions, these two indicators are

considered to have been achieved given the substantial increase in respondents’ satisfaction with SINERGIA’s quality of information under the Impact Evaluation. Specifically:

• Respondents in the Impact Evaluation reported a significant improvement in the quality of the SINERGIA Monitoring’s information, with the composite qualification (i.e., timeliness, clarity, usefulness, credibility, and relevance) increasing from 3.3 to 3.8 between 2010 and 2013.13 All five dimensions showed an improvement during the same period, with timeliness increasing from 3.2 to 3.7, clarity from 3.2 to 3.7, usefulness from 3.5 to 3.9, credibility from 3.3 to 3.9 and relevance from 3.4 to 3.8 (see Figure 3).

Figure 3. Perceived quality of information from SINERGIA Monitoring

(Average composite scores)

Source: DNP (2014)

• Likewise, there was a significant increase in the percentage of respondents

characterizing their assessment of information under SINERGIA Monitoring as “Very Good” or “Excellent.” Specifically, this percentage increased from 36 to 63 percent for timeliness; from 35 to 56 percent for clarity; from 49 to 74 percent for usefulness; from 38 to 74 percent for credibility; and from 44 to 69 percent for relevance between 2010 and 2013 (see Figure 4). When including those respondents that responded “Good,” the percentage of satisfaction increases to 84 percent or higher for the various dimensions of quality.

13 The score range was between one (1.0) to five (5.0), where 1.0 was poor quality and 5.0 was excellent.

17

Figure 4. Percentage of responses characterizing information under SINERGIA

Monitoring as “Very Good” or “Excellent”

Source: DNP (2014)

• Respondents also rated favorably the quality of the documents produced under SINERGIA Monitoring, including the Annual Report to Congress and the President, the Balance of Results, and methodological documents, with ratings consistently above 4.0 in a scale from 1 (low quality) to 5 (high quality) (see Table 5).

Table 5. Quality assessment of documents produced under SINERGIA Monitoring

Source: DNP (2014)

SINERGIA Evaluation • Respondents reported a significant improvement in the quality of the SINERGIA’s

information, with the average composite score (i.e., timeliness, clarity, usefulness, credibility, and relevance) increasing from 3.5 to 4.1 between 2010 and 2013.14 All five dimensions showed an improvement during the same period, with timeliness increasing from 3.2 to 4.3, clarity from 3.6 to 4.0, usefulness from 3.6 to 4.2, credibility from 3.6 to 4.1, and relevance from 3.5 to 4.1 (see Figure 5).

Figure 5. Perceived quality of information from SINERGIA Evaluation

(Average composite scores)

Source: DNP (2014)

18

• Likewise, there was a significant increase in the percentage of respondents

characterizing their assessment as “Very Good” or “Excellent.” Specifically, this percentage increased from 34 to 84 percent for timeliness; from 52 to 89 percent for clarity; from 54 to 89 percent for usefulness; from 46 to 91 percent for credibility; and from 46 to 91 percent for relevance between 2010 and 2013 (see Figure 6). When including those respondents that responded “Good,” the percentage of satisfaction increases to 94 percent or higher for the various dimensions of quality.

Figure 6. Percentage of responses characterizing information under SINERGIA

Evaluation as “Very Good” or “Excellent”

Source: DNP (2014)

PDO 3 – Relevance of public information (i.e., Project’s Outcomes)

High Achievement: As noted by OECD, “evidence-based policy making is clearly informing important policy decision taken by the GoC, notably in relation to the implementation of its National Development Plan” (OECD, 2013, p. 29). The report then goes on to describe how SINERGIA is systematically used by the GoC to evaluate and monitor the performance of key Plan-related public policies and programs, based on the information provided by the system in terms of whether policy objectives are being reached.

• Knowledge of SINERGIA Monitoring among respondents increased from 84 to 97 percent between 2010 and 2013, indicating that this M&E tool is a routine mechanism of consultation by government agencies as well as citizens.

• Knowledge of SINERGIA Evaluations among respondents increased from 65 to 69 percent between 2010 and 2013, signaling that there is still some room for improvement regarding its dissemination and insertion within Colombia’s public sector.

3.a Program and policy design by CONPES and government entities KPI 7 - Percentage of increase in visits of high-government officials to control panels (342 percent level of achievement). • The usage of the SINERGIA Monitoring control panels has exceeded expectations,

with the number of visits from top government officials increasing from 1,706 to 12,144 between January 2012 and December 2014 (equivalent to a 712 percent increase compared to a 6 percent target).

14 The score range was between one (1.0) to five (5.0), where 1.0 was poor quality and 5.0 was excellent.

19

• The number of visits from other government users increased from 11,921 to 68,543 between January 2012 and December 2014 (equivalent to a 575 percent increase).

• Under the Impact Evaluation, more than half of SINERGIA users (51 percent) report utilizing information from SINERGIA Evaluation to enhance program and policy design, including adjusting program targets (27 percent), policy redesign (18 percent), and reallocation of financial resources (6 percent).

3.b Social accountability by the Office of the President • Highly visible reports based on information from SINERGIA Monitoring are

published periodically reporting progress toward the achievement of NDP targets, including the Annual Report to the Congress and Balances of Results.

• Data from SINERGIA Monitoring is publicly available online, where it is being increasingly accessed by non-government users.

KPI 6 - Percentage of increase in visits from citizens to SINERGIA. • The usage of the SINERGIA by non-government users has exceeded expectations,

with the number of visits per month increasing from 44,932 to 325,276 in between January 2012 and December 2014 (equivalent to 724 percent increase compared to a 13 percent target).

• Roughly 23 percent of SINERGIA Evaluation’s users report utilizing the information for dissemination purposes.

• In 2014, the President visited each line Ministry to review progress toward the 2010-2014 NDP using SINERGIA Monitoring’s sector dashboards. Dashboards are also routinely utilized during ministerial meetings.

• Evaluation results from SINERIGIA Evaluation are reported in the Report of the President to Congress.

3.c Investment planning by DNP KPI 5 – 40 percent of recommendations of evaluations completed during project lifetime have been implemented by corresponding authorities at the end of the project. • A baseline was being constructed in 2011 but measurement was discontinued as the

indicator was dropped. A review of four programs that were evaluated under SINERGIA Evaluation indicated that the implementing government agency agreed to implement on average 59.5 percent of the recommendations.15

• Results from program evaluations conducted under SINERGIA Evaluation have resulted in changes being introduced in several programs.16

• More stringent mechanisms for monitoring the implementation of recommendations have since been adopted, including the Recommendation Management Matrix (Matríz de Gestión de Recomendaciones - MGR). This matrix defines the recommendations to be implemented by the agency responsible for the program, including responsible parties, action plan and timetable.

3.d Monitoring and evaluation capabilities at the local and regional levels • 94 percent of municipal officials reported a positive change as a result of the technical

assistance received under SINERGIA Territorial.

15 Política Nacional Ambiental para el desarrollo sostenible de los Espacios Oceánicos (45 percent); Proyecto de Fortalecimiento de los Servicios a la Justicia (52 percent); Programa Nacional de Vivienda de Interés Social Rural (61 percent); and Programa de Desarrollo Rural con Enfoque Territorial (80 percent). 16 These programs include: Familias en Acción, Vivienda de Interés Social Urbana, Condiciones de Habitabilidad, Política de Reintegración, Agro Ingreso Seguro, Programa de Renovación de la Administración Pública, Asistencia Técnica del Sector Agrícola, BATUTA and the Programa Integral de Rehabilitación y Mantenimiento.

20

• Municipal capacity has improved among participating territorial entities, as measured by the performance index built by DNP for subnational governments (DNP, 2014b). Between 2012 and 2013, 50 of the 85 municipal governments that participated in SINERGIA Territorial (58.8 percent) improved their Efficiency Score, while 59 of them (69.4 percent) improved their Overall Municipal Performance score.

3.3 Efficiency Rating: Substantial The Project’s economic efficiency is deemed Substantial. As shown on Annex 1, the overall cost of the operation was as anticipated. Although there was a reallocation of costs among to components to reflect changes introduced under the 2012 restructuring, as reported by the Borrower, prices of goods and services procured under the Project were in line with those originally anticipated (see Annex 3). No formal economic analysis was conducted during Appraisal since this type of Project is not amenable to traditional cost-benefit analysis. Alternatively, the PAD indicated that the economic benefit from this operation was to be derived from the provision of higher quality and more timely information, which, in turn, would allow the GoC to take better decisions and an improved allocation of public resources. The efficient allocation of public resources was particularly relevant in the case of Colombia given the increase in government participation in the economy exhibited during the previous 15 years, as well as the ongoing process of decentralization, which was accompanied with increasing expenditure responsibilities at the subnational level, thus meriting the strengthening of the M&E at both the national and sub-national levels as supported under the Project. While it is still not possible to conduct a formal economic analysis at closing given the lack of data in terms of increased efficiency due to the enhanced M&E framework developed with the Project’s support, experience worldwide indicates that whole-of-government evidence-based decision-making leads to greater effectiveness and efficiencies in achieving strategic outcomes (OECD, 2013). In the case of this operation, the economic benefits of the operation can be expected to be substantial given the particularly large mass of expenditures covered under SUIFT and SINERGIA. As an illustration, Table 3.1 shows the economic benefits that can be expected from enhanced M&E under the SUIFT and SINERGIA Monitoring, SINERGIA Evaluation and SINERGIA Territorial, as well as the corresponding mass of investments that can expected to be positively affected each year. This analysis indicates that the amount of public investments being monitored and/or evaluated under these M&E systems amount to roughly US$128.7 billion per year. Thus, it would require only a marginal improvement in the economic return from these investments (a mere 0.0012 percent) as a result of enhanced M&E to exceed the cost of the entire operation (US$15.8 million) in one year alone (see Annex 3).

3.4 Justification of Overall Outcome Rating Rating: Satisfactory The operation is rated Satisfactory during both the pre- and post-restructuring periods, resulting in an overall Satisfactory rating. Specifically: Pre-restructuring period: Overall rating is considered Satisfactory given the High relevance of objectives, Substantial relevance of design, High efficiency and Substantial efficacy in the achievement of development objectives.

21

Post-restructuring period: Overall rating is considered Satisfactory given the High relevance of objectives, High relevance of design, High efficiency, and Substantial efficacy in the achievement of development objectives.

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development The operation has supported the joint efforts of DNP and DANE to ensure the comparability of poverty measures over time and to complement traditional measures of monetary poverty with a more encompassing measurement of poverty across multiple dimensions. It also helped enhanced the governance of poverty measure by supporting the newly established expert poverty committees, which are now responsible for validating monetary and multi-dimensional poverty data generated by DANE. In terms of social development, the Project has set the foundations for enhancing social accountability and generating greater credibility with civil society. In response to the initiatives of the UN Office of Evaluation, gender was explicitly incorporated into program evaluations conducted under SINERGIA Evaluation. The gender perspective has already been included in four program evaluations and now part of the methodological requirements to be followed for future evaluations.17 It is also noteworthy that, in the context of the 2014-2018 NDP, SINERGIA Monitoring will include a special component focusing on Indigenous Peoples, including the design of culturally appropriate indicators and a control dashboard. (b) Institutional Change/Strengthening By promoting a greater use of M&E results for budget, planning, policy development, implementation, and social accountability functions, and by strengthening the information base of the M&E system, the Project is expected to have a considerable long-term, government-wide impact on policy and program design and implementation. Thus, its contribution to Colombia’s institutional development is deemed high. (c) Other Unintended Outcomes and Impacts (positive or negative) • The Project has contributed to Colombia playing an active role in sharing its experience in

development of SINERGIA in international forums and, thus, contributing to South-South capacity building. The OECD and the International Initiative for Impact Evaluation (IIIE) have amply disseminated SINERGIA as a successful experience. As a new member organization of the IIIE, the DNP is also playing a more active role in sharing its experience in international forums.

• SINERGIA has also contributed to enhancing transparency and accountability by facilitating internal auditing procedures.

17 Land Titling for Peasant Families; Food and Nutritional Security, the Presidential Program for Comprehensive Action against Anti-personnel Mines; and Promotion of Biofuels Policy.

22

• As reported by technical staff from other line ministries during the ICR mission, SINERGIA Evaluation and its participatory approach toward evaluation has contributed to promoting an evaluation culture within those government agencies responsible for the implementation of programs evaluated. Likewise, more than one third of users of SINERGIA Evaluation report utilizing the evaluations and methodological documents to learn about evaluation methodologies, including external users (DNP, 2014).

• With support from the Bank, a strategy was developed and implemented to foster the supply of private sector firms with the technical expertise needed to conduct program evaluations.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Results from the surveys with SINERGIA users under the 2013 Impact Evaluation are reported in Section 3.2 – Achievement of Objectives.

4. Assessment of Risk to Development Outcome Rating: Low The outcomes achieved under this operation are expected to be maintained over time, with several factors underlying their sustainability. These include: • High government commitment to the continuous development and institutionalization of

SINERGIA as demonstrated by the strong support received and substantial progress made under this Project;

• The strong ownership by key M&E stakeholders, including DNP, MHCP, the Office of the President in general demonstrated under this operation;

• The allocation of resources to DNP under the Medium-Term Expenditure Framework to specifically continue with the consolidation of SINERGIA Monitoring and SINERGIA Evaluation;

• The positive assessment on the part of government and non-government users of the quality and usefulness of the information provided under SINERGIA;

• The actions supported under the Project areas are already fully embedded within DNP’s organizational and institutional structure, as its technical units were directly responsible for Project implementation; and

• The strong legislative and regulatory support for SINERGIA, including Decree 1,290/2014 that formally established SINERGIA’s organizing principles, composition, organization, and M&E responsibilities of the various government entities, as well as Decree 2844/2010 that requires the use of SUIFP by all government entities.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory During preparation, the Bank effectively capitalized on the ongoing momentum and commitment on the part of top government authorities and key constituencies toward further developing a whole-of-government M&E system, with a particular emphasis on program evaluation. Project preparation also benefited from sound technical evaluations that provided a sound diagnosis of the system’s weaknesses and challenges, including an evaluation of SINERGIA on the part of IEG. It was also critical in building capacity as well as consensus among the key institutional stakeholders. Although the original project design was in retrospect overly ambitious, it reflected a systemic, whole-of-government view that encompassed the enhancement of M&E processes under the

23

various agencies as well as the utilization of M&E outputs by critical stakeholders, such as top government officials and the public. Although the PRF included important aspects, such as the impact evaluations under the operation’s original design, it also exhibited some weaknesses, including the lack of full a alignment between the KPIs and some of the dimensions embedded in the PDO, the lack of clear operational definition for some indicators and the excessive number of intermediate indicators. Also, in retrospect, the original design appears to have been overly ambitious, not just in terms of scope but also the number of institutions involved, including several departments within DNP, DANE, and municipal and departamental entities. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Satisfactory The operation was regularly supervised with well-documented missions that included the support of sector specialists being conducted semi-annually. Throughout the Project’s lifetime the Bank provided relevant technical expertise in the areas of program evaluation, poverty measurement, public sector management tools, and IT, which contributed to the project achieving its objectives. Supervision efforts also included close monitoring of fiduciary compliance, which, together with the training in FM and procurement that was provided to DNP administrative staff, resulted in substantial capacity building in these areas. While the need to restructure the operation was identified early on during implementation, the processing of the restructuring took too long. Changes in TTLs may have been one of the contributing factors to this delay. However, it is important to note that delays in the restructuring and changes in TTL did not affect the achievement of Project outcomes. Although the Borrower also acknowledged shortfalls in supervision, primarily the delay in processing the 2012 restructuring, the assessment of the Bank’s overall performance is Satisfactory (see Annex 7). In addition, it was explicitly acknowledged during the ICR mission the Bank’s contribution in terms of the continuous technical assistance provided throughout the Project’s lifetime. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory Overall Bank performance is deemed Moderately Satisfactory to reflect the moderately satisfactory performance at preparation and the satisfactory performance in supervision.

5.2 Borrower Performance (a) Government Performance Rating: Satisfactory The Project received full support from the GoC, including the Office of the President, which gave high visibility to SINERGIA. Government support was also demonstrated with the passage of supportive regulations (among others, Decree 1290/2014) and the allocation of budgetary resources. In this regard, it is important to note that, due to delays experienced in the restructuring, the GoC contributed additional resources to avoid interrupting activities being supported under the Project.

Wide support from top government officials for the Project in particular and the strong commitment toward the development of a whole-of-government M&E system in general was also demonstrated by Colombia’s high level of participation in the 2009 Conference of the Latin American and

24

Caribbean (LAC) Monitoring and Evaluation Network,18 which was hosted by the DNP in Bogotá. About 150 Colombian participants, including both professionals and high government officials from DNP, the Colombian Ministry of Finance, and the Office of the Presidency attended the conference. The Colombian government also took this opportunity to launch a national chapter of the M&E Network. The GoC continued to have a strong presence at subsequent annual conferences of the LAC Monitoring and Evaluation Network.

(b) Implementing Agency or Agencies Performance Rating: Satisfactory The DNP and its directorates exhibited a remarkable degree of ownership of both SINERGIA and this Project, The DNP effectively coordinated implementation efforts among its various directorates and, during the pre-restructuring period, with DANE as well. Although minor weaknesses in procurement persisted throughout the Project’s lifetime, DNP administrative units effectively overcame their lack of previous experience with Bank FM and procurement procedures and provided adequate support to Project implementation. Although swifter action in the context of the restructuring would have been desirable, DNP’s overall performance was otherwise outstanding. As mentioned earlier, it did not affect the achievement of Project outcomes, as the DNP allocated its own resources to avoid disruptions in implementation. The DSEPP was the critical engine in Project implementation as a result of its robust organization and the strong technical capacity and high motivation of its professional staff. The DSEPP also had a clear strategic vision of SINERGIA, which was critical in steering the course during implementation and ensuring the achievement of the Project’s PDOs. While reporting on the PRF was weak during the pre-restructuring period, it improved substantially during the second half of the operation’s lifetime. The DSEPP submitted its ICR early in the evaluation process and provided ample support during ICR preparation. DNP’s Directorate of Public Finance and Investments (DIFP) exhibited similar qualities in implementing the activities under Component 1, including a strong sense of ownership, clear vision, and strong technical capacity. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory Overall Borrower performance is deemed Satisfactory to reflect the satisfactory performance on the part of both the Government and the DNP.

6. Lessons Learned • Technical assistance during Project preparation is not only critical to ensure sound Project

design but it can also result in other benefits, such as capacity and consensus building among key stakeholders.

18 The Latin American and Caribbean (LAC) Monitoring and Evaluation Network was established in 2005 by the World Bank and the Inter-American Development Bank (IDB) to promote and strengthen the use of M&E systems in Latin America and the Caribbean. The Fifth Annual Conference of the Network took place in Bogota, Colombia, from November 5–6, 2009. The Conference gathered more than 350 practitioners, government officials, academia, and international organizations (IDB, World Bank, the Food and Agriculture Organization, Comisión Económica de América Latina, RELAC) from more than 30 countries around the world, and an additional 300 people watched the event by Webcast.

25

• While conceiving a whole-of-government M&E system it is critical to include both technical and political economy considerations to ensure that such system is both technically sound and that it’s effectively utilized by its intended users. In the case of this Project, the design of dashboards tailored to the needs of the Office of the Presidency and line ministers, illustrates the importance of addressing the needs of strategic users.

• The institutional and economic impact of an operation is not necessarily correlated to its size. While this operation was very small in the context of the Bank’s portfolio in Colombia, it has had a significant impact in Colombia and beyond, as SINERGIA is considered by OECD standards as a cutting-edge M&E system with characteristics to be emulated elsewhere.

• It is important to note that if a Project of similar characteristics were to be prepared today, Reimbursable Advisory Services (RAS) would be a more adequate instrument than an investment loan.

• Political support and ownership on the part of the Borrower are key ingredients for project success. This finding does not, of course, constitute a novelty. However, its importance was once again underscored in the context of this Project, in which support from top government authorities, key government agencies, as well as the DNP were critical engines in furthering the development and consolidation of SINERGIA.

• Borrowers, particularly sophisticated ones such as Colombia, can be the natural drivers of sector-wide initiatives. While the Bank played a key role in helping Colombia build awareness and consensus over the importance of strengthening its M&E system, the DNP played a central role in steering implementation of this Project. This, in turn, brings attention to the range of roles to be played by the Bank in sector-wide interventions in which IBRD funding is relatively small and the Borrower has strong technical and institutional capacity and strategic vision.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies See Annex 7. (b) Co-financiers Not Applicable (c) Other partners and stakeholders Not Applicable

26

Annex 1. Project Costs and Financing

(a) Project Cost under IBRD Financing by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

1. Institutional strengthening for M&E of public investment projects at the national and sub national level.

0.8 0.6 75.0

2. Consolidate and expand the use of the national M&E system. 4.3 4.8 110.0

3. Support the development of local and regional M&E systems. 0.3 1.0 330.0

4. Establish the mechanisms to improve quality and relevance of public information.

2.2 1.2 50.0

5. Project Administration 0.9 0.9 100.0 Total Baseline Cost 8.5 8.5 100.0 Total Financing Required 8.5 8.5 100.0

(b) Financing

Source of Funds Type of Co-financing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower

Earmarked allocation of

budgetary resources (BPIN)

7.3 7.3 100.0

International Bank for Reconstruction and Development Loan 8.5 8.5 100.0

27

Annex 2. Outputs by Component Component 1: Institutional strengthening for M&E of public investment projects at the national and sub-national level • This component supported the establishment of standards for collection and classification of

public investment spending by region. As a result, regionalized budget information budget is now available for 94 percent of the investment budget that is subject to being regionalized in 2014.

• Additionally, this component supported the design of methodologies, indicators and guidelines for the monitoring and evaluation of investment projects at the national and regional level under the Unified System for Investment and Public Finance (Sistema Unificado de Inversiones y Finanzas Públicas - SUIFP), including the development and implementation of modules for: i) formulation; ii) programming; iii) project execution; and iv) monitoring of public investments financed under the national budget.

• It also supported the delivery of training and dissemination on the use of the SUIFP and the Project Monitoring System (SIP), which tracks progress toward project implementation.

Table 2.1 Evolution in the number of projects under SUIFP

Year SUIFP – National Budget SUIPT - General Royalty System (SGR)

SPI (Projects with resource

allocation) 2008 n.a. n.a. 1634 2009 n.a. n.a. 1628 2010 1226 n.a. 1564 2011 1193 n.a. 1550 2012 1268 1571 1554 2013 1331 5051 1589 2014 1277 2769 1456

Source: DNP (2015).

Component 2: Consolidate and expand the use of the national M&E system Under this component the operation supported the continuous development of SINERGIA Evaluation, a system to evaluate the outcomes of the main public policies and programs implemented within the framework of the NDP. Every year, the policies that will be evaluated are elected by a Committee of the DNP and approved by the CONPES. Additional resources were made available to Component 2 in the 2012 restructuring to support an expanded evaluation agenda that fed into the current National Development Plan. Specific outcomes under SINERGIA Evaluation include:

• Several improvements were introduced, including the standardization of evaluation methodologies, the creation of an annual evaluation agenda, the development of improvement plans and monitoring to ensure that recommendations were being properly implemented. The experience built under the operation resulted in the creation of the National Assessment System (SISDEVAL).

• A total of 84 program evaluations were conducted during the operation’s lifetime, 23 of which were financed under the operation. In addition to the increase in the number of evaluations, there was significant progress toward sector diversification.

• Evaluations are conducted by recognized, experienced third parties so as to guarantee objectivity and transparency in the process.

• Evaluation results are increasingly being used as inputs in the design of the NDP 2010- 2014 and reported in the Report of the President to Congress.

28

• Results from program evaluations conducted under SINERGIA Evaluation have resulted in changes being introduced in several programs.19

• The new “Radar” initiative developed under SINERGIA Evaluation serves as an evaluations clearinghouse, systematically compiling evaluations performed by both public and no-public entities. Currently there are 171 evaluations available for downloading at the SINERGIA Evaluation website and an additional 530 are being reviewed.20

• Dissemination of results of evaluations were actively disseminated to enhance public awareness, with results reports results being published and a series of events on evaluation being performed.

Component 3: Support the development of local and regional M&E systems During the pre-restructuring period, this component provided support to municipalities and departments that were reform-oriented, with the aim of demonstrating the benefits of sound M&E tools at the subnational level. Some of the specific outcomes included: • Medium-term expenditure framework was being presented to the municipal council yearly in

the context of budget discussions in Medellin; • By December 2011, two departments (i.e., Atlántico and Santander) and four municipalities

had designed and implemented an evaluation agenda. • The PBB annex was presented to the municipal council yearly with the budget in Medellin and

Pasto. During the post-restructuring period, this component supported the development and implementation of a regional strategy aimed at enhancing the M&E function at the sub-national level under the so-called SINERGIA Territorial. Technical assistance was provided to 95 territorial entities (i.e., municipalities and governaciones) in four areas: i) M&E methodologies; ii) information systems; iii) training; and iv) follow-up and verification of the monitoring of regional and local development plans. These activities helped the development and implementation of M&E system within territorial entities, including production of quality performance information by subnational entities to assess the performance of their local development plans. This capacity building in the M&E function at the subnational level is critical in the context of the ongoing decentralization as well as the efforts to strengthen monitoring of the NDP at the subnational level. Some of the specific outcomes include: • Between 2012 and 2013, 59 municipalities improved in the Integrated Performance Score

(IDM). • Between 2012 and 2013, 50 municipalities improved their relative positions in the efficiency

index • 63 of the 95 territorial entities now have performance dashboards • 63 of the 95 territorial entities generate regular monitoring reports • 78 of the 95 territorial entities created and/or strengthened their monitoring teams • 53 of the 95 territorial entities formalized its M&E system through decrees, resolutions or

ordinances • There was substantial capacity building achieved, as shown by the progress towards higher-

levels of capacity of territorial entities, with 15 achieving the sustainability level at closing (see on Table 2.2).

19 These programs include: Familias en Acción, Vivienda de Interés Social Urbana, Condiciones de Habitabilidad, Política de Reintegración, Agro Ingreso Seguro, Programa de Renovación de la Administración Pública, Asistencia Técnica del Sector Agrícola, BATUTA and the Programa Integral de Rehabilitación y Mantenimiento. 20 https://sinergia.dnp.gov.co/SISDEVAL/paginas/Evaluaciones/RadarEvaluaciones.aspx

29

Table 2.2 Level of M&E capacity of participating territorial entities

Month 2014 Readiness Capacity building Consolidation Sustainability Total

June 46 31 17 1 95 48.4% 32.6% 17.9% 1.1% 100.0% August 46 29 18 2 95 48.4% 30.5% 18.9% 2.1% 100.0% November 34 26 20 15 95

35.8% 27.4% 21.1% 15.8% 100.0%

Component 4: Establish the mechanisms to improve quality and relevance of public information This component supported the modernization, integration and improvement of the quality of public sector information flow (generation, distribution and storage) and strengthening the institutional, legal, regulatory framework and technical areas. Component 4 also supported improving the technical capacity of SINERGIA to monitor the quality of performance information provided by government agencies. Finally, Component 4 supported the improvement of the DNP web platform and interface to make it more user-friendly with greater ease and access to the use of the data. Specific outcomes during the pre-restructuring period include: • Formulation of the DANE’s National Statistical Plan (PENDES), including the reclassification

of 1,227 statistical procedures to improve information flows and therefore, the quality and availability of information. These activities were continued by DANE outside the operation after the 2012 restructuring.

• The Project contributed to restoring long-term comparability in the measurement of poverty after a disruption during 2006-2007 as a result of changes in data collection instruments. In addition, DANE has assumed responsibility for calculating and publishing the results of the official measurements of poverty in Colombia, including traditional measures of monetary poverty as well as multi-dimensional poverty. Multi-dimensional poverty measurements have been available since 2011.

Specific outcomes during the pre- and post-restructuring period include the continuous strengthening of SINERGIA Monitoring, which consists a set of performance indicators that measures policy outputs and outcomes identified by the NDP. The system is built following a pyramidal structure with three main levels: strategic, sector and management. Strategic indicators are at the top and are related to the main government pillars as stated in the NDP. The President and the Council of Ministers monitor progress toward strategic indicators. Sector indicators describe sector-specific goals and are monitored by the President and each Minister in bilateral meetings and within each Ministry. Finally, Management indicators are standard indicators that are measured for all of the entities to track institutional efficiency.

Some of the specific outcomes include: • Control dashboards were designed and implemented to provide government officials with early

warnings on delays progress and the achievement of NDP targets. These dashboards are designed to communicate the information effectively, utilizing different colors to show various degrees of progress. There are three types of boards: i) sector dashboards for DSEPP internal use; ii)) bi-lateral ones to support dialogue between the President and line ministers; and iii) cross-sector ones to track progress made in different sectors under a transversal policy area.

• In 2013 the DSEPP began the documentation of databases according to international standards and best practices under the Accelerated Data Program (ADP), including ensuring the

30

anonymity of micro-data, to encourage open dissemination of information. 21 By 2014, there were 29 documented databases. In addition, the documentation of databases has been included as a requirement under all evaluations conducted under SINERGIA Evaluation.

• With Project support, the DNP is now routinely carrying out "Public Perception Surveys” aimed at capturing relevant information from citizens regarding how they are affected by government policies and programs with the ultimate objective of ensuring their alignment.

• M&E performance methods and tools have been adopted in several national departments and has cultivated participation and interest in the Colombia chapter of the Latin American and Caribbean Monitoring and Evaluation Network.

21 Since April 2006, the Accelerated Data Program has been implemented as a satellite program of the PARIS21 Secretariat at OECD with World Bank support. It aims at strengthening country capacity in producing statistical data relevant for policy design, monitoring and evaluation by: i) better documenting, preserving, and disseminating existing micro-data; ii) better exploiting existing datasets (quality assessments, further analysis); and iii) better strategizing and aligning survey programs and statistical outputs to priority data needs.

31

Annex 3. Economic and Financial Analysis The Project’s economic efficiency is deemed High. As shown on Annex 1, the overall cost of the operation was as anticipated. As reported by the Borrower, prices of goods and services procured under the Project were in line with those originally anticipated. There was a reallocation of costs among to components to reflect changes introduced under the 2012 restructuring. Specifically: • Although all activities under Component 1 were completed, the lower actual cost of Component

1 relative to the one anticipated at Appraisal (i.e., 0.75 percent) reflects DNP’s decision to finance some of the activities with its own budgetary resources.

• The actual cost of activities under Component 2 was slightly higher than anticipated (110 percent) due to a larger evaluation program that envisioned at Appraisal

• The actual cost of activities under Component 3 was three times higher than anticipated at Appraisal (330 percent) due to the GoC’s decision to implement a more extensive intervention at the subnational level (i.e., SINERGIA Territorial) than anticipated at Appraisal.

• The actual cost of activities under Component 4 was half than anticipated at Appraisal due to the elimination of activities under DANE under the 2012 restructuring.

No formal economic analysis was conducted during Appraisal since this type of Project is not amenable to traditional cost-benefit analysis. Alternatively, the PAD indicated that the economic benefit from this operation was to be derived from the provision of higher quality and more timely information, which, in turn, would allow the GoC to take better decisions and an improved allocation of public resources. The efficient allocation of public resources was particularly relevant in the case of Colombia given the increase in government participation in the economy exhibited during the previous 15 years, as well as the ongoing process of decentralization, which was accompanied with increasing expenditure responsibilities at the subnational level, thus meriting the strengthening of the M&E at both the national and sub-national levels as supported under the Project. While it is still not possible to conduct a formal economic analysis at closing given the lack of data in terms of increased efficiency due to the enhanced M&E framework developed with the Project’s support, experience worldwide indicates that whole-of-government evidence-based decision-making leads to greater effectiveness and efficiencies in achieving strategic outcomes (OECD, 2013). In the case of this operation, the economic benefits of the operation can be expected to be substantial given the particularly large mass of expenditures covered under SUIFT and SINERGIA. As an illustration, Table 3.1 shows the economic benefits that can be expected from enhanced M&E under the SUIFT and SINERGIA Monitoring, SINERGIA Evaluation and SINERGIA Territorial, as well as the corresponding mass of investments that can expected to be positively affected each year. This analysis indicates that the amount of public investments being monitored and/or evaluated under these M&E systems amount to roughly US$128.7 billion per year. Thus, it would required only a marginal improvement in the economic return from these investments (a mere 0.0012 percent) as a result of enhanced M&E to exceed the cost of the entire operation (US$15.8 million) in one year alone.

32

Table 3.2 Estimated annual economic benefits

Source of Economic Benefit Annual Amount

US$ (2014)

Monitoring of Public investments (SUIFT) • Enhanced design of investment projects • More effective monitoring of project

implementation, including savings from averted delays

US$16.2 billion contracted investments under SPI in 2014

Monitoring of DNP 2010-2014 (SINERGIA Monitoring) • Enhanced policy design • Enhanced budgetary allocation

US$106 billion of overall budget for the central government

Program evaluation (SINERGIA Evaluation) • Enhanced program design and redesign • Enhanced budgetary allocation

USD165.3 million in the annual resources allocated to programs evaluated in 2014

Monitoring of Municipal Development Plans (SINERGIA Territorial) • Enhanced policy design at the municipal

level • Enhanced budgetary allocation at the

municipal levels

US$6.3 billion in municipal budgets of participating municipalities

33

Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members

Names Title Unit Responsibility/

Specialty Lending Juan Carlos Alvarez Senior Counsel LEGES Jairo A. Arboleda Consultant GSURR Wendy Cunningham Program Leader LCC1C Jeannette Estupinan Sr Financial Management Specia GGODR Fatima Galarraga Elias Program Assistant LCCCO Claudio Rodrigo Garcia Verdu Economist LCSPP -

HIS

Christian Yves Gonzalez Amador Senior Economist GMFDR

Gladys C. Lopez-Acevedo Lead Economist SARCE Keith Mackay Consultant GPVDR Jose M. Martinez Senior Procurement Specialist GGODR Fernando Rojas Consultant GMFDR

Marcela Rubio Sanchez E T Consultant LCSPP - HIS

Andrea Vermehren Lead Social Protection Special GSPDR

Supervision/ICR Claudia Mylenna Cardenas Garcia E T Consultant LCSFM -

HIS

Aline Coudouel Lead Social Protection Special GSPDR Henry Forero Ramirez Senior Public Sector Specialist GGODR Gladys C. Lopez-Acevedo Lead Economist SARCE Jose M. Martinez Senior Procurement Specialist GGODR Edgardo Mosqueira Medina Lead Public Sector Development GGODR Monica Penuela Jaramillo Consultant GGODR Amparo Ballivan Lead Economist DECDG Joao Pedro Wagner de Azevedo Lead Economist GPVDR

Carlos Rodriguez-Castelan Senior Economist GPVDR Jessica Terry Poverty Specialist GVPDR Indu John-Abraham Representative LCCEC Karina Brito Program Assistant GPVDR Luz Zeron Financial Management Specialist GGODR Sandra Ximena Enciso Gaitan ET Consultant GGODR

34

(b) Staff Time and Cost

Stage of Project Cycle

Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands

(including travel and consultant costs)

Lending FY06 28 136 FY07 52 302 FY08 16 81

Total: 96 519 Supervision/ICR FY09 14 59 FY10 18 162 FY11 11 106 FY12 6 88 FY13 21 71 FY14 9 43 FY15 14 102

Total: 93 631

35

Annex 5. Beneficiary Survey Results Results from the surveys with SINERGIA users under the 2013 Impact Evaluation are reported in Section 3.2 – Achievement of Objectives.

36

Annex 6. Stakeholder Workshop Report and Results Not Applicable.

37

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Introducción El Contrato de Préstamo BIRF 7620co se ejecutó durante seis (6) años, fue suscrito por las partes el 19 de diciembre de 2008 y finalizó ejecución el 31 de diciembre de 2014; concebido con el propósito de generar el fortalecimiento de los sistemas de Monitoreo y Evaluación (M&E) del Gobierno de tal forma que provea información de calidad para ser usada por los programas y los diseñadores de política, para la rendición de cuentas del Gobierno Nacional hacia la Ciudadanía en general. De acuerdo a lo anterior es necesario presentar ante el Banco Mundial y demás agentes interesados este informe final, el cual se entrega como parte de la etapa de cierre del proyecto, y tiene como objetivo presentar su estructura, los cambios realizados durante la ejecución y componentes, los logros alcanzados en su implementación, el estado de las actividades a la fecha de cierre y la sostenibilidad del proyecto sin el Crédito del Banco Mundial. Complementando lo anterior y como parte del “Implementation Completion and Results Report – ICR”, el presente informe contiene la evaluación de desempeño del prestatario y del Banco, durante la ejecución del proyecto.

1. Antecedentes En 1994 fue creado formalmente el Sistema Nacional de Evaluación de Gestión y Resultados – SINERGIA, el cual durante 1996 y 2002 tuvo un decaimiento de su uso dentro del Gobierno, pues se encontraron dificultades en el manejo de sistema y el acceso a la información. Lo anterior tuvo como consecuencia que el Presidente de la República (2002 – 2006), promoviera el uso activo de la información generada por SINERGIA para aumentar la responsabilidad de la Rama Ejecutiva y apoyar la transparencia del Gobierno hacia los Ciudadanos y el Congreso. Los esfuerzos colectivos del Alto Gobierno y otros agentes durante los últimos años, ha logrado que SINERGIA se consolide como unos de los sistemas de M&E más fuertes técnica y conceptualmente de América Latina, pero que aún cuenta con algunas deficiencia identificadas por agentes externos al DNP y que se convierten en retos por abordar a futuro y que se relacionan a continuación. La declaración de la política de Gobierno expedida en 2004, en donde el Consejo Nacional de Políticas Económica y Social – CONPES, en su documento CONPES 3294 aportó un diagnóstico de la situación de M&E en Colombia, en donde los principales vacíos fueron: La falta de un manual metodológico único y claro sobre cómo llevar a cabo el M&E La necesidad de aclarar los roles y responsabilidades de los ministerios y agencias

involucradas en el sistema de M&E del Gobierno La ausencia de conexiones claras entre planeación, presupuesto y evaluación de políticas

públicas Problemas con la disponibilidad, frecuencia y calidad de la información.

38

Estas falencias y retos que enfrentaba el sistema de M&E en Colombia, también habían sido identificados por el Banco Mundial22, en los diagnósticos realizados al sistema en 2006, en donde se identificaron los retos que esperaba apoyar con el Crédito BIRF 7620co: Información limitada acerca de la inversión del gobierno central a nivel sub –nacional. No

existían datos sobre la ejecución del prepuesto por parte de los departamentos y municipios; al igual que la falencia o ausencia de metodologías para la formulación estandarizada de proyecto de inversión.

Fortalecer el marco institucional de SINERGIA para asegurar su consolidación y sostenibilidad. Con lo anterior se busca que la desde la Dirección de Seguimiento y Evaluación de Políticas Públicas – DSEPP, estandarice procedimientos y transfiera capacidades a los ministerios y entidades.

Teniendo en cuenta los pilotos realizados en Medellín, Tocancipa y Pasto, en donde se mostró interés de las administraciones públicas territoriales, para fortalecer los sistemas de M&E, desarrollar y transferir capacidades de M&E a los gobiernos locales.

La información que en su momento alimentaba SINERGIA a nivel nacional era de baja calidad debido a la falta de estándares unificados y protocolos para producir, auditar y diseminar la información a través de los ministerios y entidades del gobierno.

De acuerdo a lo anterior y también partiendo de la premisa de que a un sistema de M&E, no solo lo hace exitoso la calidad de la información que provee, sino también el uso que el gobierno y la ciudadanía le dé a la misma, siendo sostenible en el tiempo y sobreviviendo a los cambios de gobierno, en otras palabras que logre ser un sistema institucionalizado. De esta manera el Banco Mundial fundamentó su intervención, afirmando que con el fortaleciendo de SINERGIA se ayudaría al Gobierno de Colombia a mejorar su efectividad en la inversión pública del país, con el de cumplir con objetivos económicos y sociales más ambiciosos; partiendo del acompañamiento esxitoso que ha tenido el Banco Mundial en temas de M&E, promoción de transparencia de los gobiernos centrales, responsabilidad social mediante evaluaciones periódicas y una veeduría ciudadana más participativa.

2. Estructura del proyecto Objetivo Ayudar al Gobierno en el fortalecimiento del sistema de monitoreo y evaluación – M&E, que asegure la disponibilidad y generación de información de calidad pertinente para mejorar: 1. La formulación y diseño de la política por parte del CONPES y las entidades del gobierno, 2. La rendición de cuenta a la ciudadanía desde la Presidencia, 3. Planeación de las inversiones por parte del DNP, y 4. La capacidad de monitoreo y evaluación, en el nivel local y regional. Valor total de la operación: Usd$8.500.000 Periodo de ejecución: Enero de 2009 a diciembre de 2014

22 Grupo de Evaluación Independiente, Diagnóstico del Sistema Nacional de M&E de Colombia, Sinergia, ECD Serie de Documentos de Trabajo N°. 17; Febrero 2007, El Banco Mundial

39

Estructura por componentes Tal como fue concebido, el Proyecto se estructuró en cinco componentes: 1) El fortalecimiento institucional del Prestatario y evaluación de sus proyectos de inversión pública nacional y sub - nacional; 2) Consolidación y expansión del uso del sistema nacional de M&E del Prestatario; 3) Respaldo al desarrollo local de los sistemas de M&E locales y regionales; 4) Establecimiento de mecanismos para mejorar la calidad, relevancia y efectividad al costo de la información pública y 5) Ejecución del Proyecto. A su vez cada componente, cuenta con frentes específicos de intervención o “subcomponentes” los cuales fueron ejecutados por algunas Direcciones Técnicas del DNP y otras entidades nacionales, según el área de intervención; como se muestra en el siguiente cuadro, el cual incluye los cambios efectuados a los componentes según el contrato de empréstito modificado y reformulado.

Componente 1 Subcomponente

El fortalecimiento institucional del Prestatario

y evaluación de sus proyectos de inversión pública nacional y sub -

nacional

1.1) A) El fortalecimiento de las capacidades técnicas e institucionales del DNP para la evaluación y monitoreo de la inversión pública de acuerdo con la estrategia nacional de inversión pública del Prestatario. B) El diseño de metodologías, indicadores y directrices normales para la formulación de proyectos de inversión permitiendo la ejecución del M&E, en el ámbito nacional y regional. C) La realización de una revisión en profundidad de los proyectos de inversión seleccionados del Prestatario D) La realización de programas de capacitación en la formulación del proyecto de inversión para el personal seleccionado del Prestatario. 1.2) Fortalecimiento de la capacidad del Prestatario para el monitoreo y evaluación de los proyectos de inversión en el ámbito regional, por medio de la recolección sistemática de información sobre indicadores regionales y su inclusión en los sistemas existentes de información de inversión pública del Prestatario.


Consolidación y expansión del uso del sistema

nacional de M&E del Prestatario

2.1 Desarrollo de las capacidades institucionales, operativas y técnicas del DNP, CONPES, SINERGIA y demás entidades gubernamentales para la realización del M&E y las actividades de monitoreo de la pobreza y la puesta en marcha del programa de evaluación del Prestatario. 2.2 A) Promover el desarrollo y uso de la información del M&E requerida para las actividades de planeación y gestión de las entidades gubernamentales, por medio del establecimiento de, entre otros, normas de calidad de la evaluación y la realización de las capacidades de formación de capacidad para los usuarios públicos y privados y productores de información del M&E; y su tendencia dominante a los departamentos y municipalidades del Prestatario con el fin de evaluar la puesta en marcha del Plan Nacional de Desarrollo y documentar las prácticas de M&E tanto en el ámbito departamental como municipal. B) Estimulo al mejoramiento de la gobernabilidad y la rendición de cuentas mediante la difusión al público de los resultados del M&E.


40


Respaldo al desarrollo local de los sistemas de

M&E locales y regionales

3.1 A) Fortalecimiento de los sistemas piloto municipales de M&E para la planificación, presupuesto y rendición de cuentas en las municipalidades de Medellín y Pasto del Prestatario B) Desarrollo de una estrategia de gestión de resultados para el departamento del Choco del Prestatario C) Fortalecimiento de la calidad de las normas para la medición del desempeño y la entrega de información relacionadas con las transferencias fiscales, por medio de la realización de un programa de capacitación para el personal de uno de los departamentos del Prestatario a ser seleccionados por el DNP. D) Respaldo al desarrollo y puesta en marcha de los sistemas de M&E y su ampliación y tendencia dominante hacia los departamentos y municipalidades del Prestatario con el fin de evaluar la puesta en marcha del Plan Nacional de Desarrollo y fomentar las prácticas de M&E tanto en el ámbito departamental como municipal.


Establecimiento de mecanismos para mejorar

la calidad, relevancia y efectividad al costo de la

información pública

4.1 A) La provisión de respaldo a: i) la puesta en marcha del plan estratégico nacional del Prestatario para la estadística; y ii) la mejora de la metodología utilizada por el Prestatario para el diseño y difusión de encuestas de hogar dirigidas al monitoreo de la pobreza.

B) La ampliación de las capacidades técnicas de SINERGIA para vigilar la calidad de la información de desempeño entregada por las entidades gubernamentales.

C) La provisión de respaldo para el desarrollo de una política nacional sobre datos abiertos mediante la provisión de una evaluación del sistema de gestión de la información pública del Prestatario.


Ejecución del proyecto 5.1 El fortalecimiento de las capacidades de ejecución, monitoreo y evaluación del Proyecto del DNP a través de la provisión de asistencia técnica y de los costos operativos.

Indicadores

Componente Indicadores Unidad de medida

Línea de base23

Marco lógico

Porcentaje del incremento en las visitas de los ciudadanos a Sinergia

Porcentaje 69.57%

Porcentaje del incremento en las visitas de Alto Gobierno a los paneles de control

Porcentaje 3.29%

1 Porcentaje del presupuesto regionalizado

del total del presupuesto de inversión. Porcentaje 68.7%

2 Porcentaje de evaluaciones totales

aprobadas en la agenda que alcanzan la fase de diseño.

Porcentaje 60%

23 31 de diciembre de 2011

41

Componente Indicadores Unidad de

medida Línea de base23

3 Entidades Territoriales que firmaron el

memorando de entendimiento Número 56

4 Número de encuestas con PAD / Total

encuestas para evaluaciones Porcentaje 0

Modificaciones durante la ejecución En agosto de 2011, el DNP solicitó al Banco Mundial una enmienda que tenía como fin realizar una revisión al proyecto y la reprogramación de los recursos disponibles con el fin de dar respuesta a las nuevas prioridades del Gobierno Nacional. Si bien el crédito tenía objetivos concretos, se hizo necesario realizar reestructuración al proyecto con el fin de focalizar los recursos disponibles en áreas y temas que generaran mayor beneficio e impacto ajustándose a las necesidades del momento. Con el fin de responder a las nuevas necesidades del seguimiento y la evaluación de las políticas públicas en Colombia, y garantizar una mejor acción gubernamental en materia de diseño de política, rendición de cuentas, planeación de la inversión y generación de capacidades en las entidades territoriales, en noviembre de 2011, la DSEPP como Director Técnico del proyecto, solicitó al Ministerio de Hacienda y Crédito Público - MHCP tramitar ante el Banco Mundial una restructuración enfocándose en 5 aspectos fundamentales:

1. Modificar el PDO (Project Development Objective) para que incluya el CONPES (antes Comité Intersectorial de Evaluación - CIE)

2. Modificar algunas de las actividades del PAD (Project Appraisal Document) en los componentes del proyecto que lo requerían.

3. Modificar PDO y los indicadores de resultado intermedio en la matriz de marco lógico. 4. Revisar los costos del proyecto por componente para reflejar las circunstancias actuales,

después de 3 años de implementación. 5. Revisar las asignaciones de cada componente de acuerdo al préstamo y una re -

categorización

Cambios en la medición del objetivo general del programa: es levemente modificada en cuanto se cambia el comité responsable de las decisiones del sistema de M&E, en particular de la aprobación de la agenda de evaluación. En efecto, el Comité Intersectorial de Evaluación se reemplaza por el CONPES. De esta forma, todas las actividades asignadas en el programa al comité pasan al CONPES y el objetivo del programa queda así: “apoyar al Gobierno de Colombia a fortalecer su sistema de monitoreo y evaluación para asegurar la disponibilidad y la producción de información de calidad para mejorar: i) el diseño de programas y políticas por parte del CONPES y las entidades de gobierno24; ii) la rendición de cuentas por parte de la Presidencia de la República; iii) la planeación de la inversión por parte del Departamento Nacional de Planeación, y iv) las capacidades de monitoreo y evaluación en los niveles regionales y locales”.

24 Es de aclarar que a partir de 2011 con base en el artículo 229 de la Ley 1450, el DNP debe producir informes periódicos sobre los avances del Plan Nacional de Desarrollo como son: i) el Informe Anual sobre el balance del PND que debe contener al menos los indicadores de seguimiento al Plan con corte a 31 de diciembre del año anterior, los resultados de las evaluaciones de política pública adelantadas durante el año anterior y la agenda de evaluación vigente y ii) el Informe al Congreso sobre el avance del PND.

42

Cambios en el componente 2 : Se modifica a través de la inclusión de un subcomponente que busca apoyar el desarrollo y el pilotaje de una plataforma de open data construida sobre cuatro (4) bases de datos del DNP; así como sobre las bases de datos entregadas al DNP en el marco de las evaluaciones. Para esto último se plantea incluir en los términos de referencia una cláusula para obligar al contratista a entregar la documentación técnica de las bases de datos de las evaluaciones de acuerdo a estándares internacionales. Esto ya se está realizando en los términos de referencia de las evaluaciones. Cambios en el componente 3: Se modifica incluyendo el desarrollo e implementación del sistema de M&E a departamentos y municipios (SINERGIA Territorial) de tal forma que se logre el seguimiento a la ejecución del Plan Nacional de Desarrollo. El desarrollo del sistema se prevé a través de metodologías, sistemas de información, capacitación y seguimiento y verificación a los planes de desarrollo locales. Cambios en el componente 4: Se excluye el apoyo al DANE, en la medida que la entidad requiere un programa de mayor alcance para atender sus necesidades. De la misma forma, dado que las acciones inicialmente propuestas para apoyar la Comisión Intersectorial de Políticas de Información y Gestión para la Administración Pública - COINFO perdieron pertinencia, éstas no se continuaron. Por otra parte, se incorpora un nuevo subcomponente para la modernización y la integración de flujos de información a través de la estandarización de procesos que mejoren la calidad de la información de SINERGIA. Igualmente, se promueve y apoya la política nacional de Open Data. En septiembre de 2012 el Banco envía la enmienda para firma por parte del Gobierno Nacional – Director del DNP y Ministro de Hacienda y Crédito Público. El 31 de enero de 2013, el Banco emitió un comunicado mediante el cual autorizaba la extensión del periodo establecido para la firma de la enmienda hasta el 15 de febrero, sin embargo, ésta fue firmada por el Ministro de Hacienda Mauricio Cárdenas el 5 de febrero y remitida al Banco. Teniendo en cuenta el tiempo de aprobación (1 año y 3 meses) que se tomó la enmienda en marzo de 2013 fue necesario replantear la destinación de los recursos entre componentes de acuerdo a las necesidades de la DSEPP para esta fecha, igualmente la extensión de la fecha de cierre del Crédito BIRF 7620-CO del 15 de marzo de 2014 hasta 31 de diciembre 2014 fue aprobada por el Banco Mundial el 26 de noviembre de 2013 mediante una Restructuración de segundo nivel. En marzo de 2014 se solicitó una reasignación entre componentes, con el fin de financiar 2 evaluaciones de la agenda de evaluaciones 2014, la mencionada reasignación fue aprobada mediante no objeción por parte del Banco Mundial el 15 de abril de 2014. La información que se presenta a continuación, contiene todos los cambios que se realizaron la estructura inicial con la mencionada enmienda.

3. Evaluación de desempeño

3.1 Evaluación del objetivo de la operación De acuerdo a la estructura general del proyecto que se presentó en el capítulo 2 del presente informe, a continuación se detallan los logros y avances obtenido en cada uno de los 5 componentes y el resultado final del avance de los indicadores a 31 de diciembre de 2014.

43

Vale la pena resaltar que los logros del proyecto se evidencian a partir de los responsables de su ejecución, en donde el componente 1 fortaleció el ciclo de la inversión pública, mediante el Sistema Unificado de Inversiones y Finanzas Públicas, los componentes 2, 3 y 4 se enfocar en fortalecer el modelo de M&E mediante Sinergia, y finalmente por medio del el componente 5 se ejecutó todo lo relacionado con la administración y operación del crédito, como se pude observar en la siguiente imagen.

La asignación de los recursos por componentes se muestra en la siguiente grafica con su respectiva ejecución, en donde se evidencia que la ejecución total fue de 99%, quedando por ejecutar $85.000 USD.

Componente Apropiación

$ USD Participación Ejecución

1 590.572 7% 100% 2 4.789.427 56% 96% 3 993.459 12% 100% 4 1.259.406 15% 100% 5 867.136 10% 99%

3.2 Evaluación de los resultados de la operación

3.2.1 Componente 1: El fortalecimiento institucional del Prestatario y evaluación de sus

proyectos de inversión pública nacional y sub – nacional

Se cuenta actualmente con información de presupuesto regionalizada, lo que implica que la inversión del Plan Nacional de Desarrollo se encuentra actualizada con las bases históricas y mensuales, igualmente se lleva a cabo el seguimiento de la regionalización del presupuesto. Las fichas técnicas se han rediseñado de acuerdo a los requerimientos del Director General del

7%

56%12%

15%

10%

44

DNP, y se incluyeron comparativos de los periodos presidenciales 2007-2010 y 2011-2014, en las diferentes fuentes de la inversión pública a nivel departamental, tales como: Sistema General de Regalías -SGR, Sistema General de Participaciones –SGP y Presupuesto General de la Nación- PGN. Lo anterior demuestra el fortalecimiento de las capacidades técnicas e institucionales del DNP en la formulación y el seguimiento de la inversión pública de acuerdo con la estrategia nacional de inversión pública. Se desarrollaron los módulos de formulación, programación, ejecución y seguimiento del Sistema Unificado de Inversiones y Finanzas Públicas (SUIFP) que da apoyo durante el ciclo de la inversión pública y fue creado y reglamentado a partir del Decreto 2844 de 2010.

Indicador componente 1 Unidad de medida

Línea de Base25

Avance a 31 de diciembre de

201426 Periodicidad

Porcentaje del presupuesto regionalizado del total del presupuesto

de inversión.

Porcentaje

68,7%

81.2%

Anual

3.2.2 Componente 2: Consolidación y expansión del uso del sistema nacional de M&E del

Prestatario

• Con los recursos del crédito se contrataron 23 Evaluaciones. Ver anexo 1. • Se cuenta con una herramienta (RADAR) que ha permitido agrupar algunas de las

evaluaciones realizadas en el país. El objetivo general del RADAR nacional es “potencializar el uso de la evaluación como una herramienta efectiva de gestión pública en Colombia a partir de su compilación y sistematización; de esta forma, busca mejorar la toma de decisiones de los gerentes públicos y la interacción entre los diferentes responsables temáticos” (Garzón Mora 2010), esta herramienta se puede consultar en el siguiente enlace: https://sinergia.dnp.gov.co/SISDEVAL/paginas/Evaluaciones/RadarEvaluaciones.aspx

• Se realizaron los 3 primeros levantamientos de la Encuesta de percepción ciudadana, contrato que tuvo por objeto: “Diseñar, ejecutar y analizar el levantamiento de información que le permita al gobierno nacional hacer seguimiento a la evaluación ciudadana de resultados de sus acciones, en lo que concierne a los temas contenidos en el plan nacional de desarrollo 2010 – 2014 y otros que sean definidos como prioritarios por el Departamento Nacional de Planeación – DNP” a 31 de diciembre se han realizado 8 levantamientos financiados con el Presupuesto General de la Nación- PGN.

25 31 de diciembre de 2011 26 Del total del presupuesto hay recursos por regionalizar 5,3% y no regionalizable 13,5%.

45

https://sinergia.dnp.gov.co/SISDEVAL/paginas/Evaluaciones/RadarEvaluaciones.aspx

• Se apoyó técnicamente el fortalecimiento institucional en relación con la medición de la pobreza por ingresos, en su fase II finalizó las nuevas canastas de pobreza y pobreza extrema, con base en la información de la Encuesta Nacional de Ingresos y Gastos 2006 / 2007 y efectuó la actualización de las series 2002 y 2009. La Subdirección de Promoción Social y Calidad de Vida, del DNP desarrolló la versión para Colombia del Índice de Pobreza Multidimensional (IPM- Colombia), siguiendo la metodología del Oxford Poverty and Human Development Initiative (OPHDI).

• Se inició con el desarrollo de la metodología de la estrategia de “Articulación metas y presupuesto” en donde se presentó el documento “Lineamientos para la construcción y escalabilidad del Plan Nacional de Desarrollo”. Este documento tuvo como propósito orientar las acciones de los actores que participan en los procesos de formulación, presupuestación, ejecución, seguimiento y evaluación del Plan Nacional de Desarrollo -PND, según los procedimientos establecidos en las normas orgánicas nacionales que regulan la materia, de tal forma que permita hacer más coherente la ejecución presupuestal y el seguimiento estratégico de los objetivos planteados en el PND.

• Se realizó una contratación que tuvo como fin el diseño e implementación de lineamientos de comunicación efectiva basada en lenguaje ciudadano, la cual contenía la aplicación adecuar los productos de conocimiento que se generan al tipo de usuario que los utiliza, de manera tal que se promueva el entendimiento del mensaje transmitido y una adecuada utilización de las herramientas de difusión existentes.

• En el marco del CONPES 3654 de 2011 se diseñó la estrategia de seguimiento y monitoreo de los procesos de Rendición de Cuentas de las Entidades del Orden Nacional, lo cual sirvió como insumo para la elaboración del manual único de rendición de cuentas, que contribuyó a fortalecer el seguimiento a metas de gobierno en materia de rendición de cuentas.

Indicador Componente 2

Unidad de medida

Línea de Base27


201428 Periodicidad

Porcentaje de evaluaciones totales

aprobadas en la agenda que alcanzan la fase de

diseño.

Porcentaje

60,0%

88,5%

Anual

27 31 de diciembre de 2011

28 Las evaluaciones aprobadas en la Agenda de Evaluaciones 2014 y que han alcanzado la fase de diseño a la fecha se encuentran listadas en el anexo 2.

46

3.2.3 Componente 3: Respaldo al desarrollo local de los sistemas de M&E locales y regionales

• Se alcanzaron los objetivos planteados para los pilotos de Medellín, Pasto y Tocancipá, los

cuales tenían como fin la consolidación de sus experiencias referente a sus sistemas de M&E para así diseminarlas y expandirlas como ejemplo de buena práctica.

• En Chocó y Medellín se dio apoyo y asesoría administrativa, jurídica y financiera a los funcionarios encargados, con el fin de generar transparencia, mejor información y crear las bases de una cultura de M&E en dichas regiones.

• En lo referente a la mejora y retroalimentación de la Metodología de Medición del Desempeño Municipal que utiliza el Departamento Nacional de Planeación para medir la gestión local el DNP, se identificaron recomendaciones viables a ser implementadas, acorde a parámetros técnicos y económicos y la difusión de experiencias exitosas a través del desarrollo de dos talleres de divulgación y socialización.

• En 2012 se realizó una batería de indicadores para Sinergia Territorial, y la articulación, con lo anterior se finalizó la propuesta conceptual, en donde se definieron los municipios y departamentos para la primera fase de la estrategia y se elaboró el plan de trabajo para 2012.

• Fortalecimiento de la capacitad institucional para la medición del desempeño y la entrega de información relacionada con las transferencias fiscales (capacitaciones).

• Desde 2012 la DSEPP consolida información territorial, a partir de unas fichas municipales y departamentales, que contiene información relacionada con la implementación del Plan Nacional de Desarrollo en el territorio.

• Asistencia técnica insitu mediante la Estrategia de Sinergia Territorial - ST, se tiene como principales logros la asistencia brindada a los Municipios y Departamentos que así lo solicitan y el Seguimiento Territorial. Durante el 2014 se logró consolidar sistemas de seguimiento en más del 80% de las 86 entidades territoriales – ET vinculadas a la estrategia; así mismo, más del 70% de las ET vienen realizando seguimiento a sus planes de desarrollo de manera permanente y periódica, realizan informes de seguimiento y han formalizado e interiorizado a través de equipos y decretos (u ordenanzas) los sistemas de seguimiento con lo cual se garantizará la sostenibilidad de los logros hasta ahora alcanzados. Vale la pena resaltar que la operación de la estrategia de Sinergia Territorial a partir de 2014, se está financiando a través del Proyecto de Fortalecimiento de las Entidades Territoriales, Crédito BIRF 8320-CO, ejecutado por la Dirección de Desarrollo Territorial Sostenible - DDTS del DNP con la asistencia técnica de Findeter.

47

Indicador componente 3 Unidad de medida

Línea de Base29


201430 Periodicidad

Entidades Territoriales que firmaron el memorando de

entendimiento

Número

56

86

Anual

3.2.4 Componente 4: Establecimiento de mecanismos para mejorar la calidad, relevancia y

efectividad al costo de la información pública

• Formulación del Plan Estadístico Nacional, en donde se depuraron y reclasificaron de 1227 operaciones estadísticas que permitan tomar acciones correctivas para mejorar los flujos de información y por ende, la calidad y disponibilidad de la información. El DANE es el propietario actual de esta información.

• Implementación de Tableros de Control para seguimiento de PND. • En 2010 se puso en funcionamiento las Plataformas Tecnológicas SISDEVAL: se

desarrolló la carga y consulta de la dinámica de la agenda de evaluaciones y de las evaluaciones finalizadas, y SISMEG: se desarrolló el Datamart básico para la consulta de información gerencial y se terminó el módulo de consulta para el ciudadano. Se independizó la consulta de indicadores SIGOB, es decir toda la información ahora se carga y se consulta en SISMEG.

• Dentro del segundo subcomponente, el cual tiene como fin la modernización e integración de los mecanismos técnicos, sistemas y canales de entrega de los flujos de información, se efectuó la contratación de la firma Activa Marketing en el año 2013, la cual correspondió a la etapa de generación de un plan para mejorar la experiencia del usuario en el sistema Sinergia, a través de un diseño parcial con técnicas y herramientas de última generación en desarrollo de páginas web, diseñando un desarrollo de una plataforma tecnológica sólida e integral para Sinergia.

• En la actualidad el Sistema cuenta con un total de 170 interfaces de usuario que requieren ser rediseñadas e implementadas para poder mejorar la usabilidad y la experiencia del usuario final, utilizando herramientas de última tecnología en diseño e implementación web para computadores y dispositivos móviles.

• Por lo anterior y con el fin de dar continuidad al desarrollo tecnológico y a la contratación con la firma Activa, para 2014 el DNP ha aunado esfuerzos en conjunto con el Programa de Tierras & Desarrollo Rural de USAID – “Agency for International Development” para realizar el rediseño e implementación del nuevo “Front End” de Sinergia.

• Desde el año 2012 la DSEPP ha venido trabajando en la implementación efectiva de la iniciativa de Datos Abiertos promovida por parte de la OCDE adelantando las siguientes acciones a la fecha:

29 31 de diciembre de 2011 30 Listado de municipios y etapa de la estrategia (ver anexo 3)

48

2012: Se contrataron dos profesionales con experiencia específica para documentar bajo la iniciativa en cuestión, un total de trece evaluaciones de distintos sectores y metodologías. 2013: Se incluyó en los contratos con las firmas consultoras la actividad correspondiente a documentar las evaluaciones. Se organizaron talleres de capacitación en el software Nesstar para que las firmas consultoras se familiarizaran con el proceso. 2014: Se contó con el apoyo de un consultor del Banco Mundial, para la elaboración de la Guía Protocolo de Difusión de Información Estadística del Departamento Nacional de Planeación. Dentro de los contratos de evaluaciones se modificó el número de entregas de la documentación, pasando de una sola entrega al final de la evaluación, a cuatro entregas parciales con el fin de facilitar el ejercicio de revisión, seguimiento y consolidación de la información.

Indicador componente 4

Unidad de medida

Línea de

Base31

Avance a 31 de dic de 201432

Periodicidad

Número de encuestas con PAD / Total encuestas

para evaluaciones

Porcentaje

0

100%

Anual

3.2.5 Componente 5: Ejecución del proyecto

Por este componente se financió la administración y operación del proyecto, garantizando así una correcta ejecución, contratación y operación; teniendo en cuenta que los consultores contratados para tal fin estaban capacitados en las nomas de Selección y Contratación de Consultores del Banco Mundial, así como en las operaciones y administración financiera; quienes además contaban con la experticia para asesorar al Director del proyecto, a la Secretaria General del DNP y demás Direcciones Técnicas que participaron en la ejecución del proyecto: los roles financiados por el componente administrativo fueron:

31 31 de diciembre de 2011 32 Se logró el 100% teniendo en cuenta que a partir de 2014 todos los contratos de evaluaciones incluyen una clausula en la cual se solicita que las firmas entreguen las bases de datos documentadas (para este año fueron 11), por otra parte de las 85 evaluaciones que se han ejecutado desde el 2002 (no incluye evaluaciones ejecutivas), en listado se encuentran evaluaciones realizadas antes de 2011 que solo después de finalizada su ejecución se realizó la documentación de los datos (ver anexo 4)

49

• Especialista en adquisiciones • Apoyo en adquisiciones • Especialista financiera • Coordinador del crédito • Apoyo administrativo • Auditoría financiera del proyecto a partir del 2012

- Vigencia 2012 y primer semestre de 2013 - Segundo semestre de 2013 - Vigencia 2014 y cierre del proyecto

3.3 Evaluación del desempeño del prestatario durante la operación.

Con base en los resultados descritos lo largo de este informe se generan a continuación unas conclusiones generales sobre la implementación del programa y el desempeño del DNP como ejecutor: • Pertinencia: De acuerdo al objetivo del programa y de los distintos componentes, se evidencia

que son pertinentes para las necesidades del país en cuanto a información relacionada con presupuesto, seguimiento y evaluación, resaltando también que la Dirección de Seguimiento y Evaluación – DSEPP y la Direcciones de Inversiones y Finanzas Públicas - DIFP del DNP, eran la instancias técnicas del Gobierno nacional llamadas constitucionalmente para el desarrollo e implementación de las actividades en marcadas en este crédito.

• Eficiencia: En cuanto a los costos finales del programa, estos correspondieron con los costos previstos, las actividades y productos se obtuvieron con los recursos programados, se dieron todas las contrataciones previstas y no hubo sobre precios. En los casos en que se presentó desfinanciación de algún componente el DNP oportunamente solicitó reasignación de recursos entre componentes.

Al 31 de diciembre de 2014 se ejecutaron en 99% los recursos del crédito, quedan así un saldo de usd$85.000.

• Eficacia: En cuanto al cumplimiento de las metas, estas se lograron satisfactoriamente. Sin

embargo, los indicadores definidos inicialmente para el seguimiento y la evaluación del programa no fueron pertinentes. Con la enmienda los 5 indicadores de resultado se eliminan y se acuerdan 2 nuevos indicadores; lo mismo sucedió con los indicadores de cada componente en donde se pasó de 17 indicadores a 4, lo que simplificó su seguimiento y facilito el entendimiento de los logros a partir de los indicadores.

• Lecciones aprendidas:

- Este crédito además de los fines técnicos con los que se concibió, también fue un piloto

para la implementación y ejecución de créditos al interior del DNP, teniendo en cuenta que este fue el primero en ser ejecutado directamente por la entidad.

50

- La estructura para la implementación del programa al interior del DNP probó ser un

esquema viable y efectivo, que además permitió generar documentos que registraron los avances y decisiones estratégicos del programa, como el Manual Operativo, el Plan de Adquisiciones y el Plan Operativo Anual.

- También se evidenció que el organigrama con el que se ejecutó el crédito al interior del DNP, fue exitoso en su implementación; teniendo en cuenta las diferentes instancias de aprobación y que las decisiones no estaban centradas en la DSEPP como responsable técnico. Lo anterior se evidencia también con la figura de comité operativo que se constituyó para los fines administrativos operativos del crédito.

- Se presentó un período de tiempo considerable entre la solicitud de la enmienda y su

aprobación (de junio de 2011 a febrero de 2013), la ejecución de las actividades no se retrasó debido a que el gobierno financió con sus recursos del Presupuesto General de la Nación – PGN, las actividades requeridas en dicho período.

- Se evidenció la importancia y relevancia de formular indicadores de los programas de manera consistente con los objetivos del proyecto planeando un mediano plazo y sobre todo de disponer de manera rápida y sistemática de la información necesaria para su construcción tanto en la línea de base como en el seguimiento. No contar con los indicadores idóneos dificulta la valoración de los avances y la toma de decisiones oportuna para los correctivos en caso de requerirse.

- Durante la ejecución, el crédito contó con 6 coordinadores diferentes por parte del DNP, pero aun así el DNP garantizó que no se perdiera la trazabilidad de las actividades y el seguimiento de las mismas.

3.4 Evaluación del desempeño del Banco durante la operación. Se agradece al Banco los estudios anteriores al crédito, en los cuales se identificaron las necesidades existentes del Gobierno en materia de información, monitoreo y evaluación; los cuales fueron insumos primordiales en el momento de las estructuración del proyecto. Desde el punto de vista del DNP, el desempeño del Banco durante la preparación y ejecución de la operación se considera satisfactorio, incluyendo la participación activa y siempre pertinente de los especialistas de administración financiera y de adquisiciones. Durante la ejecución del proyecto, el equipo del Banco siempre atendió las consultas y solicitudes algunas veces tomando tiempos de revisión adicionales; el Banco realizó constantemente misiones de supervisión y misiones financieras, supervisando de esta manera la operación y asegurando el cumplimiento de los objetivos de desarrollo.

51

El tiempo de trámite y respuesta que tomó la enmienda, fue un periodo desperdiciado en términos de ejecución, en este caso falto más participación del Banco para involucrarse con los temas y agilizar de alguna manera esta situación, esta fue la razón principal para extender el crédito por 9 meses más y de esta manera lograr mayor ejecución de los recursos y culminar con los objetivos propuestos.

3.5 Descripción de las medidas propuestas para la futura operación del proyecto. De acuerdo al ciclo y apropiación presupuestal de los recursos que ejecutan las entidades nacionales, el ciclo de la inversión pública definido por el DNP y el MHCP implica la formulación de un proyecto de inversión, mediante el cual se apropian y se ejecutan los recursos independientemente de la fuente de financiación (crédito, nación, donación, otros). Asimismo fue como se ejecutaron los recursos apropiados del crédito en donde el proyecto “Fortalecimiento y fomento de la cultura del seguimiento y evaluación de políticas públicas en el nivel nacional y territorial” con código Bpin 2011011000244, contó con la financiación de recursos crédito y nación. Lo que implicó que si bien el crédito no contaba con contrapartida según el contrato de empréstito, la DSEPP contó con recursos adicionales para el cumplimento de las actividades misionales también incluidas en los componentes del crédito. Con el fin de continuar fortaleciendo el sistema de M&E y teniendo en cuenta que desde 2015 no se apropiaron recursos de crédito, se formuló un nuevo proyecto de inversión solicitando un incremento en los recursos del PGN, con lo que se logre compensar los recursos de crédito. En donde sus principales características son: Nombre: Fortalecimiento del sistema nacional de evaluación y de gestión de resultados - Sinergia en el nivel nacional y territorial Código Bpin: 2014011000087 Horizonte del proyecto: 2015 - 2018 Con una programación presupuestal, distribuida anualmente de la siguiente manera:

2015 2016 2017 2018

Recursos solicitados33 9.847.615.961 11.704.742.967 11.771.616.972 12.538.412.572

33 Los recursos solicitados están en la formulación del proyecto, lo cual no implica que en la ejecución se vayan a apropiar el 100% de los mismos, lo anterior está sujeto a las discusiones del presupuesto que se dan anualmente en el Congreso de la República y al respectivo Decreto de Liquidación que se expide para cada vigencia.

52

Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Not Applicable.

53

Annex 9. List of Supporting Documents Consejo Nacional de Politica Económica y Social - CONPES (2004), Renovation of Public

Administration: Management by Results and the Reform of the National System of Evaluation, CONPES 3.294/2004, Bogotá.

Departamento Nacional de Planeación- DNP (2015), Informe Final - Fortalecimiento de la información pública, seguimiento y evaluación para la gestión por resultados en Colombia (Crédito BIRF 7620-CO), Dirección de Seguimiento y Evaluación de Políticas Públicas – DSEPP, March 2015, Bogotá.

DNP (2014b), Evaluación del desempeño integral de los municipios y distritos, Vigencia 2013, September 5, 2014, Bogotá.

DNP (2014) SINERGIA, 20 años mejorando la gestión pública, Dirección de Seguimiento y Evaluación de Políticas Públicas, Unpublished Manuscript, Bogotá.

Independent Evaluation Group – IEG (2007), A Diagnosis of Colombia’s National M&E System - SINERGIA, ECD Working Paper Series, No.17: February 2007, Washington, D.C.: World Bank.

OECD (2013), Colombia: Implementing Good Governance, OECD Public Governance Reviews, OECD Publishing, Paris.

World Bank (2008-2014) Colombia - Strengthening Public Information, Monitoring, Evaluation for Results Management in Colombia - Implementation Status Results Reports, Sequence 1-11, Washington, D.C.: World Bank.

World Bank (2012), Colombia - Strengthening Public Information, Monitoring and Evaluation for Results Management Project: Restructuring, Washington, DC: World Bank. http://documents.worldbank.org/curated/en/2012/06/16484658/colombia-strengthening-public-information-monitoring-evaluation-results-management-project-restructuring

World Bank (2011), Colombia - Country Partnership Strategy (CPS) for the period FY2012-2016, Report No. 60620-CO, June 12, 2011, Washington, D.C.: World Bank. http://documents.worldbank.org/curated/en/2011/06/14494030/colombia-country-partnership-strategy-cps-period-fy2012-2016

World Bank (2008b), Colombia - Strengthening Public Information, Monitoring and Evaluation for Results Management Project, Project Appraisal Document, Washington, DC: World Bank. http://documents.worldbank.org/curated/en/2008/11/10066508/colombia-strengthening-public-information-monitoring-evaluation-results-management-project

World Bank (2008a), PHRD: Colombia-Monitoring and Evaluation Project, Grant Reporting and Monitoring Completion Report, PHRD Fund-Technical Assistance Trust Fund, Washington, DC: World Bank.

Stakeholders consulted as part of the ICR Mission National Planning Department – DNP Jenny Fabiola Paez, Delegada del Director DNP Adriana Camacho, Directora de Seguimiento y Evaluación de Politicas Públicas – DSEEP Diego Dorado, Ex Director de Seguimiento y Evaluación de Politicas Públicas – DSEEP José Mauricio Cuestas, Director de Inversiones y Finanzas Publicas Alexandra Gonzalez, Subdirectora de Crédito Julián Castro, Asesor de la Subdirección de Crédito Margarita Borda, Coordinadora del Proyecto Alejandro Torres, Coordinador de Evaluaciones Jasson Cruz, Coordinador de Seguimiento Adriana Marquez, Coordinadora Innovación e Investigación

54

http://documents.worldbank.org/curated/en/2008/11/10066508/colombia-strengthening-public-information-monitoring-evaluation-results-management-project

http://documents.worldbank.org/curated/en/2008/11/10066508/colombia-strengthening-public-information-monitoring-evaluation-results-management-project

Camila Maria Aguilar, Directora de Desarrollo Territorial Hernan Orozco, Coordinador Sinergia Territorial - DDTS Diego Riaño, Subdirector de SFPT-DDT Andrés Salazar, Asesor, DSEEP Libia Janeth Mogollón, Asesora, Direccion de Inversiones y Finanzas Públicas Jeannette Venegas, Consultora de SUIFP Carolina Beltran, Enlace temas relacionados con la Banca, Secretaría General DNP Jose Rodrigo Bermudez, Especialista en Adquisiciones Rosalbina Trejos, Especialista Financiera Stakeholders that participated in “SiNERGIA Seguimiento” Ana Marta Miranda, Jefa de Planeación y Estudios Sectoriales, Ministerio de Tecnologías de la Información y las Comunicaciones - MINTIC Marcela Costa, Planning Office, MINTIC Stakeholders that participated in ‘SINERGIA Evaluaciones” Alejandro Torres, Coordinador de Evaluaciones Ana María Montañez, Contratista DNP - DSEPP Yachay Tolosa, Contratista DNP - Grupo de Proyectos Especiales Oscar Rico, Programa de Atención Humanitaria a Población Desplazada, UARIV Nelly Moreno, Programa de Atención Humanitaria a Población Desplazada, UARIV Constanza Salamanca, Programa de Atención Humanitaria a Población Desplazada, UARIV Oscar Rodriguez, Director de la evaluación - Econometría S.A. Yimer Yezid Botiva Gutiérrez, S.E.I. Stakeholders that participated in “SiNERGIA Territorial” Jorge Humberto Valencia, Secretario, Planeación, Alcaldía de Guadalajara de Bugo Miguel Vásquez, Planeación, Alcaldía de Lebrija Santiago Echandia Gutiérrez, Secretario, Planeación, Alcaldía de Chía Luz Elena Chávez Pena, Directora, Planeación, Alcaldía de Chía World Bank ICR Team Carlos Rodriguez Castelán, Senior Economist, TTL Maria Dolores Arribas, Advisor, World Bank Poverty Directorate Jessica Terry, Operation Analyst, Poverty Directorate Cecilia Zanetta, Consultant

55

Cor

di l ler

a Orie

nta l

Cor

dil le

ra C

entr

al

Cord

i l lera O

ccident a

l

PicoCristóbal Colón(5,775 m)

C A Q U E T Á VA U P É S

G U AV I A R E

M E T A

H U L AC A U C A

C H O C Ó

A N T I O Q U I A

VALLE DELCAUCA

RISARALDÁ

QUINDIO

C U N D I N A -M A R C A

DISTRITOCAPITAL

G U A I N Í A

V I C H A D A

C A S A N A R E

C E S A R

NORTE DESANTANDER

A R A U C A

A M A Z O N A S

N A R I Ñ O

P U T U MAYO

CALD A S

CÓRDOBA

BOL Í

VA

R

BO

YAC

Á

SUC

RE

S A N TA N D E R

TOL IMA

La Pedrera

PuertoSantander

PuertoPizarro

PuertoLeguízamo

PuertoHuitoto

Macujer

Calamar

Mapiripana

Santa Rita

PuertoNueva

Chaviva

San Juande Arama

GarzonGuapí

Ipiales

GaviotasSan Pedro

Miraflores

San Vicentedel Caguán

Giradot

Palmira

Buga

Cartago

Chiquinquirá

Socorro

Turbo

Yarumal

Quibdo

PuertoAsis

El Encanto

Locas deCahuinari

Lérida

Yavarate

Tabaquén

Brujas

San Rafael

El Baneo

Ocaña

Maganqué

Leticia

Mitu

PuertoInirída

PuertoCarreño

Florencia

Neiva

Popayan

Cali

Pasto

San Josédel Guaviare

Villavincencio

Mocoa

ArmeniaPereira

TunjaYopalManizales

Bucaramanga

Monteria

Sincelejo

Valledupar

Cúcuta

Medellin

Ibaque

BOGOTÁ

MA

GD

ALE

NA

LA G

UAJ I RA

Arauca

La Pedrera

PuertoSantander

PuertoPizarro

PuertoLeguízamo

PuertoHuitoto

Macujer

Calamar

Mapiripana

Santa Rita

PuertoNueva

Chaviva

San Juande Arama

GarzonGuapí

Tumaco

Ipiales

GaviotasSan Pedro

Miraflores

San Vicentedel Caguán

Giradot

Palmira

BugaBuenaventura

Cartago

Chiquinquirá

Socorro

Turbo

Acandí

Yarumal

Quibdo

PuertoAsis

El Encanto

Locas deCahuinari

Lérida

Yavarate

Tabaquén

Brujas

San Rafael

El Baneo

Ocaña

Maganqué

Puerto Bolívar

Leticia

Mitu

PuertoInirída

PuertoCarreño

Arauca

Florencia

Neiva

Popayan

Cali

Pasto

San Josédel Guaviare

Villavincencio

Mocoa

ArmeniaPereira

TunjaYopalManizales

Bucaramanga

Monteria

Sincelejo

Cartegena

BarranquillaSanta Marta

Valledupar

Cúcuta

Ríohacha

Medellin

Ibaque

BOGOTÁ

C A Q U E T Á VA U P É S

G U AV I A R E

M E T A

H U L AC A U C A

C H O C Ó

A N T I O Q U I A

VALLE DELCAUCA

RISARALDÁ

QUINDIO

C U N D I N A -M A R C A

DISTRITOCAPITAL

G U A I N Í A

V I C H A D A

C A S A N A R E

C E S A R

NORTE DESANTANDER

AT L Á N T I C O

A R A U C A

A M A Z O N A S

N A R I Ñ O

P U T U MAYO

CALD A S

CÓRDOBA

BOL Í

VA

R

BO

YAC

Á

SUC

RE

MA

GD

ALE

NA

LA G

UAJ I RA

S A N TA N D E R

TOL IMA

R.B. DEVENEZUELA

BRAZIL

PERU

ECUADOR

PANAM

A

Guaviare

Negro

Cauc

a

Atrato

Vichada

Patía

Ptumayo

Casanare

Atacavi

Meta

Vaupés

Caquetá

Caguán

Magdalena

Caribbean SeaLago de

Maracaibo

PACIFICOCEAN

To Maracaibo

To Mérida

To Guasdualito

To Ibarra

Cor

di l ler

a Orie

nta l

Cor

dil le

ra C

entr

al

Cord

i l lera O

ccident a

l

PicoCristóbal Colón(5,775 m)

75°W 70°W

75°W 70°W

5°N

10°N

0°N

5°N

0°N

COLOMBIA

0 80 160 240

0 1208040 160 200 Miles

320 Kilometers

IBRD 33388R1

DECEMBER 2013

COLOMBIACITIES AND TOWNS

DEPARTMENT CAPITALS

NATIONAL CAPITAL

RIVERS

MAIN ROADS

RAILROADS

DEPARTMENT BOUNDARIES

INTERNATIONAL BOUNDARIES

This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank Group, any judgment on the legal status of any territory, or any endorsement or acceptance of such boundaries.

GSDPMMap Design Unit

San Andres

12°30’

12°35’

81°40’

13°20’

81°22’

MILES

0 2MILES

0 2

SanAndres

Providencia

SantaCatalina

SAN ANDRÉS, PROVIDENCIAand SANTA CATALINA

The World Bank · 2016-07-08 · SIGOB Sistema de Programación y Gestión por Objetivos y...

Documents

Transcript of The World Bank · 2016-07-08 · SIGOB Sistema de Programación y Gestión por Objetivos y...