Document of The World Bankdocuments.worldbank.org/curated/pt/... · Document of The World Bank....

Document of The World Bank

Report No: ICR00001911

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-48390)

ON A LOAN

IN THE AMOUNT OF EURO 29.6 MILLION (US$37.7 MILLION EQUIVALENT)

TO

ROMANIA

FOR AN

AVIAN INFLUENZA CONTROL AND HUMAN PANDEMIC

PREPAREDNESS AND RESPONSE PROJECT

UNDER THE

GLOBAL PROGRAM FOR AVIAN INFLUENZA AND HUMAN PANDEMIC PREPAREDNESS AND RESPONSE (GPAI)

June 27, 2011

Sustainable Development Department and Human Development Department Romania Country Unit Europe and Central Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS (Exchange Rate Effective May 2011)

Currency Unit = New Romanian Lei (RON)

1 EURO = 4.128 RON 1

FISCAL YEAR January 1 – December 31

ABBREVIATIONS AND ACRONYMS

AI Avian Influenza ANSVSA National Sanitary Veterinary and Food

Safety Authority APCP Agricultural Pollution Control Project BSL Biosafety Level CI ` Cantacuzino Institute ERL Emergency Recovery Loan EU European Union FAO Food and Agriculture Organization of

the United Nations GMP Good Manufacturing Practice GOR Government of Romania GPAI Global Program for Avian Influenza

and Human Pandemic Preparedness and Response

HD Human Development HPAI Highly Pathogenic Avian Influenza ICU intensive care unit ILO International Labor Organization IDAH Institute for Diagnosis and Animal

Health ISR Implementation Status Report

M&E Monitoring and Evaluation MEF Ministry of Economy and Finance MAKIS Modernizing the Agricultural

Knowledge and Information Systems MOPF Ministry of Public Finance MOH Ministry of Public Health MTR Mid-Term Review NGO Non-Governmental Organization NIL National Influenza Laboratory (at the

Cantacuzino Institute) OIE Office International des Epizooties

(The World Organization for Animal Health)

PAD Project Appraisal Document PDO Project Development Objective PMU Project Management Unit QAG Quality Assurance Group RDP Rural Development Project RFP Rural Finance Project SARS Severe Acute Respiratory Syndrome SD Social Development TTL Task Team Leader USAID United States Agency for

International Development WHO World Health Organization

Vice President: Philippe Le Houerou Country Director: Peter C. Harrold

Sector Manager: Daniel Dulitzky Project Team Leader: Wezi Marianne Msisha

ICR Team Leader: Lorena Kostallari

ROMANIA Avian Influenza Control and Human Pandemic Preparedness and Response Project

CONTENTS

DATA SHEET A. Basic Information ........................................................................................................... 1 B. Key Dates ....................................................................................................................... 1 C. Ratings Summary ........................................................................................................... 1 D. Sector and Theme Codes ................................................................................................ 2 E. Bank Staff ....................................................................................................................... 2 F. Results Framework Analysis .......................................................................................... 3 G. Ratings of Project Performance in ISRs ........................................................................ 8 H. Restructuring (if any) ..................................................................................................... 9 I. Disbursement Profile ...................................................................................................... 9 1. Project Context, Development Objectives and Design ................................................. 10 2. Key Factors Affecting Implementation and Outcomes ................................................ 16 3. Assessment of Outcomes .............................................................................................. 22 4. Assessment of Risk to Development Outcome ............................................................. 26 5. Assessment of Bank and Borrower Performance ......................................................... 27 6. Lessons Learned............................................................................................................ 30 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners............... 32 Annex 1. Project Costs and Financing .............................................................................. 33 Annex 2. Outputs by Component...................................................................................... 34 Annex 3. Economic and Financial Analysis ..................................................................... 36 Annex 4. Bank Lending and Implementation Support/Supervision Processes ................. 40 Annex 5. Beneficiary Survey Results ............................................................................... 42 Annex 5. Beneficiary Survey Results ............................................................................... 42 Annex 6. Stakeholder Workshop Report and Results ....................................................... 43 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 44 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................... 53 Annex 9. List of Supporting Documents .......................................................................... 54

MAP

1

DATA SHEET A. Basic Information

Country: Romania Project Name:

Avian Influenza Control & Human Pandemic Preparedness & Response Project

Project ID: P100470 L/C/TF Number(s): IBRD-48390 ICR Date: 06/27/2011 ICR Type: Core ICR Lending Instrument: ERL Borrower: ROMANIA Original Total Commitment:

USD 37.7M Disbursed Amount: USD 24.2M

Revised Amount: USD 21.7M Environmental Category: B Implementing Agencies: Ministry of Health National Agency for Sanitary, Veterinary and Food Safety (ANSVSA) Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 06/29/2006 Effectiveness: 03/19/2007 03/19/2007 Appraisal: 07/05/2006 Restructuring(s): 07/28/2009 Approval: 09/08/2006 Mid-term Review: 08/18/2008 11/24/2008 Closing: 12/31/2009 12/31/2010 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Unsatisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies:

Moderately Unsatisfactory

Overall Bank Performance: Moderately Satisfactory Overall Borrower

Performance: Moderately Unsatisfactory

2

C.3 Quality at Entry and Implementation Performance Indicators Implementation

Performance Indicators QAG Assessments (if any) Rating

Potential Problem Project at any time (Yes/No):

Yes Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Agricultural extension and research 13 13 General public administration sector 35 35 Health 50 50 Solid waste management 2 2

Theme Code (as % of total Bank financing) Health system performance 13 13 Natural disaster management 24 24 Other communicable diseases 25 25 Pollution management and environmental health 13 13 Rural services and infrastructure 25 25 E. Bank Staff

Positions At ICR At Approval Vice President: Philippe H. Le Houerou Shigeo Katsu Country Director: Peter C. Harrold Anand K. Seth Sector Manager: Daniel Dulitzky Benoit Paul Blarel Project Team Leader: Wezi Marianne Msisha Matthias Grueninger ICR Team Leader: Lorena Kostallari ICR Primary Author: Lorena Kostallari Sati Achath

3

F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The overall development objective of the project was to reduce the threat posed to humans and the poultry sector in Romania by HPAI and other zoonoses, and to support the preparation for, control of, and response to influenza pandemics and other infectious disease emergencies in humans. Revised Project Development Objectives (as approved by original approving authority) The revised project objective was to assist the Borrower in building its capacity to respond to potential future infections of humans by HPAI and other zoonoses, and preparing for, controlling and responding to influenza pandemics and other infectious disease emergencies in humans. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Appropriate contingency plans prepared and effective capacity to implement contingency plans built.

Value quantitative or Qualitative)

Plans incomplete and lacking coordination, needing substantial upgrading. Mechanisms and capacities for plan implementation are weak

Appropriate plans exist and are officially approved / issued. Effective mechanisms with demonstrated capacity for plan implementation are in place.

Indicator eliminated in July 2009 restructuring.

Date achieved 06/29/2006 08/31/2009 12/16/2009

Comments (incl. % achievement)

Appropriate contingency plans for avian influenza and human influenza pandemic were prepared, coordination mechanisms were defined and legislation revised. Capacity to implement the contingency plan was demonstrated in the district of Tulcea in Nov 2007.

Indicator 2 : HPAI outbreaks in poultry effectively contained


183 outbreaks between October 2005 and June 2006. Effectiveness of prevention of and response to HPAI outbreaks in poultry needs to be improved.

HPAI outbreaks in poultry are effectively contained (actual outbreaks or simulation exercises).

Indicator eliminated in July 2009 restructuring

Date achieved 06/29/2006 08/31/2009 12/16/2009 Comments (incl. % achievement)

There have been no HPAI outbreaks in poultry since the project became effective, however this result can not necessarily be attributed to the activities that took place under the project.

4

Indicator 3 : Risk of human infections reduced and treatment improved.


No human cases as of July 5, 2006. Vaccine production capacity is limited (< 1 million doses). ICU capacity for infectious disease treatment limited.

Additional vaccine production capacity built and fully operational, and ICU capacities for infectious diseases improved

Indicator eliminated in July 2009 restructuring


At restructuring, this indicator became new PDI 4 and PDI 5 (see below).

Indicator 4 : Number of Human Cases of Avian Influenza Value quantitative or Qualitative)

0 0 0


Target met.

Indicator 5 : Vaccine production increases.


Vaccine production capacity is limited (< 1 million doses)

Vaccine production has increased (no numerical target set)

Influenza vaccine production capacity has improved from 30 doses per week in 2008 to 79 doses per week in 2009


Target met.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Veterinary staff at central and country levels have improved knowledge and skills to design and perform epidemiological surveys, and to analyze animal disease data.

Value (quantitative or Qualitative)

Mobility, institutional structure and skill level of local veterinary staff insufficient.

Local veterinary staff fully skilled and equipped for timely response, and enhanced effectiveness of disease control demonstrated in


5

real outbreaks or simulation exercises.


Mobility was not improved. Enhancements related to "institutional structure" and "skills" were not pursued and activities were dropped.

Indicator 2 : County laboratories provide timely (according to EC legislation and OIE guidelines) disease diagnoses.


Time between sampling and diagnostic result above recommended threshold (24 hours).

Recommended time threshold observed on routine basis.



Equipment and reagents were not procured. Four out of 8 lots were suspended. Handling of last outbreak demonstrated capacity to operate within recommended thresholds in affected county.

Indicator 3 : Efficient animal disease information system operational in compliance with EC legislation and OIE guidelines


Existing systems are not comprehensive and not integrated.

Information system established and fully operational and staff being trained.



Comprehensive/integrated information system was not established and activity was dropped.

Indicator 4 : Quarantine, culling and disposal measures are improved, in accordance with environmental and animal welfare guidelines.


Technical capacity for outbreak control (quarantine, culling, and disposal) needs improvement.

Trained staff have new equipment that is fully operational, enabling outbreak control measures in line with applicable guidelines.



Procurement of CO2 containers and incinerators did not start.

Indicator 5 : Evidence of behavioral and managerial changes towards higher bio-security in small-holder poultry systems, following the successful implementation of awareness campaigns and knowledge transfer activities.


Low bio-security level in small-holder poultry production systems.

Farmers aware of options for increased bio-


6

security in small-holder poultry production systems, and some farmers start to implement some of their options of choice.


No activities were carried out.

Indicator 6 : Increased surveillance and outbreak investigation capacity


Good capacity at the central level. Capacity at the regional and district level needs to be strengthened

National Influenza Laboratory upgraded to BSL3 3 & regional labs equipped with real time PCR and staff training.

Civil works for the NIL at the CI for a Bio-Safety Level (BSL 3) facility completed on time (end December 2000); however, it has not yet been accredited as a BSL3 facility. This is expected to happen in 2011. No info available on regional labs


Target partially met

Indicator 7 : Vaccine production facility at "Cantacuzino Institute" strengthened upgraded


Current vaccine production capacity is limited (< 1 million doses) and needs to be developed.

Vaccine production unit at the Cantacuzino Institute upgraded.

Influenza vaccine production capacity has improved from 30 doses per week in 2008 to 79 doses per week in 2009


Target met

Indicator 8 : Isolation and intensive care units in 7 regional hospitals established.


Insufficient isolation capacity at central and regional level.

Infectious disease hospitals (2) in Bucharest upgraded and ICUs in regional hospitals (7) built

All equipment delivered to all nine hospitals and almost all installed and in use

7

and operational. Date achieved 06/29/2006 12/31/2010 12/22/2010 Comments (incl. % achievement)

Target largely met All equipment was installed and in use by Dec. 31, 2010.

Indicator 9 : Political and civic leadership organized around a national strategic risk communication plan


AI Contingency Plan to be revised to include a National Strategic Risk Communication Plan.

100% complete Indicator eliminated in July 2009 restructuring.


Activity was dropped.

Indicator 10 : Awareness-raising and behavior change interventions with population at risk using appropriate communications channels consistent.


25% of knowledge, attitude and practices studies and design of communications products implemented.

100% complete Indicator eliminated in July 2009 restructuring.


Draft strategy was prepared. No studies or campaigns were undertaken, but the Government was successful in managing communication by itself.

Indicator 11 : Program Planning and coordination enhanced (Human Health)


Plan exists but needs further development and operationalization. Chain of command needs clarification and coordination enhancement.

Plan operationalized and public health services improved

Plan was updated and made public. This activity was undertaken by the MOH in July 2009 in view of the H1N1 pandemic but not funded through the project.


Target met, but cannot be attributed to the project. CI did it on their own. The project was meant to support but they managed to do it with their own means, so that T/A activity was cancelled.

The following indicators which were mentioned in the PAD were not tracked/monitored during implementation

Animal Health: 1 Evaluation of veterinary services completed according to OIE guidelines.

2 Contingency plans revised and consolidated, and implementation guidelines and manuals available.

3 National reference laboratory provides timely confirmation of the local diagnoses.

8

4 National Institute for Control of Biological Products and Veterinary Drugs effectively applies molecular-biological methods for quality control of veterinary medicinal products.

5 ANSVSA has appropriate designs for applied veterinary research program and institute to fulfill its new mandate.

6 The National Institute for Control of Biological Products and Veterinary Drugs can operate facilities for administering highly infectious pathogens on live animals.

7 Poultry vaccination policy, contingency plan, and implementation manual developed.

8 Options for bio-security investments developed and known to poultry producers, and bio-security manuals available

Human Health 9

Procedures and standards upgraded and staff trained in their use; Technical guidelines and manuals updated Social distancing plan prepared; Clean out plans for hospitals prepared

10 Health professionals and support personnel trained for active surveillance, case finding, and proper diagnosis, treatment and care

11 Case fatality rate below the international average level during both pre-epidemic and pan-endemic phases ;

12 Research-based risk communication strategies and products developed, responding to the needs of priority audiences;

13 Communications strategies and products developed and used highlighting the actions and investments of participating ministries and the mobilization of group resources to contain the epidemic, generating social trust and credibility;

14 Evidence existent of consistent communication and information technologies, to promote reporting of outbreaks, fast response and an uninterrupted social dialogue;

15 Informational products developed and disseminated that educate priority audiences about possible scenarios and mitigation and control actions to be undertaken.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 02/14/2007 Satisfactory Satisfactory 0.00 2 07/20/2007 Satisfactory Moderately Satisfactory 0.09

3 12/20/2007 Moderately Satisfactory Moderately Unsatisfactory 0.38


5 12/29/2008 Moderately Unsatisfactory Unsatisfactory 1.14

6 03/10/2009 Moderately Unsatisfactory Unsatisfactory 2.36


9




H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made DO IP

07/28/2009 Y MU U 2.59 If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Moderately Unsatisfactory Against Formally Revised PDO/Targets Moderately Satisfactory Overall (weighted) rating Moderately Unsatisfactory

I. Disbursement Profile

10

1. Project Context, Development Objectives and Design 1.1 Context at Appraisal

The outbreak of Highly Pathogenic Avian Influenza (HPAI), which started in late 2003 in the Southeast Asian countries, quickly spread over Europe, Middle East, and Africa. Between February and May 2006, the first cases of H5N1 (or H5) infections in poultry or wild birds were reported in 34 countries, historically marking the fastest and most extensive geographic spread of the HPAI virus. In addition, June 2006 marked an increased evidence of human H5N1, including cases of human-to-human transmission. The HPAI situation in Romania: The first H5N1 virus in animals was detected in Tulcea county, (east of the capital Bucharest) at the beginning of October 2005. This case was followed by 52 cases in nine other counties. This first wave of Avian Influenza (AI) in Romania lasted until April 2006, followed by a second one in May 2006. While no human cases were experienced during that period, about one million birds died or were culled, resulting in a total cost of disease control and economic losses of around Euro 90 million. The outbreak resulted in market collapse due to loss of consumer confidence. It also caused a significant effect on the country’s agriculture sector and the economy as a whole. The aggravated situation in the country, especially after the second wave of the outbreak, revealed a number of major concerns: (i) Weak bio-security, particularly in backyard poultry systems. Poultry production played an increasingly important role in Romania’s agricultural sector, i.e., domestic production of poultry meat at project design stage doubled compared to the year 2000. This production was mainly based on small-holder (backyard) systems, in which 70 percent of the country’s poultry population was kept. (ii) Institutional weaknesses in the public veterinary institutions included the following issues:

(a) Lack of proper monitoring at national level. Systematic investigation of the population for analyzing disease prevalence was carried out only in high risk areas. This was considered to be inadequate.

(b) The country lacked an active surveillance system. At project appraisal stage,

Romania had in place a passive system, which was only reacting to reports of disease and testing dead or sick birds. Following the virus outbreaks, it became evident that there was a need for an active system for surveillance, which was going to increase samples tested.

(c) Lack of reliable epidemiologic information. The lack of reliable information

hampered the development of rational, targeted disease control measures as well as slowed down containment of the disease.

(d) Weak information system for animal disease. The country lacked an integrated

national system, including registration of poultry farms. (e) Culling and disposal of carcasses. The disposal of carcasses did not meet the

required safety level.

11

(iii) Issues in the Public Health Care System:

(a) A national plan and several government orders were already in place at project appraisal, constituting an acceptable strategic framework, including presentation of the information flows. However, the management process, including distribution of tasks, needed significant improvements. Furthermore, additional work was needed with regard to preparations for a possible flu pandemic and measures for reducing the virus effects.

(b) Although the existing surveillance system had proven to function properly, the risk of

AI outbreaks spread in humans, especially in remote areas, imposed the need for capacity strengthening at the central and local level to effectively investigate and control outbreaks. The capacity of district and regional laboratories needed to be significantly upgraded to perform all major tests, while the National Influenza Reference laboratory needed to be upgraded to bio-safety level 31

(BSL-3).

(c) In the context of virus spread or a pandemic situation, existing hospital capacity needed significant improvement. Hence, establishment of intensive care units in seven regional hospital/health facilities was considered necessary. Furthermore, despite the access to the World Health Organization’s (WHO’s) international stockpile of vaccines, Romania could not produce a significant share of vaccines required to cover the national needs. In order to meet its needs and be able to respond to a medium–size pandemic, the vaccine production unit at the Cantacuzino Institute (CI) needed major upgrading.

Rationale for Bank assistance: The justification for the Bank involvement was the Global Public Goods aspect of the HPAI, one of many emerging and re-emerging zoonoses2

, and its strong link to poverty reduction. HPAI control programs required a multi-disciplinary approach to integrate technical, social, economic, political, policy, and regulatory issues in addressing a complex problem. The Bank was well-placed to build upon its knowledge of multi-disciplinary approaches needed in the project, which drew on evidence and lessons learned in the various regions regarding emergency preparedness responses and multi-disciplinary approaches.

The Bank’s experience with multisectoral emergency response and risk-mitigation projects gave it considerable qualifications in bringing together the relevant ministries, government agencies, and the donor community, in understanding and addressing the social and economic impact, and in assuring high level political coordination. Given the Bank’s work with the Food and

1 A Biosafety level is the level of the bio-containment precautions required to isolate dangerous biological agents in an enclosed facility. The levels of containment range from the lowest biosafety level 1 to the highest at level 4. Bio Safety Level 3 (BSL-3) is applicable to clinical, diagnostic, teaching, research, or production facilities in which work is done with indigenous or exotic agents, which may cause serious or potentially lethal disease after inhalation. It includes various bacteria, parasites, and viruses that can cause severe to fatal disease in humans but for which treatments exist. BSL-3 facility is expected to improve the country’s capacity for surveillance of pandemic flu and other infections. 2 Zoonose is any infectious disease that can be transmitted from non-human animals, both wild and domestic, to humans or from humans to non-human animals (the latter is sometimes called reverse zoonosis or anthroponosis).

http://en.wikipedia.org/wiki/Biocontainment�

http://en.wikipedia.org/wiki/Biological_agent�

http://en.wikipedia.org/wiki/Biological_agent�

http://en.wikipedia.org/wiki/Infectious_disease�

http://en.wikipedia.org/wiki/Animal�

http://en.wikipedia.org/wiki/Human�

http://en.wikipedia.org/wiki/Anthroponosis�

12

Agriculture Organization (FAO), WHO, the World Organization for Animal Health (OIE 3

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved)

), European Union (EU), United States Agency for International Development (USAID), and other partners within Romania and at the international level to address both preparedness and outbreaks, and to assist with institutional assessments, the Bank could assist Romania in leveraging additional resources from other international and bilateral agencies. In addition to its financial role, the technical assistance provided by the Bank had been important in similar global or regional emergency situations such as Severe Acute Respiratory Syndrome (SARS), Tsunami relief, and the HIV/AIDS pandemic.

The overall development objective of the project at the time of Board approval was to reduce the threat posed to humans and the poultry sector in Romania by HPAI and other zoonoses, and to support the preparation for, control of, and response to influenza pandemics and other infectious disease emergencies in humans. Key indicators. The achievement of the PDO was to be measured by the following three outcome indicators as identified in the PAD: (i) evidence of institutional mechanisms for effective implementation of appropriate disease

contingency plans; (ii) evidence of an effective surveillance system for both animals and humans for HPAI and

other zoonoses; and (iii) evidence of increased capacities to handle human cases of infectious diseases.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification As further explained in Section 1.7, the Project was restructured in June 2009. The following Table shows the original and revised PDOs and their respective indicators:

Original Revised Objective: “to assist the Borrower in reducing the threat posed to humans and the poultry sector by HPAI and other zoonoses, and preparing for, controlling and responding to influenza pandemics and other infectious disease emergencies in humans.”

Objective: “to assist the Borrower in building its capacity to respond to potential future infections of humans by HPAI and other zoonoses, and preparing for, controlling and responding to influenza pandemics and other infectious disease emergencies in humans.”

Project Outcome Indicators: Project Outcome Indicators

3 The World Organization for Animal Health (OIE). The need to fight animal diseases at the global level led to the creation of the Office International des Epizooties through the international agreement signed on January 25, 1924. In May 2003, the Office became the World Organization for Animal Health but kept its historical acronym OIE.

13

Original Revised PDI1 Appropriate contingency plans prepared and effective capacity to implement contingency plans built.

- dropped -

PDI2 HPAI outbreaks in poultry effectively contained.

- dropped -

PDI3 Risk of human infections reduced and treatment improved.

PDI4 Risk of human infections reduced. Isolation and case management for infectious diseases improved. Number of human cases of Avian Influenza.

PDI5 Vaccine production increased (No numerical target set)

The restructuring was undertaken for the following reasons: • To remove the animal health component (Component 1) and related management and

communication activities from Component 3; • To reallocate part of the resources from the animal health to the human health component; • To revise the Project Development Objective (PDO) and results framework by deleting

objectives and results related to animal health activities from the relevant sections; and • To remove some other indicators and provide greater specificity or clarity to those

remaining. The component related to animal health was removed at Government request, due to implementation problems and the desire to shift priorities to human health.

1.4 Main Beneficiaries The expected beneficiaries of the original project included: • Smallholder poultry farmers who would receive competitive grants to improve their on-

farm biosecurity. • Poultry breeding farm owners who would receive compensation for culled poultry. • Veterinary networks, laboratory systems, disease surveillance and extension systems. • Institutional beneficiaries including ANSVSA (National Sanitary Veterinary and Food

Safety Authority) and MOH and their staff and their laboratories, surveillance and inspection services, and the infectious disease reference hospitals.

• Cantacuzino Institute which would receive funding for the purchase of equipment and infrastructural rehabilitations.

• Consumers and other stakeholders who would receive information and public awareness training to guard against HPAI.

1.5 Original Components (as approved) The project had three components, with a total value of Euro 37.5 million, out of which the Bank was expected to finance Euro 29.6 million and the Government to contribute Euro 7.9 million. COMPONENT 1: ANIMAL HEALTH (total cost: Euro 9.15 million). It included nine sub-components:

14

(i) Strengthening the Institutional Framework (Euro 1.148 million). Key areas of support under this sub-component were: (a) evaluation of National Veterinary Services, and (b) Strengthening Veterinary and Food Safety Institutions. (ii) Improving HPAI Prevention and Control Planning (Euro 0.355 million). The sub-component would finance technical assistance for supporting the process of preparing the draft contingency plans (produced by the Government) and developing necessary detailed implementation guidelines and manual for relevant agencies and services. (iii) Strengthening Monitoring and Surveillance Systems (Euro 0.188 million). The key activities included collection of samples around outbreak areas, as well as routine investigation of dead wild birds. (iv) Strengthening Diagnosis Capacity (Euro 8.142 million). This sub-component was to support the existing veterinary laboratories for the increased work load under active surveillance scheme, through purchasing of laboratory equipment and consumables as well as technical assistance and training aimed at increasing the existing capacity for virology tests, screening surveys for confirmation of the disease and differentiation of AI sub-types. (v) Strengthening Animal Disease Information Systems (Euro 2.236 million). The sub-component was to support development of an integrated national animal disease information system to enable the country participate in global disease information and comply with its obligations as a member of OIE (The World Organization for Animal Health). (vi) Strengthening Applied Veterinary Research Capacity (Euro 0.310 million) The activities under the sub-component included technical assistance to support ANSVSA to design the research program on epidemiology, bird migration, and ecology as well as assist with the construction and refurbishment plans for the physical facilities of the Research Center. (vii) Targeting Virus Eradication at the Source (Euro 1.475 million). Activities under the sub-component included purchasing and operations of containers where CO2 was applied in an acceptable manner, as well as purchasing and operation of mobile incinerators. (viii) Supporting Poultry Vaccination Campaign (Euro 0.103 million). This sub component included technical assistance and training for developing the country’s vaccination policy and implementation manuals, as well as training for Government officials to strengthen their capacity for vaccination campaigns and post-vaccination monitoring. (ix) Improving Bio-security in Poultry Production and Trade (Euro 0.293 million). Activities included: (a) supporting awareness campaigns and knowledge transfer activities; and (b) providing technical assistance for a comprehensive poultry sector assessment. COMPONENT 2: HUMAN HEALTH (total cost: Euro 19.8 million). It was composed of three main sub-components, as follows: (i) Enhancing Public Health Program Planning and Coordination (Euro 0.160 million) through providing technical assistance and funding for research, conferences, and workshops. (ii) Strengthening of National Public Health Surveillance Systems (Euro 1.267 million). Activities included: (a) minor civil works and supply of additional security devices, aimed at upgrading the National Influenza Laboratory to BSL 3; and (b) supply of critical equipment and staff training, for strengthening the regional public health laboratories.

15

(iii) Strengthening Health System Response Capacity (Euro 18.297 million). Activities included: (a) purchasing of equipment for intensive care units in the two main infectious diseases hospitals in Bucharest, establishment of isolation and intensive care units in seven regional centers for infectious diseases (namely: Brasov, Craiova, Cluj, Iasi, Timisoara, and Targu Mures); and (b) strengthening the capacity at the CI, aimed at increasing production of vaccines to up to 25 million doses, in compliance with the required Good Manufacturing Practice Standards (GMP).

COMPONENT 3: PUBLIC AWARENESS, COMMUNICATIONS, AND IMPLEMENTATION SUPPORT. The component was to support two main areas, as follows: (i) Public Awareness and Communications (Euro 1.641 million). The activities were to support: (a) technical assistance and training for the communication staff of ANSVSA (National Sanitary Veterinary and Food Safety Authority); (b) development of the integrated strategy and related action plan for human and animal health, and (c) delivery of a number of information campaigns aiming at obtaining public support as well as wide participation in the reporting and monitoring process. (ii) Implementation Support, Monitoring and Evaluation (Euro 1.824 million). The project was to finance: (a) incremental costs of the two Project Management Units (PMUs) in terms of operating costs and staff; (b) studies and manuals; and (c) surveys for monitoring and evaluation purposes at the Ministry of Health4

1.6 Revised Components

(MOH) and ANSVSA.

At the time of project restructuring in July 2009, the Animal Health component and related management and communication activities from Component 3 were cancelled. See Section 1.7 for details.

1.7 Other Significant Changes (i) Partial Suspension of Disbursement. Due to ANSVSA’s inability to effectively manage the implementation of the Project, there were significant delays in implementation. In addition, there were a number of unresolved governance issues related to some procurement processes. Thus, in November 2008, at the time of the Mid-Term Review, the Bank partially suspended disbursements of the loan as it related to activities managed by ANSVSA, with the exception of ongoing salary contracts for PMU staff and for some lots for which contracts had already been awarded.

(ii) Project Restructuring. The project was restructured in July 2009, which included the:

• removal of the Animal Health component and related management and communication activities from Component 3;

• reallocation of part of the freed resources from the Animal Health to the Human Health component;

4 Ministry of Public Health (MOH) was the previous name of the Ministry of Health (MOH).

16

• revising the Project Development Objective (PDO) and results framework in that objectives and results related to animal health activities were deleted from the relevant sections; and

• removal of some other indicators and amendment of others to provide greater specificity or clarity.

The restructuring was undertaken pursuant to a request by the Borrower in March 2009, for reallocation and partial cancellation of the Loan proceeds. Pursuant to this request, the Bank cancelled 7.32 million Euro of the Loan in April 2009, leaving 3.9 million Euro available from the ANSVSA-managed activities for reallocation to the Human Health component as requested by the Borrower. This amount of 3.9 million Euro was allocated to support the upgrading of the vaccine production capacity at the CI, which, in light of the emergence of H1N1 at that time, was even more crucial. The request by the Borrower was made against the following background: • First, ANSVSA, the institution responsible for the implementation of the Animal Health

component, had proven unable to effectively manage the implementation of the Project in accordance with the provisions of the Loan Agreement and the Project Operational Manual, resulting in significant implementation delays that made the achievement of the (related part of the) PDO unlikely. ANSVSA-managed parts of the project were placed under suspension in November 2008, and the corrective action required for the Bank to consider the lifting of the suspension was not taken by the set deadline of March 31, 2009.

• Second, the MOH – responsible for the implementation of Component 2 of the project – had requested additional resources to cover a cost overrun in actual bids and to cover expenditures for additional items not foreseen at appraisal, which were needed to complete the agreed activities under that component.

(iii) Project Schedule. As part of the restructuring, the closing date of the loan was extended for 12 months from December 30, 2009 to December 30, 2010 (Government request of March 2009). The exact timing of the renovation of the CI vaccine production line was difficult to anticipate. The Institute was then producing the H1N1 vaccines for the MOH and neighboring countries had indicated that they would be interested in procuring vaccines from Romania. Production was likely to continue until March or April 2010, and only when this was done could the civil works start.

In September 2010, the Government requested a second extension of the loan closing date. The extension was meant to allow the completion of the remaining civil works at the CI. However, in late December 2010, the Bank reached a decision not to extend the closing date because project implementation was not satisfactory.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry Project Background. The Project was prepared and approved in response to the emerging threat of Avian Influenza and other zoonoses worldwide and acute outbreaks of Avian Influenza in Romania. Recognizing the need to act effectively and fast, the Project was financed through an Emergency Recovery Loan (ERL) under the Global Program for Avian Influenza Control and Human Pandemic Preparedness and Response (GPAI). Under the GPAI, individual countries could obtain separate loans (and credits and grants, depending on the specific country case) to

17

finance national programs. The GPAI was endorsed with a comprehensive list of eligible activities and set out the criteria for participation in this global program. The Bank's emergency policy and business procedures, OP/BP 8.50 was applicable to this Project; and it was financed as a new and self-standing operation on standard IBRD terms. The project was initially working jointly on animal and human health under the leadership of the Sustainable Development (SD) staff with a Human Development (HD) staff member managing the Human Health component. The SD Task Team Leader (TTL) managed the restructuring of the project, which included canceling the Animal Health component of the project. Subsequently, the team leadership was naturally moved to the only sector still involved (HD-Health). The HD team member who had been involved in the project from the beginning remained as a team member for the entire duration of the project, and consequently, the transition was smooth. Lessons of Earlier Operations Taken Into Account. The following general lessons learned from some of the emergency projects supported by the Bank in various regions were listed in the PAD as being of relevance to the design of the project: (i) emergency projects should avoid policy conditionality; (ii) project design must be simple and take into account a realistic assessment of the existing capacity of the Borrower and other stakeholders, e.g., Non-Governmental Organizations (NGOs); (iii) a speedy appraisal and approval are crucial to provide a prompt response and a substantial contribution to project success; (iv) procurement arrangements should be finalized at an early stage but need to remain flexible; (v) mitigation and prevention measures should be included in the design to minimize impacts of a possible recurrence of the disaster; and (vi) realistic assessments should be made of the counterparts’ absorptive capacity, as well as of the effectiveness of communications and coordination mechanisms among all relevant stakeholders. Some of these were reflected in the project design (including i, iii, and v), while others (ii, iv, and vi) unfortunately were not and ultimately impacted negatively on the implementation and results of the project. Adequacy of Government Commitment. At the outset, the Government’s commitment and the existence of appropriate structures and mechanisms for the fight against HPAI were evident, inter alia, in the establishment of a National Disease Command Center, chaired by the Prime Minister; the existence of effective control measures demonstrated during the first wave of outbreaks in October 2005 through April 2006; the rapid detection of the second wave of outbreaks and efforts to detect the source of infection; and the absence of human infections. The Government had demonstrated full commitment in fighting AI and had shown good practice and efficiency in containing the disease and closing all outbreaks during both the first and the second AI wave, both prior to project effectiveness. The level of transparency was also outstanding, and the information to the public was beneficial in avoiding any human case transmissions. The national level institutional structures and command lines were established and Romania had planned for improved preparedness and response capacity. The Government had set up emergency response structures at the national and local levels for the management of HPAI control. The Anti-Epizootic Command Center at the national level and the animal disease command centers at the county and municipality levels formed the backbone of the HPAI emergency institutional structure, activated in times of disease outbreak. As such, the Bank team assessed commitment to be high and designed a comprehensive set of activities under the project.

18

2.2 Implementation The project was approved on September 8, 2006. The Loan Agreement was signed on October 5, 2006, and the original effectiveness date was agreed for February 2, 2007. As a result of delays in Parliament’s sessions, an extension of the effectiveness deadline was granted, and the project became effective on March 19, 2007. There were delays in starting the implementation of project activities (even after eight months of effectiveness, the disbursement ratio was only 0.9 percent) due to inefficiencies in institutional decision making, mainly from the ANSVA side. Once project activities got underway, the following factors affected implementation:

(i) H1N1 crisis. The first case of H1N1 (swine flu) was detected in Romania in October 2009. Between October 2009 and January 2010, the country had a total of 7,000 laboratory confirmed cases of H1N1, including 500 pregnant women. The outbreak of H1N1 distracted the Government from its earlier focus on avian influenza as it understandably gave higher priority in responding to the more immediate H1N1 pandemic. (ii) ANSVSA's institutional weaknesses. When the project was prepared and negotiated, the Bank had a very serious and committed partner in ANSVSA. It had many staff members experienced in the implementation of Bank projects, which included the ongoing Modernizing the Agricultural Knowledge and Information System (MAKIS), Agricultural Pollution Control Project (APCP), Rural Finance Project (RFP), Forestry Development Project (FDP), Rural Development Project (RDP), Irrigation Rehabilitation Project, and Hazard Risk Mitigation and Emergency Preparedness Project. However, immediately after the project was approved and signed, ANSVSA’s president was changed and the new one brought in a new management team, which was much less interested in the Avian Flu project. The President was changed several times throughout the project implementation, and this triggered changes at lower levels as well. Furthermore, problems between ANSVSA and the Project Management Unit (PMU), including the lack of delegation of authority and competences to the PMU, very centralized decision making processes, limited collaboration between the technical departments of ANSVSA and the PMU, weak procurement capacity, and cumbersome internal procedures in ANSVSA all contributed to significant implementation challenges in the Animal Health component. (iii) MOH’s institutional weaknesses. The Human Health component also suffered from institutional, coordination and managerial weaknesses at the level of MOH and of its institutes and agencies (CI and Matei Bals Hospital), and as a result, progress on the Human Health activities was also very slow at first. The change of the State Secretary in charge of the component in 2008 also created some delays in the project activities. The positions of PMU Director, assistant manager, procurement expert, and M&E expert remained vacant from 2009 until the end of the project. As a result, the PMU Technical Coordinator functioned as the de-facto Director from that time until the end of the project and had to manage multiple responsibilities. The Financial Manager also resigned in late 2010. Attempts to hire a new person for this position failed, and the FM functions were subsequently carried out by the Accountant for the last few months of project implementation.

19

(iv) Problems at CI. The problems at CI included the following:

(a) Vaccine Production Unit.

• Since the CI produces the stock of the annual seasonal influenza and other vaccines, and this vaccine must be produced every year5, rehabilitation works6

• Further aggravating this situation was the suspension of the CI’s accreditation for vaccine production by the National Drug Agency in April 2010. A temporary accreditation was eventually provided in May 2010, after some remedial actions were taken. Since vaccine production for that year had not been completed at the time of the suspension, this further delayed the civil works, which at the time were anticipated to begin in the summer of 2010. However production of seasonal influenza vaccine was only completed in October 2010, by which time, the project closing date was less than three months away. As a result, the rehabilitation works could not start under the project.

(estimated to last 4 to 6 months) could only be carried out once production had stopped. However, the initiation of the civil works was dependent on the availability of completed technical designs, which in turn were delayed due to the failure to resolve a misunderstanding with the firm contracted to prepare the technical designs for the vaccine production line. The firm completed approximately 90 percent of the designs, but the remaining part was needed, as otherwise none of the works could start at all even with the 90 percent of that done.

(b) BSL 3 Upgrade.

With regard to civil works for upgrading of the National Influenza Laboratory (NIL) to Bio Safety Level 3, the contract with the construction company was signed on October 12, 2009, and the works started soon after. However, construction was suspended in December 2009, due to the lack of Government permits for the construction. A building permit was finally obtained on May 6, 2010, and the Government approved the investment. However, by then, the contracts signed by the CI with the site supervisors expired, as initially, construction was expected to be completed by April 2010. As the CI no longer had the budget to support this, the MOH PMU had to contract a new site supervisor to monitor the three sites for the civil works, resulting in a cumulative delay in implementation of about six months.

5 The production of the bulk influenza vaccine begins after the WHO announces the composition of the vaccine (the virus strains). This happens usually around mid-February and takes normally six months (i.e., until mid-August). The filling of this bulk product into ampoules is done in July-August. 6 The works were in two parts: one for the vaccine production area which was affected by the issue above (i.e., incomplete designs). Second part was rehabilitation of the vaccine filling area; which required vaccine production to stop or be finished for that year in order for the civil works to be done. Both part 1 and part 2 works needed vaccine production to not be in process for works to start, but part 1 could not start anyway without the designs.

20

(v) Lack of intra-agency coordination. The absence of an effective Steering Committee was one of the critical bottlenecks (and non-compliance issues), which prevented effective cross-sectoral coordination and collaboration in implementation of the project activities.

(vi) Lack of Government commitment. During the life of the project, as the avian flu crisis subsided and the H1N1 (swine flu) virus emerged, commitment from the Government to this project somewhat waned, as it did in many countries. Further, Romania became an EU member on January 1, 2007, and stated its intention to work more closely with the European Commission and less with the Bank and the IMF, thus redefining the relationship between the Bank and the Government – a fact that affected the smooth implementation of this project and indeed the entire Romania portfolio. (vii) 2010 Budget and priorities. Due to the economic crisis and the pressure it put on public finances in Romania, the share of the 2010 budget earmarked for projects funded by international organizations was lower than the amount initially requested by the PMUs. The budget for this Project was short by Euro 3.4 million. As a consequence, funding for the civil works needed for the rehabilitation of the vaccine production unit at the CI was not available for that year. The Bank conducted a Midterm Review (MTR) in November 2008, jointly with the implementing agencies and the Ministry of Economy and Finance (MEF). The objective of the MTR was: (i) to assess progress made in meeting the project’s objectives; (ii) to select activities under the Animal Health (ANSVSA) component that could still be implemented in the remaining lifetime of the project; (iii) to agree on the use of project resources that would become available after dropping Animal Health activities that could not be implemented before loan closing; (iv) to clarify the role of the Bank in the financing of the construction of a new building for the disease control center proposed by ANSVSA; and (v) to consider the reallocation of some project resources from the Animal Health to the Human Health component, in order to provide for adequate funding for agreed activities. The MTR mission also discussed in great detail the implications of the suspension of the loan related to non-fulfillment of most of the activities under the responsibility of ANSVSA and the actions by ANSVSA required for the Bank to consider the lifting of the suspension.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

M&E design. The initial results framework tried to measure all possible aspects of the project activities under Animal Health and Human Health components. As a result, there were too many indicators which were difficult to monitor. Targets were set for these indicators, but they were broad and, as a result, it was difficult to quantify the progress made in comparison to the baseline. At the time of restructuring, those indicators which related to animal health activities or which were difficult to monitor were dropped. The task team also tried to introduce some quantitative elements. For example, a quantitative target was introduced on the vaccine production prior to the equipment and after equipment installation to show that functioning did improve.

M&E implementation. The indicators were used to monitor progress on implementation but by virtue of the fact that they were process-oriented, could not have shown what results were being achieved. The restructuring of the project dropped indicators which had become irrelevant due to the new nature of the project or were difficult to monitor, but an opportunity to revisit the results framework was missed and the new shorter version was approved by the Board. Soon after, an in-depth analysis by the new task team revealed the shortcomings of the indicators, but at that point, given that the project was about to close and had just been restructured, the team, in

21

consultation with management, decided not to undertake a formal restructuring for the purpose of improving the results framework.

M&E utilization. Project data was used to restructure the project. For example, the task team monitored the implementation progress, and the activities which were not making satisfactory progress were cancelled at the time of restructuring. 2.4 Safeguard and Fiduciary Compliance Safeguard issues. All works financed by the project were implemented in accordance with the health and safety requirements, and in compliance with the local and Bank’s environmental safeguards. Fiduciary issues. The project complied with fiduciary covenants during implementation. Internal control arrangements were in place, and adequate financial management system and records were maintained. The audit opinions were clean and no significant internal control issues were mentioned. 2.5 Post-completion Operation/Next Phase (i) The rehabilitation of the vaccine production unit and rehabilitation of the vaccine filling station, which could not be completed before the project closing date of December 31, 2010, have been transferred to the ongoing Bank-financed Health Sector Reform Project – Phase II. It is expected that these activities would be completed before the closing of this project on December 15, 2011. (ii) Bio Safety Level 3. Investments made on rehabilitation and equipment are in place and they are fully functional. The completion of the civil works to upgrade the National Influenza Lab to a BSL 3 was one of the main activities at the CI that was completed before the project ended. The remaining activity that was not done was the BSL 3 certification by an external agency. The CI had already identified in early 2010 three relevant agencies in Europe that could perform the certification activities for them, and the CI management has given assurances that they will be doing this task. Since this is the one remaining step to enable the lab to function7

, it is highly likely that this job will be done by the CI.

(iii) CI’s GMP license. The CI was granted a temporary GMP license in May 2010, after taking the requisite remedial action, which included the installation of a separate water and purification system. Vaccine production resumed that same month 8

7 The rehabilitation of the lab to make it BSL 3 only finished at the end of December 2010. So there was nothing to be certified prior to that. The certification can now be done this year. The lab consists of many parts, which are already functioning and are certified. What is required now is for the certification to be extended to this upgraded part- the BSL 3.

. Further, the Manufacturing

8 The CI has many parts; one is the National Influenza laboratory, which does testing, etc. The other part is the vaccine production facility, which is the part that had its GMP license temporarily revoked. They have since installed the water purification system for the vaccine production and it is functioning, and the national drug agency renewed their GMP earlier this year. GMP license is awarded for three years, but each year, an inspection is made to ensure that certain basic things are in compliance, so next year, they will get another inspection.

22

Authorization9

issued by the National Regulatory Authority (the National Medicines Agency) is renewed yearly, based on a follow-up inspection. The last such follow-up inspection was in February 2011 and the Authorization was renewed without objection.

(iv) Follow-on project: No follow-on project is anticipated nor recommended.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation Moderately Satisfactory. The project objective was clear, relevant, and important to Romania’s public health. It is timely and appropriate to the needs of the country's health sector at that time. One of the lessons identified in the PAD as important for emergency projects was to keep a simple design. However, the task team designed a project with multiple sub-components, many of which were not implemented. The strong commitment of the counterparts at the outset and the relatively strong capacity in the implementing agencies may have given assurances to the task team that a very comprehensive project would be achievable. The design of the project featured a combination of civil works, capacity building, and communications. The civil works components in turn relied on Government permits, technical design companies, and, in the case of vaccine production, on licensing process; which all incurred significant delays. Most of the reasons were beyond the project’s control. At the end of the project, most of the planned civil works had not been undertaken (with the exception of works related to BSL3 labs, which was completed successfully). The capacity building elements of the project fared somewhat better. Contingency plans for avian flu pandemic were prepared and tested in Tulcea County. Meanwhile, the communications aspects of the project were not undertaken, but the Government was successful in managing communication by itself.

Implementation arrangements were relatively straightforward, with a single agency responsible for each of the two main components – Animal Health and Human Health. Both PMUs had managed other World Bank projects, but many problems arose between the PMU and the technical departments in ANSVSA, as well as between ANSVSA’s legal and technical departments, and between these departments and the World Bank. Furthermore, there was a high turnover of staff in ANSVSA PMU and the positions remained unfilled for a long time.

3.2 Achievement of Project Development Objectives Moderately Unsatisfactory. The project was moderately successful in achieving its objectives in relation to the human health activities. As a result of the project, Romania’s human health services are currently in a better shape to deal with outbreaks of infectious diseases, such as Avian Influenza and H1N1. Investments made under the project have facilitated the strengthening of surveillance, diagnostics, control, and treatment capacities for responding to

9 GMP is for the vaccine production, the accreditation of the lab has nothing to do with the GMP or the vaccine production.

23

pandemic diseases in Romania. In brief, the project objective contributed to strengthening the capacity of the country to respond to pandemic diseases and to treat patients under safer conditions. On the animal health side, however, very little was achieved. For example: (i) Mobility, institutional structure, and skill level of local veterinary staff did not improve, and activities on these aspects were dropped. For example, for strengthening epidemiological capacity, the recruitment of the epidemiological expert was not completed. (ii) County laboratories could not provide timely disease diagnoses, as equipment and reagents were not procured. (iii) Comprehensive/integrated animal disease information system was not established and activity was dropped. (iv) Quarantine, culling, and disposal measures could not be improved, as procurement of CO2 containers and incinerators did not even start. (v) No activities for awareness campaigns and knowledge transfer activities were carried out to improve behavioral and managerial changes towards higher bio-security in small-holder poultry systems. Although the two project development indicators, as revised under the 2009 restructuring, were met, they do not reflect well the full PDO, and clearly, the project fell far short of the intended results. Hence, the overall rating for achievement of the original project development objectives is rated as Moderately Unsatisfactory. I. Project Outcomes Before Restructuring The outcome of the project on the Animal Health side was unsatisfactory, as there were only very modest achievements as shown below: • Appropriate contingency plans for avian influenza and human influenza pandemic were

prepared and improved. Coordination mechanisms were defined, and legislation was revised. Capacity to implement the contingency plan was demonstrated in the district of Tulcea in November 2007.

As a result of these partial achievements, the related project development objective 1 (prior to restructuring) could not be met and was removed during project restructuring. In addition, at the time of restructuring, the project had disbursed only 8.2 percent of the total funding. On the Human Health side, the achievement was moderately satisfactory in light of its major progress on the following two aspects: • Rehabilitation/refurbishments (small civil works) to facilitate new equipment. Using the

MOH allocations for two national and seven regional infectious disease hospitals, reinforcement works were finalized in most of the hospitals (Craiova, Iasi, Constanta, Bucuresti – Babes Hospital and Bals Institute, Timisoara); and the other hospital was in the final stages of rehabilitation.

• Training. Training was implemented according to the agreed plan. The CI staff received

training on Good Manufacturing Practice (GMP) standards in vaccine production, based on four modules, with five technical staff from CI completing the necessary training modules.

24

On the whole, the project’s outcome before restructuring was Moderately Unsatisfactory. II. Project Outcomes After Restructuring After the project was restructured, there were significant achievements as demonstrated by the following:

• According to the National Influenza Laboratory, there were no confirmed human cases of H5N1 since 2007 and until the end of the project.

• Upgrading of the NIL to BSL-3. The certification of the NIL as a BSL-3 facility will be

undertaken in 2011 and financed by the CI. According to EU and WHO standards, the certification could be granted after an inspection of the facility by an independent internationally accredited laboratory. This upgrade will allow CI to perform more complex tests than they previously were able to. Even though the NIL has yet to be certified, the upgrade supported by the project is expected to be sufficient to ensure that this certification is granted.

• Strengthening Health System Response Capacity The health system’s capacity to respond

to avian and other pandemic influenza has been significantly improved through the provision of equipment to the two main infectious diseases hospitals in Bucharest and seven regional hospitals and training of personnel in the hospitals. Equipment provided included ventilation machines and monitors for intensive care units (ICUs), furniture and intensive therapy accessories, clinical examination and treatment equipment, and laboratory and mobile examination equipment (including portable x-ray machines).

• Upgrading of the influenza vaccine production unit at the CI. The purchase of new

equipment has boosted vaccine production at the CI two-fold from 30 doses of trivalent influenza vaccine per week in 2007-2008 to 79 doses per week in 2008-2009. The CI can now produce up to 8,000 vaccine doses per month.

• National contingency plan is in place since 2009, and implemented during H1N1 outbreak.

However, improvements after project restructuring could not cover for all previous shortcomings. Some main activities that could not be completed by the project’s end were cancelled. In terms of financing, the total funds cancelled reached to Euro 11,631,799 (out of which Euro 7,317,233 during project restructuring and Euro 4,314,566 at project closure). The uncompleted activities, as given below, have been transferred to the ongoing Bank-financed Health Sector Reform Project-Phase II: a) Rehabilitation works for the influenza vaccine production unit at the Cantacuzino Institute

(Euro 1.1 million); b) Rehabilitation of the aseptic ampoule sealing and filling station (Euro 1.8 million); c) Modification of the filling line (Euro 0.85 million); d) Site Supervisor for the rehabilitation works (vaccine production unit and sealing and filling

station) (Euro 0.01 million); and e) Designs for vaccine production upgrade at the Cantacuzino Institute (Euro 0.06 million).

Based on the above, the project’s performance after restructuring was Moderately Satisfactory.

25

3.3 Efficiency Project benefits are primarily driven by key assumptions regarding the gross attack rate for the human population and for the mortality rate among those affected as well as by the likely frequency of HPAI outbreaks. This is to be expected in a project that is intended to prepare for a contingency for which neither timing nor severity can be reliably predicted. Nevertheless, the overall benefits from the project in terms of human health sector benefits, decrease in human suffering and deaths alone are significant and can be argued to justify this and other similar projects. The analysis further shows that even in the absence of human infection, as has happened in the past in Romania, benefits from reduction in infection rate in the poultry could generate benefits that make the project worthwhile. The benefits from increased capacity of Veterinary and Animal health systems, which may accrue even in the absence of an outbreak, are additional benefits of the project that are not considered in this analysis. It may be noted that analysis do not consider the additional benefits like costs averted in repopulating the poultry stock, effects on prices, losses from reduced consumption after an outbreak and impacts on foreign trade, if any. The inclusion of these benefits will further improve the returns on the investment in the project. Human health benefits substantially outweigh those in the poultry production sector (detailed analyses in attached Annex 3). The two parameters that drive the economic value of the human health benefits are (i) the project impact on the percentage of the human population affected by HPAI infection, and (ii) the project effectiveness in reducing the expected mortality rate from HPAI infection. Note that even with parameters indicating relatively small improvements in the percentage of the human population affected by HPAI and in the number of human deaths attributable to HPAI, the returns remain robust. Changes in the key parameters for the poultry industry have less impact on the overall outcomes.

3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory

The project has a number of satisfactory achievements to its credit. For example: (i) Appropriate contingency plans for Avian Influenza and human influenza pandemic were prepared and improved. Coordination mechanisms were defined, and legislation was revised. (ii) There were no confirmed human cases of H5N1 since 2007 and until the end of the project. (iii) The CI staff received training on Good Manufacturing Practice (GMP) standards in vaccine production. (iv) Upgrading of the NIL to BSL 3 will allow CI to perform more complex tests soon after its certification. (v) The health system’s capacity to respond to avian and other pandemic influenza has been significantly improved through the provision of equipment to the two main infectious diseases hospitals in Bucharest and seven regional hospitals and training of personnel in the hospitals. (vi) The CI can now produce up to 8,000 vaccine doses per month.

However, as already mentioned above, a number of project activities were cancelled or transferred, thus resulting in a very partial achievement of the original project outcomes. Therefore, the overall rating for the original project outcomes is Moderately Unsatisfactory.

26

3.5 Overarching Themes, Other Outcomes, and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development Poverty Impacts and Gender Aspects. There were no specific, recorded impacts on poverty and gender. Social Development. Even though there is no counter-factual data to prove the impact, anecdotal evidence shows that the project has benefited the Romanian population in terms of protecting them from pandemic diseases. (b) Institutional Change/Strengthening The project resulted in a substantial institutional development impact as demonstrated by the following:

(i) The Project has strengthened the capacity of employees of CI by providing GMP training

for those involved in vaccine production. There were four GMP training sessions in 2008, which benefitted five employees of the Institute. The courses were organized by the European Compliance Academy (the leading European association with regard to GMP and regulatory compliance). In addition, there was on-site training provided by the suppliers of process equipment, for operation and maintenance of the equipment.

(ii) Investments made at two national and seven regional infectious diseases hospitals and training given to hospital personnel have enabled these hospital to respond to many infectious diseases, including AI, more effectively and efficiently.

(iii) The project has strengthened the capacity of the MOH PMU staff in financial management, procurement, environmental management, and many other aspects of project management. For example: the financial coordinator and the accountant expert had a one-week training in financial management at the Bank’s country office in Bucharest and a one-week training in project management organized by the International Labor Organization (ILO) in Turin (Italy). The procurement expert has attended a three-week training in procurement organized by the ILO in Turin.

(c) Other Unintended Outcomes and Impacts (positive or negative) None

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops N/A

4. Assessment of Risk to Development Outcome Ratings: Moderate Production of vaccines at the CI’s National Vaccine Center appears to be sustainable. Incomplete activities that have been transferred to the Health Sector Reform Project should be completed before project closing on December 15, 2011. At that time, the CI will be in a better position to keep up with the production at a higher level. This will enable them to meet routine national

27

vaccine needs as well as increased demands for vaccines during crisis periods (e.g., influenza outbreaks) for the country as well as neighboring countries.

However, because of the country’s economic problems, currently, CI is financially in a very difficult situation. Although CI is state-owned, it does not receive any financing from the government budget, and is self-financed, mainly from the revenue generated by selling vaccines and other products and services, such as laboratory services and research projects. However, given that the CI is Romania’s only such institute, chances of its complete closure are low, with the Government’s financial support. The same can be said for the contingency plans, which were prepared by the Ministry of Health. Since there is no immediate threat of influenza pandemic to the country, the contingency plans are not being used for the moment. However, the plans are published in the MOH website and are publicly available.

In case the budget situation deteriorates further because of the country’s economic problems, maintenance of equipment at project-financed infectious diseases hospitals could be adversely affected. The current freeze on public sector staff recruitment, which may remain in place for the foreseeable future, could affect provision of services to the population.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory. There was no official assessment of the quality at entry by the Quality Assurance Group (QAG). The ICR deems the quality at entry to be moderately satisfactory. Project preparation was carried out with an adequate number of specialists from two sector units who had the required technical skill mix necessary to guide project design. The Bank provided adequate resources in terms of staff weeks and budget to ensure quality preparation and appraisal work. The Project was consistent with the CAS and government priorities in the sector at the time. It is also worth pointing out that the Bank prepared the project at a difficult time, in a very short period of time due to the emergency nature of the project. To this extent, the Bank performance was satisfactory. On the other hand, the project design had the following two major shortcomings: • Considering the emergency nature of the project, many of the project activities could have

been avoided and the project design simplified. • The M&E framework was weak. It had some indicators that were difficult to monitor (many

of which were dropped after the restructuring). Targets were set for these indicators, but they were broad and as a result, it was difficult to quantify progress.

Considering the above factors, the overall quality at entry is rated Moderately Satisfactory.

28

(b) Quality of Supervision Rating: Satisfactory. The Bank's performance during the implementation of the project was satisfactory. Sufficient budget and staff resources were allocated, and the project was adequately and closely monitored. The task team prepared Aide-Memoires systematically and alerted the Government and the PMUs to problems with project execution and tried its best to facilitate remedies in a timely manner, in conformity with Bank procedures. The Implementation Status Reports (ISRs) generally realistically rated the performance of the project both in terms of achievement of development objectives and project implementation. The task team carried out a Mid-Term Review in November 2008. The task team also monitored safeguard and fiduciary compliances. The team worked closely with the project management teams in both the ANSVSA and the MOH and held meetings with representatives of ANSVSA, the CI, and the Matei Balş Hospital. The team also made site visits including to the Institute for Diagnosis and Animal Health (IDAH), and undertook consultations with the EU's Resident Twinning Advisor in ANSVSA. The Bank was proactive in terms of taking appropriate remedial measures whenever required. For example, in November 2008, the Bank imposed partial suspension of disbursements of the loan as it related to activities managed by ANSVSA due to operational and institutional non-compliance issues. Likewise, the task team initiated restructuring process when it became obvious that it was essential for achieving the project objective. The restructuring narrowed the scope of activities and facilitated achievement of many positive and creditable results. As mentioned in Section 2.1, when the TTL was from Sustainable Development, a senior staff member from Human Development was involved with the implementation right from the beginning, and they worked together closely and effectively. When the TTL-ship changed from the Sustainable Development to Human Development after MTR, detailed hand-over discussions were conducted between these two TTLs and the transition was thus made smooth. Even though the TTL had changed three times over a period of four years of the project, the task team remained more or less the same, and this helped in mitigating the frequent turnover of TTLs. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory Based on the Bank performance during the preparation phase and supervision as discussed above, overall Bank performance is rated as Moderately Satisfactory.

5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory As mentioned in Section 2.1, the Government had shown strong commitment to the objective of the project at the time of project preparation. However, this initial commitment somewhat faded during the course of implementation for the reasons mentioned in Section 2.2. For example, there were some leadership issues in CI until towards the end of the project. Based on the overall factors, the Government performance is rated as Moderately Satisfactory.

29

(b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory MOH was committed to the project. However, frequent change in leadership at the Minister’s level affected the implementation. For some period of time in 2009, there was no Minister of Health. Further, the overall economic situation in Romania adversely affected implementation. For example, the public sector budget freeze in late 2009 affected payment of salary to PMU staff, causing some staff to leave their positions. There was adequate commitment to the project on the part of PMU, which tried to manage the challenges during implementation including working without a Director and other technical staff. For example, the technical coordinator, despite having her own tasks to manage, also ended up filling in as the Director for more than a year and a half.

Internal Controls. The PMU had adequate internal controls for the project, including regular reconciliation of bank accounts, adequate segregation of duties and monthly reconciliation of disbursement summaries of the World Bank with project accounting records. The PMU used a comprehensive set of accounting policies and internal control procedures in accordance with the Romanian legislation and the project financial management manual. Financial Management. The semi-annual unaudited financial monitoring reports were received in general on time. With certain follow up to clarify any discrepancies, all the interim financial reports had been accepted by the Bank. All the project audit reports were received and accepted by the Bank, but the reports were submitted to the Bank with some delays due to late selection of the auditors. The audit opinions were clean and no significant internal control issues were mentioned. The project was affected to a certain extent by delays in the provision of timely and adequate counterpart funding and, as such, counterpart financing received a moderately satisfactory rating throughout the life of the project. Procurement Arrangements. Procurement was significantly behind schedule and the project suffered the cumulative consequences of poor planning and management. The most salient problems related to the upgrading of the vaccine production line. The design of the vaccine production line was delayed due to a problem with the technical design firm that failed to be resolved despite several attempts by the PMU to reach some consensus.

Reporting Arrangements. All records under the project are kept by the PMU in its existing accounting software (CIEL). The outputs from the software were used to prepare the semi annually financial monitoring reports, which were aggregated for the whole project by the MOH PMU and thereafter sent to the Bank’s FM team’s for review. The PMU also prepared monthly and quarterly trial balances and other reports as required by the Ministry of Public Finance (MOPF), and MOH. Considering the above factors, the performance of MOH is rated Moderately Satisfactory ANSVSA. There was a lack of commitment to the project on the part of the ANSVSA. Its highly centralized decision-making, and the nomination in April 2007 of a new President and Project Coordinator, had resulted in serious start up delays. Due to the project restructuring and the cancellation of Euro 7.32 million loan funds (ANSVSA components), the PMU was left with only very few contracts after the restructuring.

30

Internal Controls. Several layers of control were in place at ANSVSA, related to the approval of each contract and then of each invoice and payment, however, due to the staffing situation, these controls were not functioning effectively and resulted in significant delays in processing transactions. Financial Management. The PMU had been affected by its internal management weakness and lack of sufficient FM staff and thus had a lower FM capacity. The semi-annual unaudited financial monitoring reports were received with some delays from the PMU components in the early part of project life. All the project audit reports were received and accepted by the Bank, but the reports were submitted to the Bank with some delays due to late selection of the auditors. The audit opinions were clean and no significant internal control issues were mentioned.

Procurement Arrangements. Due to disagreements between ANSVSA's technical and legal departments and the Bank, procurement activities were inordinately delayed. For example, ANSVSA's technical and legal departments were constantly in disagreement on each other’s role during preparation of procurement packages. As none of the legal department representatives was familiar with Bank financed procurement, almost every contract was blocked by the legal department despite the fact that No Objection Letter (NOL) to contract award was issued. The Bank organized a joint meeting with the representatives of both departments in order to bring them together and provide guidance on possible ways of interaction in order to resolve the deadlock. The Bank also invited them for a one-day training for their legal staff, and during the meeting, they accepted our proposal, but later on turned it down by proposing to send only junior staff who did not have decision making authority.

Further, following the change in the PMU Director and resignation of the Procurement Specialists in 2008/09, procurement capacity in the PMU for the ANSVSA-implemented component was considerably limited.

Reporting Arrangements. All records under the project are kept by the PMU in its existing accounting software (SAGA). The outputs from the software were used to prepare the semi annually financial monitoring reports, which were sent to MOH PMU for consolidation for the whole project. The PMU also prepared monthly and quarterly trial balances and other reports as required by the MOPF and ANSVSA.

Considering the above factors, the performance of the ANSVSA is rated Unsatisfactory. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory. In light of the Government, MOH, and ANSVSA performances as discussed above, the overall performance of the Borrower is rated as Moderately Unsatisfactory.

6. Lessons Learned Important factors for the success of a project:

• It is important for the Bank to be proactive and use its various tools (such as suspension of disbursements, re-allocation to faster disbursing sub-components, cancellation, and restructuring) to give a project a chance to go forward more efficiently and smoothly. For example, restructuring of the project enabled it to achieve many positive results.

31

Project Design:

• Even though a project such as an Avian Influenza Project needs to be prepared in a hurry and within a short period of time to respond to an emergency situation, it is essential to see that basics aspects for successful implementation are in place.

Implementation:

• It is necessary to have an idea of the country’s laws and procedures and factoring them into the time framework of a project. Especially, when a project includes civil works, it is critical to decide at the preparation stage a realistic time framework for planning and managing them. This would minimize implementation delays. Also, it is necessary to have a civil works engineer in the project team to monitor the progress of the civil works and resolve implementation issues expeditiously.

Monitoring: • A strong M&E framework needs to be prepared before starting the implementation.

Under this framework, indicators need to be clearly defined, and the targets should be specific, realistic, and achievable during the project’s time line.

Procurement:

• Purchasing equipment for health projects, especially for an input-based project such as an

avian influenza project, is generally complicated and time consuming, and a lot of efforts will be required to manage these types of projects. For these reasons, sufficient time should be allowed for completing the project activities, with considerations for possible delays in procurement.

32

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies By letter of June 15, 2011, the Government has confirmed that it did not have comments on the draft ICR.

(b) Cofinanciers NA (c) Other partners and stakeholders NA

33

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in EUR Million equivalent)

Components Appraisal Estimate (EUR millions)

Actual/Latest Estimate (EUR

millions)

Percentage of Appraisal

Component 1. Animal Health 0.29 0.22 Filled up

automatically in the System

Component 2. Human Health 20.85 19.06 Component 3. Public Awareness & Communication 1.14 0.71

Total Baseline Cost 22.28 20.53

Physical Contingencies 0.00

0.00

Price Contingencies 0.00

0.00

Total Project Costs 22.28 20.53 Front-end fee PPF 0.00 0.00 Front-end fee IBRD 0.00 0.00

Total Financing Required 22.28 20.53

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(EUR millions)

Actual/Latest Estimate

(EUR millions)

Percentage of Appraisal

Borrower 2.76 2.55 Filled up automatically in the System

International Bank for Reconstruction and Development 22.28 17.98

34

Annex 2. Outputs by Component COMPONENT 1: ANIMAL HEALTH (i) Strengthening Veterinary Institutions: Evaluation of National Veterinary Services. Assessment of Romania’s veterinary services was completed and the report was made available. The assessment concluded that Romania’s veterinary services and crisis response mechanisms perform at high levels, with particular emphasis on veterinary services coverage across the country, national and regional laboratory equipment, epidemiological surveillance, and access to electronic means of communication and access to information. At the same time, the report also highlighted some gaps (such as the mobility of veterinarians in districts) and the notable exception of ANSVSA’s headquarters as not offering appropriate facilities (space). (ii) Strengthening Monitoring and Surveillance Systems (a) Surveillance program. The 2008 National Strategic Program for Surveillance was prepared. (b) Strengthening Animal Diseases Information Systems. Study tours to Germany and the Netherlands were undertaken. (iii) Improving Bio-security in poultry production and trade Activity merged with communications activities. Recognizing that the initiation of behavioral change in backyard poultry systems and markets (the key challenge in increasing bio-security in these systems) is primarily a communications issue and against the background of ANSVSA’s AI communication strategy covering the approach towards backyard poultry owners, bio-security related project activities were covered under “Communications” (Component 3.A). COMPONENT 2: HUMAN HEALTH Enhancing Public Health Program Planning and Coordination

• Public Health Program was prepared in 2006, complying with EU requirements • MoH finalized the national contingency plan in July 2009.

Strengthening of National Public Health Surveillance Systems (i) Upgrading of the National Influenza Laboratory to NBL 3.

• NBL3 Lab was improved to: (i) the threat of high risk avian influenza and other zoonosis; (ii) increase the training capacity, control and prompt response in case of pandemic influenza or other human infectious diseases. Activities included redevelopment, rehabilitation and modernization of the Laboratory area with level 2 biosecurity to upgrade to international level 3 biosecurity requirements.

• The rehabilitation works for NBL 3 were completed in December 2010 (ii) Strengthening of regional laboratories by providing critical equipment and training of staff.

• Cantacuzino Institute representative was trained in US in 2010 for laboratory supervision with the support of the Cantacuzino Institute and WHO.

Strengthening Health System Response Capacity (i) The strengthening of the isolation and case management capacity in both national and

regional hospitals.

35

• All contracts for procurement of equipment for Intensive Care Units were cleared, and signed in 2009/2010. All equipment packages for the infectious disease hospitals were delivered and installed by December 2010.

(ii) The upgrading of the influenza vaccine production unit at the Cantacuzino Institute.

• This activity included the rehabilitation and equipment of the influenza production department, and the completion and equipment of the filling and sealing unit. Cantacuzino Institute now has the capacity to produce major part of the vaccines needed in Romania during the inter-pandemic and pandemic periods.

TA for engineering design and procurement support (vaccine production and filling units).

• Due to the economic crisis MoH could earmark only a lower amount for projects funded by international organizations than the amount initially requested by the PMU. The Ministry of Public Finance could allocate for 2010 only 14.6 million Euro instead of 18 million Euro requested by the PMU. This amount was used to finance on-going contracts and/or contracts which were about to be awarded. Consequently, funding for the civil works needed for the rehabilitation of the vaccine production at the Cantacuzino Institute was not available for 2010.

Equipment

• Equipment for upgrading the inoculation and harvesting machine at CI was delivered and installed in October 2010.

COMPONENT 3: PUBLIC AWARENESS, COMMUNICATIONS, AND IMPLEMENTATION SUPPORT Implementation Support, Monitoring and Evaluation

• MOH and ANSVSA used existing PMUs in these agencies to ensure effective implementation of the project in accordance with Bank fiduciary and procurement policies and guidelines. The project financed: (i) Incremental support to the two PMUs in terms of staff and operating costs; (ii) Studies, technical services, manuals; and (iii) Surveys for monitoring and evaluation.

• Cantacuzino Institute hosted the MoH PMU as an autonomous entity. After the PMU Director resigned in June 2009, Financial Coordinator in July 2010 and the three positions remained vacant from 2008, three members of the PMU (Technical Coordinator, Procurement specialist and Accountant) successfully completed the project

36

Annex 3. Economic and Financial Analysis (including assumptions in the analysis)

1. Project benefits The expected project benefits estimated include human health benefits and poultry sector benefits. Since reliable data were not available, other benefits, such as the benefits from improved capacity to detect and respond to other zoonoses and infectious diseases, the reduction in scale and cost of poultry culling and compensation programs due to better surveillance and control measures are not considered. Secondary effects, such as the prevention of abrupt and severe changes in poultry prices on farm incomes or consumer incomes and of cross-price elasticity effects on other livestock subsectors were also ignored. The human health benefits include cost of human hospitalizations averted and the income losses avoided due to reduced rates of human infection, hospitalization and deaths. The project benefits in the poultry sector considered include the production losses of poultry meat and eggs averted, savings from compensation programs to poultry producers and cost of culling affected poultry in the event of an outbreak. In order to compute the project benefits it is assumed, for the base-case scenario, that a severe HPAI outbreak with a mutated virulent virus would occur every five years, with the first one in 2012, after project completion. The time period for the benefits stream being considered is 2012-2025. A discount rate of 12 percent per annum and an exchange rate of I EUR = 1.42 US$ was used in this analysis. 1.1. Human health benefits The project’s main impact on human health is to decrease the percentage of the human population infected by HPAI, thereby decreasing the number of hospitalizations required and decreasing the number of deaths attributable to HPAI. Hence, the human health benefits are expressed in terms of loss avoidance and include:

(i) The economic value of hospitalization costs averted: Hospitalization costs for an average 7-day stay per hospitalized person for treatment of HPAI infection is up to 2,000 Euros per day, which translates to a 14,000 Euros per 7 days hospitalization for an average stay of 7 days per hospitalized person;

(ii) The economic value of income loss avoidance from days lost due to illness: assumes 25 percent of the affected population would lose 3 days of take-home income and 50 percent of the hospitalized survivors would lose 30 days of income. The income lost due to illness is imputed at the national average income, which is 250 Euros per month 10

(iii) Income loss avoidance from fatal HPAI cases: People that could be infected are between 20 and 40 years old, and life prognosis is between 53 and 33 years assuming a life

;

10 The other 50 percent are assumed to be children and retirement-age adults.

37

expectancy of 73 years. Then the remaining lifetime earnings is assumed to be between 159,000 Euros and 99,000 Euros11

.

In order to conduct the analysis, the incidence of HPAI infection among the human population (Gross Attack Rate) in the base-case scenario without the project was assumed at 2 percent, with 10 percent of those affected requiring hospitalization and 2 percent of those affected dying from the disease (the Without-Project scenario) 12

. The With-Project scenario on the other hand assumes that only 1 percent of the population would be affected by HPAI infection, of which 10 percent of the affected being hospitalized and a death rate of only 1.5 percent among those affected.

1.2. Benefits in the poultry sector

Poultry production is widespread across the country, with the backyard holdings accounting for 70 percent of Romania’s total poultry production. Typical household in Romania comprises chickens, ducks, turkeys, geese, swine, sheep and goats, cows and horses living in an immediate proximity. As regards the commercial poultry production, about 15 large producers of poultry meat account for about 60 percent of the total domestic production. In 2005, the total Romanian consumption of poultry was around 375,000 tons, of which 220,000 tons were provided by the domestic producers (59 percent). If production for self-consumption is also considered, the total domestic output of poultry could be estimated around 360,000 tons. This analysis accounted for the following benefits to the poultry sector in the form of averted expenditures in the event of HPAI outbreak:

1. Value of poultry that needs to be culled 2. Compensation to poultry holders 3. Disinfection, culling, incineration of poultry, etc 4. Production loss during the outbreak control (disinfection and surveillance) period 5. Additional storage costs and disinfection costs for commercial farms

Since data were not available, the effect drop in poultry demand (less sales), falling prices (sales at lower prices), and losses in export earnings from an export ban on the project’s net worth are not considered in this analysis. The analysis followed comparison of the losses and expenditures under project to corresponding values without the project. The costs and losses averted with the project are then used to assess the net worth and rates of return from the project investment. For the present analysis, under the base case scenario without the project, a Gross Attack Rate (poultry population affected by an outbreak) of HPAI was assumed at 3 percent of the commercial and backyard poultry population -a “low infection rate” scenario- with a 100 percent mortality rate of the birds affected by HPAI. In addition, an equal number of birds, in and around the locations where HPAI outbreaks are confirmed, were assumed to be destroyed through culling as a precautionary virus control measure. The with-Project scenario on the other hand assumes that only 1 percent of the commercial farm and household poultry would be affected by HPAI and would die, and that an equal number would be destroyed by the eradication program. Note that the project can only affect the number of poultry that become infected and that need to be culled, but it has no effect of decreasing the mortality rate among the poultry flocks infected. The first

11 The number of persons in this group is assumed to decline by 2 percent annually, reflecting normal mortality rates. 12 Similar World Bank projects used, at appraisal, Gross Attack Rates as high as 30 percent.

38

outbreak is assumed to happen in the first year after the completion of the project. Subsequent outbreaks are assumed to occur once in every five years thereafter. The analysis covers the period from 2007 to 2025. 2. Estimation of costs averted with the project in the event of an outbreak This section estimates the expenditures averted with the project in the event of an outbreak of HPAI. The estimates were based on data on actual expenditures incurred by the GOR to combat the last outbreak. Table 1: Costs that could be averted due the project following an outbreak under base case (Million US $) 1 Disinfection, culling, incineration of poultry, etc. 4.98 2 Compensation to poultry holders 3.46 3 Vaccination of civil population and security forces 1.4 4 Preventive medication with anti-viral drugs (Tamiflu 0.28 5 Sanitary and protection equipment, information materials

related to human health 0.42

Total 10.54 These expenditures will be incurred if and when there is an outbreak and are based on conservative estimates provided by the GOR. There may be additional cost savings with the project, which will increase the project that are not included here. 3. Economic Rate of Return and Sensitivity Analysis 3.1. Base case scenario Although Romania experienced two recent episodes of HPAI, human infections were not reported. Hence, in the base case scenario, two cases are presented in Table 2. The first is the case when there is an outbreak in the poultry sector and there is no human infection. A gross attack rate of 3 percent among the industrial and backyard farms is assumed without the project. The Gross Attack Rate is assumed to fall 1 percent with the project. In the second case, without the project, in addition to 3 percent gross attack rate among the poultry, it is assumed that 2 percent of the population is affected by the epidemic and the mortality rate of the affected human population is 2 percent. With the project the attack rate among the poultry is assumed to fall to 1 percent, and incidence among the human population is assumed to fall to 1 percent of those affected.

Table 2. Economic returns and project worth under the base case scenario* Infection rate

humans (%) Mortality rate humans (%)

Infection rate poultry (%)

2012-2025 period

Scenario Without Project

With Project

Without Project

With Project

Without Project

With Project

NPV (US

million)

ERR(%) B/C ratio

1 outbreak every five years 2012-2025, first outbreak in 2012 1 0 0 0 0 3 1 14.83 28 1.72 2 2 1 2 2 3 1 69.96 86 4.39

*Estimates are based on data provided by the GOR and additional data collected from various sources; ERR: Economic Rates of Return.

39

The rates of return shows that, under the base case scenario, even in the absence of human infection yields rate of return of 28 percent over the period considered. The benefit/cost ratio also showed significant returns on the investment. It may be noted that the rates of return and projects worth increases significantly if there is a disease incidence among the human population. The Economic rate of return and Benefit: Cost ratios were 86 percent and 4.39 percent respectively. These results are in line with the ex-ante estimates of return from the project, as provided in the Project Appraisal Document (PAD). The PAD estimates that a reduction in incidence rates as low as about 10 percent in the first two years and 5 percent in years 3 and 4 would yield an internal rate of return of 12 percent on the project’s investments. The base case scenario in the present analysis considers reduction in infection rate from 3 percent without the project to 1 percent with the project and the resulting ERR of 28 percent without human infection and 86 percent, if there is human infection as in Scenario 2 in Table 2. 3.2. Sensitivity analysis Sensitivity analysis was done with alternative scenarios to examine the effect of changing the different assumptions/parameters on the returns from the project. Four different scenarios are presented to test the sensitivity of the returns to investment in the project to infection rates among poultry and in the humans. The results seem to suggest that in the event of an outbreak, the investment in the project is worth.

Table 3. Economic returns and sensitivity analysis Infection rate

humans Mortality rate

humans Infection rate

poultry 2011-2025 period

Scenario Without Project

With Project

Without Project

With Project

Without Project

With Project

NPV (EUR

million)

ERR B/C ratio

1 outbreak every five years, first outbreak in 2011 1 0 0 0 0 3 2 8.70 1.42 1.42 2 0 0 0 0 5 2 20.61 34 2.0 3 2 1 2 2 3 2 63.84 81 4.09 4 2 1 2 2 5 2 75.74 89 4.67 5 1 0.5 2 2 3 2 56.32 54 3.24

The human health benefits substantially outweigh those in the poultry production sector. The two parameters that drive the economic value of the human health benefits are (i) the project impact on the percentage of the human population affected by HPAI infection, and (ii) the project effectiveness in reducing the expected mortality rate from HPAI infection. Note that even with parameters indicating relatively small improvements in the percentage of the human population affected by HPAI and in the number of human deaths attributable to HPAI, the returns remain robust. Changes in the key parameters for the poultry industry have less impact on the overall outcomes.

40

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending Mathias Grueninger Senior Agriculture Economist ECSS1 Vladimir Savic Consultant ECSSD Andriy Rozstalnyy AI Coordinator FAO Francois Decaillet Lead Public Health Specialist ECSHD Victor Olsavszky Public Health Specialist WHO Frans Stobbelaar Consultant ECSHD Codruta Hedesiu Communication Officer UNICEF Holger Kray Senior Sector Economist ECSSD Goetz Schreiber Lead Economist ECSSD Irena Ramniceanu Economist ECSSD Stefan Nicolau Consultant ECSSD Violeta Visan Consultant ECSSD Radhika Srinivasan Senior Social Scientist ECSSD Nicholay Chistyakov Senior Finance Officer LOAG1 Jean-Charles de Daruvar Senior Counsel LEGEC Hans-Juergen Gruss Chief Counsel LEGEC Rohan Selvaratnam Senior Program Assistant ECSSD

Supervision/ICR Nurul Alam Senior Procurement Specialist ECSO2 Nadia Badea Operations Analyst ECSS1 Brian G. Bedard Senior Livestock Specialist ECSS1 Irene Bomani Senior Program Assistant ECSSD Bogdan Constantin Constantinescu

Senior Financial Management Specialist ECSO3

Ruxandra Costache Counsel LEGEM Agnes Couffinahl Senior Health Economist ECSH1 Amy Evans Consultant ECSS1 Richard Florescu Senior Operations Officer ECSH3 Camelia Gusescu Program Assistant ECCRO Ana Maria Ihora Program Assistant ECCRO Vladislav Krasikov Senior Procurement Specialist EAPPR Carmen Laurente Senior Program Assistant ECSHD Gabriela Doina Manea Resource Management Analyst HRSRM Wezi Marianne Msisha Health Specialist ECSH1 Cesar Niculescu Environmental Specialist ECSS3

Doina Petrescu Senior Rural Development Specialist ECSS1

41

Dan Ioan Sava Consultant ECSPE Barbara Ziolkowska Procurement Analyst ECSO2 Rob Coenraad de Rooij Consultant ECSSD Anneliese Viorela Financial Management Specialist ECCRO

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY06 81.46 FY07 55.08 FY08 0.00

Total: 136.54 Supervision/ICR

FY06 0.00 FY07 81.23 FY08 106.78

Total: 188.01

42

Annex 5. Beneficiary Survey Results (if any) N/A

43

Annex 6. Stakeholder Workshop Report and Results (if any) N/A

44

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

ROMANIA

MINISTRY OF HEALTH

PROJECT MANAGEMENT UNIT OF AVIAN INFLUENZA

AVIAN INFLUENZA CONTROL AND HUMAN PANDEMIC PREPAREDNESS AND RESPONSE PROJECT

IMPLEMENTATION COMPLETION RESULTS (ICR)

LOAN 4839RO The program was carried out between March 2007 and December 2010 and was financed by a World Bank loan of EUR 29.6 million. It was the first World Bank project to support the avian influenza control in Romania. Given Romania's endowment with waterfowl habitats (Danube Delta, UNESCO world heritage site) coupled with the dominance of backyard poultry with low bio-security, future AI outbreaks are likely. The natural and farming system environment create a real possibility that the virus could become endemic which would further increase the risk of future outbreaks. Spreading of the H5N1 HPAI among backyard poultry in Romania will significantly affect Romania's agriculture and economy in general; it poses a risk of human infection and also presents a danger of spread of the infection to poultry in other countries in the region. Project objectives: • To support and constitute an integral part of a national program to increase Romania’s avian

and human influenza preparedness, i.e. the country’s capacity to prevent and control the spread of influenza among animals and humans.

• To reduce the threat posed to humans and the poultry sector in Romania by HPAI and other

zoonoses, and to support the preparation for, control of, and response to influenza pandemics and other infectious disease emergencies in humans

• To support activities in the three areas of prevention, preparedness and contingency planning,

response and containment under the three components: (i) animal health; (ii) human health; and (iii) public awareness, communications and implementation support.

I. Project implementation – Main Achievements

COMPONENT 2: HUMAN HEALTH Funds availability: EUR 16.95 million World Bank loan

EUR 2.8 million Romanian Government

45

(A) Enhancing Public Health Program Planning and Coordination Funds availability: EUR 0.16 million World Bank loan Objectives: Activities under the first subcomponent will aim at enhancing coordination and program planning by providing financial and technical support for additional studies and research as well as for conferences and workshops, including the organization of a regional coordination meeting on preparations for a human influenza pandemic. The governmental authorities have developed individual contingency plans for animal and human health, overall national contingency planning and coordination still needs improvements. The Government has established a National Disease (AI) Control Center that has started a comprehensive process of reviewing, integrating, and consolidating existing contingency plans. European Council Directive 2005/94/EC has specific provisions for every EU Member Country in order to draw up a National Contingency Plan specifying the national measures to be implemented in the event of an avian influenza disease outbreak and human influenza pandemic. While Romania has joined the EU starting with January, 1, 2007, a National Contingency Plan must be developed in accordance with the Directive provisions and shall be submitted for the EC’s approval. While governmental authorities have developed individual contingency plans for animal and human health, overall national contingency planning and coordination needs improvements. Until the loan date, the existing contingency plans were reviewed in order to be integrated into a consolidated National Contingency Plan. Main activities carried on between March 2007 and December 2010: Public Health Program was previously prepared in 2006 in accordance with EU requirements; the need to update the AI contingency plan and to prepare related guidelines and manuals was reconfirmed at the end of 2008. MoH worked on this revision with human resources available in-house and finalized the national contingency plan at the end of July 2009, the objective of the component A has been achieved. Strengthening of National Public Health Surveillance Systems Funds availability: EUR 1.27 million World Bank loan Objectives: This subcomponent will help strengthen the national public surveillance system by: (i) Upgrading of the National Influenza Laboratory to BSL 3 through the financing of minor civil works (partitioning and creation of decontamination rooms), the supply and installation of additional security devices, and the installation of a system for maintaining negative pressure in the rooms; and (ii) Strengthening of regional laboratories by providing critical equipment and training of staff. Main activities carried on between March 2007 and December 2010: (i) Upgrading of the National Influenza Laboratory to NBL 3.

46

NBL3 Lab was improved for the purpose of reducing the threat of high risk avian influenza and other zoonosis on people, also to increase the training capacity, control and prompt response in case of pandemic influenza or other human infectious diseases. Works consisted in redevelopment, rehabilitation and modernization of the Laboratory area with level 2 biosecurity in order to answer international level 3 biosecurity requirements. The contract with the construction company was signed on October 12, 2009 and the works started soon after. The construction was suspended in December 2009, due to the lack of Government permits for the construction. The building permit was obtained on April 6, 2010 and the Government approved the investment on May 26 2010. In April 2010 the Cantacuzino Institute requested the PMU to hire a site supervisor to monitor the 3 sites for the civil works. The rehabilitation works for NBL 3 were successfully completed before December 31st 2010 but it has been planned to buy the 2 equipment after completion of the works, Containment Glove Boxes and a Mobile Decontaminating System. Considering that works were completed at the ending date of the project, the 2 equipment were not purchased anymore and they should be purchased by INCDMI Cantacuzino from its own funds or transferred from other units if necessary. The total payments made for the site supervision for the rehabilitation works at the BSL 3 laboratory were in the amount of 2.521 Euros. Since funds were available under the project for this purpose, the PMU took the necessary steps and hired an individual consultant to manage all civil works activities. Works were resumed early July 2010 and have been completed with success at the end of 2010. Considering that the works have been completed at the end of the project the bio-safety level 3 laboratory certification could not be financed from the project. Cantacuzino Institute made a market research and concluded that TUV-Nord (Germany) - internationally accredited laboratory have the lowest price (approx. 50.000 Euros). (ii) Strengthening of regional laboratories by providing critical equipment and training of staff. Also, in 2010 a Cantacuzino Institute representative was trained in US for laboratory supervision; training was supported by the Cantacuzino Institute and WHO. In order to ensure best performance of the laboratory the Cantacuzino Institute has to hire or nominate an internal person and train for the bio-security officer position. Some equipment and consumables that were planned to be purchased in addition to civil Works have not been procured any more considering that the works have been completed at the end of December 2010, the closing date of the project. The equipment will be purchased by the Cantacuzino Institute from their own budget. Strengthening Health System Response Capacity Funds availability: EUR 15.52 million World Bank loan

EUR 2.8 million Romanian Government

47

(i) The strengthening of the isolation and case management capacity in both national and regional hospitals.

The project financed equipment for the intensive care units of the two Bucharest infectious diseases hospitals and the establishment of isolation and intensive care units in the 7 regional centers for infectious diseases (Brasov, Craiova, Constanta, Cluj, Iasi, Timisoara and Tg. Mures). According to the National Plan, during the inter-pandemic period, all suspected cases shall be transported to and isolated at the Institute of Infectious Diseases Prof Dr Matei Bals, where medical assistance can be provided by specialized dedicated personnel. The centralization of patient care is the only possible way at this point in time as there is no medical unit able to ensure high quality care to suspected or confirmed AI human cases in the rest of the country. However, within the project the patients would have to be hospitalized within their region of residence. Therefore, through the project were developed isolation and intensive care capacities in the regional centers for infectious diseases. Main activities carried on between March 2007 and December 2010: The only Bank-funded activity planned under this component is the procurement of equipment for Intensive Care Units. All contract awards have been cleared, and contacts signed at the end of the year 2009 and in the first trimester of 2010. All the equipment packages for the infectious disease hospitals have been delivered and installed before December 31st, 2010 – closing date of the project. The payments were made according to the contracts in 2010 and in the first trimester of 2011 (Lot 1). In the procedure “Procurement of equipment for the Isolation and Intensive Care Units of the “Prof. Dr. Matei Bals” National Institute, “Dr. Victor Babes” Clinical Hospital for tropical and Infectious diseases and seven Regional Hospitals for infectious diseases - ICB3” were purchased equipment according to the below situation:

No. Lot Equipment

National Institute of Infectious Diseases “Prof.Dr.

Matei Balş”

Clinical Hospital for Tropical and

Infectious Diseases “Dr. Victor Babeş” Bucharest and

Regional Hospitals (7 units)

Total

Qty Qty

1 I Complex ICU bed with X-ray transparency 25 3 49

2 I Electric bed for ICU, transparent for X-ray 25 3 49

3 I ICU bars 50 6 98 4 I Mayo Table 50 6 98 5 I Service Trolley 50 2 66 6 I Treatment trolley 50 2 66 7 I Treatment table 50 2 66 8 I UV Lamps 2 2 18

48

9 II Intensive Care ventilator for all categories of critical care patients (adults, children and newborns)

25 7 81

10 II Intensive Care ventilator for pediatric and adults patients 25 2 41

11 II Analyzer Blood Gas Electrolytes and Metabolites 4 2 20

12 II Air compressor 3 1 11

13 II Washer disinfector machine for anesthetic materials 1 1 9

14 III Vital signs monitor 48 9 120 15 III Central Monitoring Station 3 1 11

16 III Multifunctional complex monitor for advanced intensive care 47 9 119

17 III ECG machine with 12 leads with monitor and interpretation 3 1 11

18 IV Volumetric pump for general use 200 20 360 19 IV Syringe pump 200 20 360

20 IV Docking station for automated syringes and volumetric pumps

50 6 98

21 IV Defibrillator with monitor 5 2 21 22 V Laminary flow cabinet 1 1 9 23 V ELISA automated station 1 1 9 24 V Bench top serological centrifuge 1 1 9 25 V Vertical laboratory autoclave 1 1 9 26 V Pipette Set 2 2 18 27 V Vortex mixer 1 1 9 28 V Pipette pump 2 2 18 29 V Laboratory Refrigerator 1 1 9 30 V Freezer 1 1 9 31 V Water still 1 1 9

32 VI Portable fibrobronchoscope for difficult intubation - adults

4 1 12

33 VI Pediatric portable fibrobronchoscope for dificult intubation

4 1 12

34 VI Central station for treatment and purification medical breathing air

1 1 9

35 VI Videoendoscope system with accessories 4 1 12 36 VI Otoscope 25 4 57 37 VI Ophthalmoscope 25 4 57 38 VII X-Ray Mobil unit 1 1 9

39 VII High performance pediatric echograph Doppler 1 0 1

40 VII Pediatric echograph Doppler 1 0 1 41 VII Mobile Doppler color echograph 2 1 10

49

ICB 3 – Contracts:

No Contract (Description) Supplier Contract value

Lot I Furniture and intensive therapy accessories DRC International bv 1,260,309

Lot II Ventilation Equipment for Intensive Care Units

Odelga Handelsgeseschaft GmbH 2,641,982

Lot III Monitoring equipment for Intensive care Units Nihon Kohden Europe GmbH 5,903,481

Lot IV Treatment equipment for Intensive Care Units

Biomedica Medizinprodukte mbH

1,366,605

Lot V Laboratory equipment SC AMS 2000 SRL 589,642

Lot VI Clinical Examination Equipment Tehno Electro Medical Company

1,166,725

Lot VII Mobile Imaging Equipment. Medrom SA Luxembourg 713,479

Total 13,642,223

Considering filling this objective the expected death rate of the HPAI infected people decreased from 70% before implementation of the project to 33% after implementation. (ii) The upgrading of the influenza vaccine production unit at the Cantacuzino Institute. This activity includes the rehabilitation and equipment of the influenza production department, and the completion and equipment of the filling and sealing unit. A significant part of the vaccines needed in Romania during the inter-pandemic and pandemic periods can be produced by the Cantacuzino Institute. Making the Institute able to significantly increase up to 2.5 million doses and its production in full compliance with GMP standards requires the upgrading of both the production unit and the filling and sealing station. This intervention will help both to increase the production capacity and to ensure that the production is at the required Good Manufacturing Practice standards. TA for engineering design and procurement support (vaccine production and filling units). The PMU signed a contract with SNC Lavalin NV in March 2008 provide technical assistance for engineering design and procurement support to upgrade the production unit and the filling and sealing station. The ending date of the contract was 24.05.2009; the contract was therefore finalized from this point of view according to the contract clause. At the ending date of the contract the consultant realized just a part of the tasks: technical documentation for building permit for the investment project at Cantacuzino Institute and the technical volumes of the SBD to be used for the procurement of works contract for “Upgrading filling & sealing unit”. Considering that the contract was not fully realized under contractual terms the Ministry of Health through the Project Management Unit has tried to negotiate a new contract but this failed. The total value of payments made to SNC Lavalin NV is 160.862 Euros.

50

The remaining work: technical volumes of the SBD to be used for the procurement of works contract for “Upgrading vaccine production unit”, validation file for the vaccine production unit and validation file for the filling & sealing unit will be provided for completion of the objective by another technical assistance firm – SC Claudia Design Art SRL through the Ministry of Health – APL2 project, also financed from a world bank loan. Due to the economic crisis and the pressure it puts on public finances in Romania, the share of the MoH 2010 budget earmarked for projects funded by international organizations was lower than the amount initially requested by the Avian Influenza Project Management Unit. The Ministry of Public Finance has allocated for 2010 the amount of 14.6 million Euro instead of 18 million Euro requested by the PMU at the end of the year 2009. This allocation was used in priority to fund on-going contracts and/or contracts which were close to being awarded. As a consequence, funding for the civil works needed for the rehabilitation of the vaccine production at the Cantacuzino Institute was not available for 2010. Equipment The procedures ICB1, ICB2 and ICB4 “Supply equipment to increase capacity of vaccine production unit” were organized to achieve the objective “Upgrade of the influenza vaccine production capacity at Cantacuzino Institute” by purchasing equipment according to the situation below:

Contract (Description) Supplier Contract value

ICB#1 Bidding for vaccine unit - 8 lots

Lot 2 - Cogent M low capacity tangential ultrafiltration system SC Teknoleb SRL 65,624

Lot 4 - eKII continuous flow zonal ultracentrifuces (2 pieces) Alfa Wassermann b.v. 597,753

Lot 5 - Steris Finn-Aqua WFI water system and Distribution loops for WFI, PW and CS JV Steris Corp/Sapaco SA 1,403,873

Lot 6 - Steris Finn-Aqua Clean Steam System (PSG 450T) and Steris washing machine (Reliance 580) JV Steris Corp/Sapaco SA 259,470

Lot 7 - Dehydrator AV3000L-Waste Disposal System Rame Hart Inc. 343,555

Lot 8 - Sterinis egg decontamination system SC Synttergy Consult SRL 7,566 Lot 9 - Cold Room for cooling inoculated eggs SC Sapaco 2000 SA 22,611

Lot 12 - Control Laboratory Equipment (14 items) SC Medist SA 360,231

ICB#4 Bidding for vaccine unit - 5 lots

Lot 1 - Continuous Flow Centrifuge BW PSA/CARR 464,220

Lot 2 - Phase Separation Centrifuge CINC Deutschland GmbH 158,298

Lot 3 - Mobile Cleaning in Place (CIP) system SC Sapaco 2000 SA 169,289

Lot 4 - Ultrasonication with continuous flow cell SC Dexter Com SRL 10,944

51

Lot 5 - Horizontal standing autoclaves (2 pcs) Zirbus Technology GmbH 133,480

ICB#2 Bidding for filling unit- 2 lots

Lot 1 - Etipack Labeling system Etifiale 14 - automatic machine for labeling ampoules and vials

Elde Packaging and Coding Equipment 158,604

Lot 2 - Visomat IV inspection machine of ampoules and vials

MASCHINPEX Maschinenbau GmbH 159,264

TOTAL 4,314,784 Also, for the vaccine unit in 2010 a contract was signed with TEKNOLABO A.S.S.I. Italy to provide equipment for upgrading the inoculation and harvesting machine. All the equipment was delivered and installed in October 2010. The total amount paid for this activity is 57.719 Euros. COMPONENT 3: PUBLIC AWARENESS, COMMUNICATIONS, AND IMPLEMENTATION SUPPORT Implementation Support, Monitoring and Evaluation Funds availability: EUR 0.90 million World Bank loan The project was implemented by the MPH and ANSVSA, and used already existing Project Management Units in the two agencies at the loan date. In order to ensure effective implementation of the project in accordance with Bank fiduciary and procurement policies and guidelines, the project financed: (i) Incremental support to the two PMUs in terms of staff and operating costs; (ii) Studies, technical services, manuals; and (iii) Surveys for monitoring and evaluation. The MoH Project Management Unit was hosted by the Cantacuzino Institute, and functions as a self-standing unit, de-linked from the PMU for the Health APL 2 Project, as previously recommended by the Bank. The Government decision (issued end of December 2009) to incorporate all PMUs implementing projects with international financing into the regular structure of the Ministries, led to significant reductions of the earnings of their staff. Following the resignation of the PMU Director in June 2009, Financial Coordinator in July 2010 and the three positions remained vacant from 2008, the PMU 3 members team (Technical Coordinator, Procurement specialist and Accountant) has successfully completed the project. The cooperation with the World Bank The PMU and MoH had an excellent cooperation with the Bank’s team. They are excellent professionals, very well connected to the Romanian health needs and reality and very committed to support, to facilitate the project implementation and to find flexible ways to solve different issues. Interpersonal communication and strong relationships were built between the PMU staff and Bank’s staff reflected also in the good results.

52

It is already a tradition for the MoH and PMU to look for the Bank’s specialist guidance and agreement in the most important measures and actions, as an appreciation of their valuable knowledge and experience. The PMU highly appreciates the way the Bank’s team has cooperated and worked in this project and specially thanks for the permanent support and encouraging of the PMU for involvement in other projects.

53

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N/A

54

Annex 9. List of Supporting Documents • Project Implementation Plan

• Aide Memoires, Back-to-Office Reports, and Implementation Status Reports

• Project Progress Reports

• Technical Annex for Avian Influenza Control & Human Pandemic Preparedness &

Response Project, dated August 14, 2006 (Report No: T7679-RO )

• Borrower's Evaluation Report dated May 2011 *including electronic files

MoldoveanuMoldoveanu(2,544 m )(2,544 m )

Ba

n

at

W a l a c h i aW a l a c h i a

Do

br

uj

a

Mo

ld

av

ia

T r a n s y l v a n i a n A l p s

Olt

Olt

ArgesArges

IalomitaIalomita

BuzauBuzau

SiretSiret

SiretSiret

BistritaBistrita

Crisul Alb

PrutPrut

Mures

Timis

Som

es

Jiu

Danube

Danube

A R A DA R A D

G O R JG O R J

HUNEDOARAHUNEDOARA

CARAS -CARAS -SEVERINSEVERIN

T I M I ST I M I S

PRAHOVAPRAHOVA

BRASOVBRASOVCOVASNACOVASNA

S I B I US I B I U

ARGESARGES

IALOMITAIALOMITA

VRANCEAVRANCEA

V A S L U IV A S L U I

GALATIGALATI

T U L C E AT U L C E A

NEAMTNEAMT

I A S II A S I

BOTOSANIBOTOSANI

HARGHITAHARGHITA

SUCEAVASUCEAVAMARAMURESMARAMURES

A L B AA L B A

C L U JC L U J

MURESMURESB I H O RB I H O R

SATU MARESATU MARE

O L TO L T

D O L JD O L J

MEHEDINTIMEHEDINTI

TELEORMAN TELEORMAN GIURGIUGIURGIU CONSTANTACONSTANTA

BUCURESTIBUCURESTI

DÂMBOVITAMBOVITA

VÂLCEALCEA

BISTRITA-BISTRITA-NASAUDNASAUD

BUZAUBUZAU

BRAILABRAILA

CALARASICALARASI

SALAJSALAJ

B A C A UB A C A U

LugojLugoj HunedoaraHunedoara

TurdaTurda

PetrosaniPetrosani

OrsovaOrsova

CalafatCalafat

CaracalCaracal

MediasMedias

DejDej

RomanRoman

OnestiOnestiBirladBirlad

MedgidiaMedgidia

FetestiFetesti

HusiHusi

Turnu MagureleTurnu Magurele

TecuciTecuci

GheorgheniGheorgheni

BradBrad

DevaDeva

AradArad

SibiuSibiu

ZalauZalau

TulceaTulcea

VasluiVaslui

OradeaOradea

GiurgiuGiurgiu

CraiovaCraiova

SlatinaSlatina

SuceavaSuceava

SloboziaSlobozia

Târgu Jiurgu Jiu

Baia MareBaia Mare

Satu MareSatu Mare

AlexandriaAlexandria

AlbaAlbaIuliaIulia

Cluj-Cluj-NapocaNapoca

RâmnicumnicuVâlcealcea

Miercurea-Miercurea-CuicCuic

Drobeta-Drobeta-Turmu SeverinTurmu Severin

BistritaBistrita

TârgurguMuresMures

BotosaniBotosani

IasiIasi

Piatra-Piatra-NeamtNeamt

PloiestiPloiestiTârgovistergovistePitestiPitesti

ResitaResita

TimisoaraTimisoara

GalatiGalatiBrasovBrasov

FocsaniFocsani

BacauBacau

CalarasiCalarasi

BrailaBraila

BuzauBuzau

SfSfântuntuGheorgheGheorghe

BUCHARESTBUCHAREST

Ca

rp

at

hi

an

M

t s.

ILFOVILFOV

UKRAINEUKRAINE

UKRAINEUKRAINE

HUNGARYHUNGARY

BULGARIABULGARIA

To To ChisinauChisinau

To To ChisinauChisinau

To To Ivano-Frankivs'kIvano-Frankivs'k

To To UzhhorodUzhhorod

To To BudapestBudapest

To To BudapestBudapest

To To SuboticaSubotica

To To Novi SadNovi Sad

To To SofiyaSofiya To To

VarnaVarna

To To ShumenShumenTo To

ShumenShumen

To To BaltiBalti˘

To To Veliko TurnovoVeliko Turnovo˘

To To NisNis̆

SERBIASERBIA

Lugoj Hunedoara

Turda

Petrosani

Orsova

Calafat

Caracal

Medias

Dej

Roman

Birlad

Navodari

Mangalia

Medgidia

Fetesti

Husi

Turnu Magurele

Tecuci

Gheorgheni

Brad

Onesti

Deva

Arad

Sibiu

Zalau

Tulcea

Vaslui

Oradea

Giurgiu

Craiova

Slatina

Suceava

Slobozia

Târgu Jiu

Baia Mare

Satu Mare

Alexandria

AlbaIulia

Cluj-Napoca

RâmnicuVâlcea

Miercurea-Cuic

Drobeta-Turmu Severin

Bistrita

TârguMures

Botosani

Iasi

Piatra-Neamt

PloiestiTârgovistePitesti

Resita

Timisoara

Constanta

GalatiBrasov

Focsani

Bacau

Calarasi

Braila

Buzau

SfântuGheorghe

BUCHAREST

A R A D

G O R J

HUNEDOARA

CARAS -SEVERIN

T I M I S

PRAHOVA

BRASOVCOVASNA

S I B I U

ARGES

IALOMITA

VRANCEA

V A S L U I

GALATI

T U L C E A

NEAMT

I A S I

BOTOSANI

HARGHITA

SUCEAVAMARAMURES

A L B A

C L U J

MURESB I H O R

SATU MARE

O L T

D O L J

MEHEDINTI

TELEORMAN GIURGIU CONSTANTA

BUCURESTIILFOV

DÂMBOVITA

VÂLCEA

BISTRITA-NASAUD

BUZAU

BRAILA

CALARASI

SALAJ

B A C A U

UKRAINE

MOLDOVA

UKRAINE

HUNGARY

SERBIA

BULGARIA

Olt

Arges

Ialomita

Buzau

Siret

Siret

Bistrita

Crisul Alb

Prut

Mures

Timis

Mure

s

Som

es

Jiu

Danube

Danube B lack

Sea

To Chisinau

To Chisinau

To Ivano-Frankivs'k

To Uzhhorod

To Budapest

To Budapest

To Subotica

To Novi Sad

To Sofiya To

Varna

To ShumenTo

Shumen

To Balti˘

To Veliko Turnovo˘

To Nis̆

Ba

n

at

W a l a c h i a

Do

br

uj

a

Ca

rp

at

hi

an

M

t s.

Mo

ld

av

ia

T r a n s y l v a n i a n A l p s

Moldoveanu(2,544 m )

22°E 24°E 26°E 28°E

24°E

48°N

46°N

44°N44°N

30°E

30°E

ROMANIA

This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other informationshown on this map do not imply, on the part of The World BankGroup, any judgment on the legal status of any territory, or anyendorsement or acceptance of such boundaries.

0 25 50 75

0 25 50 75 Miles

100 Kilometers

IBRD 33469R2

FEBRUA

RY 2008

ROMANIASELECTED CITIES AND TOWNS

COUNTY (JUDET) CAPITALS

NATIONAL CAPITAL

RIVERS

MAIN ROADS

RAILROADS

COUNTY (JUDET) AND MUNICIPALITY(MUNICIPIU) BOUNDARIES

INTERNATIONAL BOUNDARIES

Document of The World Bankdocuments.worldbank.org/curated/pt/... · Document of The World Bank....

Documents

Transcript of Document of The World Bankdocuments.worldbank.org/curated/pt/... · Document of The World Bank....