Shelter Programme Monitoring and Evaluation - … · Shelter Programme Monitoring and Evaluation...

Shelter Programme Monitoring and Evaluation

Final Report

28 February 2005

Pat Foley

Monitoring and Evaluation Consultant [email protected]

CONTENTS 1. Summary ________________________________________________________________6 1.1 Introduction __________________________________________________________________________6 1.2 Performance _________________________________________________________________________6 1.3 Impact ______________________________________________________________________________8 1.4 Recommendations_____________________________________________________________________9 2. Shelter Monitoring and Evaluation __________________________________________11 2.1 M&E progress report __________________________________________________________________12 2.2 2005 shelter guidelines ________________________________________________________________16 2.3 Shelter logical framework ______________________________________________________________17 2.4 Shelter compact disc __________________________________________________________________19 2.5 M&E final report ______________________________________________________________________19 3. Programme Performance __________________________________________________21 3.1 Shelters are established according to guidelines ____________________________________________21 3.2 Families are able to complete shelters and latrines __________________________________________23 3.3 Women participate in implementation and decision making ____________________________________25 3.4 Families are selected according to eligibility criteria __________________________________________27 3.5 Shelters provide a minimum floor space of 3.5m2 per person___________________________________30 3.6 Latrines are built with every shelter _______________________________________________________32 4. Programme Impact _______________________________________________________33 4.1 At least 15% of vulnerable returnees receive shelter assistance ________________________________33 4.2 The poorest beneficiaries receive additional labour or cash support _____________________________35 4.3 Shelters remain occupied with minimal secondary displacement of beneficiaries ___________________37 4.4 Beneficiaries are able to improve their condition after returning _________________________________39 4.5 Environment, accessibility, and security are not overlooked____________________________________41 5. Recommendations________________________________________________________43 Figures 1. Shelter programme map 2002 – 2004______________________________________________________5 2. Performance analysis __________________________________________________________________7 3. Impact analysis _______________________________________________________________________8 4. M&E progress report findings ___________________________________________________________12 5. M&E progress report recommendations and implementation ___________________________________13 6. Shelter logical framework ______________________________________________________________18 7. Completed shelters ___________________________________________________________________22 8. Selection and return trends _____________________________________________________________24 9. Returnee sex by region ________________________________________________________________26 10. Returnee sex summary ________________________________________________________________26 11. UNHCR Afghanistan vulnerability categories _______________________________________________28 12. 2004 vulnerability data_________________________________________________________________29 13. Returnee family size __________________________________________________________________31 14. Families achieving floor space performance standard ________________________________________31 15. Returns and beneficiaries by year ________________________________________________________34 16. Returns and beneficiaries by region ______________________________________________________34 Annexes A. Terms of reference for shelter M&E_______________________________________________________47 B. Shelter M&E progress report (summary)___________________________________________________49 C. 2005 shelter guidelines (table of contents) _________________________________________________53 D. Fieldwork guide I _____________________________________________________________________54 E. Fieldwork guide II_____________________________________________________________________55 F. Secondary references _________________________________________________________________56 G. Shelter completion 2002 – 2004 _________________________________________________________58

UNHCR Afghanistan of 58 2004 Shelter M&E

ACRONYMS BOK Branch Office Kabul BSC Beneficiary Selection Committee ECHO European Community Humanitarian Aid Department FHH Female Headed Household FO Field Office GTZ Deutsche Gesellshaft für Technische Zusammenarbeit

(German Technical Cooperation) HOH Head of Household IDP Internally Displaced Person IP Implementing Partner LOI Letter of Intent LOU Letter of Understanding M&E Monitoring and Evaluation MORR/DORR Ministry/Directorate of Refugees and Repatriation MRRD/DRRD Ministry/Directorate of Rehabilitation and Rural Development MUDH/DUDH Ministry/Directorate of Urban Development and Housing NGO Non Governmental Organisation NOC No Objection Certificate OCM Office of the Chief of Mission OIU Operational Information Unit (at Branch Office Kabul) RCM Remote Control Management SO Sub Office UNHCR United Nations High Commissioner for Refugees VRF Voluntary Repatriation Form WFP World Food Programme



ACKNOWLEDGEMENTS This paper is the result of fieldwork, discussion, and analysis with diverse stakeholders in the Afghanistan shelter programme – including beneficiaries, government, implementing partners, and UNHCR staff. As a qualitative M&E specialist, not a shelter specialist, my role concentrated on bringing together the collective experience of colleagues and counterparts nationally. Neither the M&E outputs nor the revised 2005 Shelter Guidelines would have been possible without the substantial contributions of numerous agencies and individuals throughout the country, too many to mention individually. Credit too goes to the translators, drivers, logisticians, and regional support staff who made the fieldwork components possible. Field Office Kabul staff were a useful sounding board for ideas during the shelter M&E methodology review and revision. All other regions were equally accessible and constructive in their feedback, but FOK was consulted more frequently due to their proximity. The Operational Information Unit deserves particular thanks for their support in revising the data collection and presentation systems, and for handling repeated data requests. Responsibility for the content and presentation of this paper remains mine; comments and feedback are welcome.

Figure 1: Shelter programme map 2002 – 2004 (total completed shelters as of January 2005)

1. SUMMARY 1.1 Introduction The UNHCR shelter programme aims to assist the sustainable reintegration of returned refugees and IDPs into their communities of origin though promotion of gradual, voluntary, safe, and gender sensitive return. Between 2002 and 2004, approximately 3.8 million individuals returned to Afghanistan – including more than 460,000 internally displaced persons (IDPs). This represents just over half of the six million Afghans estimated to have fled from conflict and drought since the 1979 Soviet invasion. Although accurate data are unavailable, UNHCR estimates that perhaps 500,000 homes have been either partially or entirely destroyed. Since 2002 the programme has sought to address the shelter needs of returnees and IDPs most in need of assistance. By the end of 2004, just over 117,000 shelters have been constructed by vulnerable returnees through the combined efforts of UNHCR, the Ministry for Refugees and Repatriation, and national and international NGO implementing partners. An additional 21,500 shelters are planned for 2005 in response to the approximately 150,000 refugee families and 17,000 IDP families expected to return to their areas of origin this year. To assist the most vulnerable returnees to rebuild their homes as part of their reintegration, the shelter assistance package is normally delivered to family place of origin – typically in rural areas, where an estimated 80% of the Afghan population resides.1 The package consists of essential construction materials (beams, windows, door, tools) and a cash grant for additional materials and labour. The shelter design is based on a 32m2 mud structure composed of two rooms with windows, one corridor, and an external latrine. The shelter programme advocates self help assistance, with participating families required to contribute time, labour, and costs. For particularly poor families unable to mobilise sufficient resources, additional efforts are made to secure community assistance or to approve additional cash support so that the number of families unable to participate is minimised. This report presents the outputs and conclusions generated during a six-month monitoring and evaluation (M&E) consultancy investigating qualitative aspects of the UNHCR shelter programme in Afghanistan.2 The content of this report reflects the ideas and experience of beneficiaries, counterparts, and staff throughout the country. As much as possible, emphasis is given to themes consistent across all regions to emphasise their importance to the national programme as a whole. Attempts are also made to unite several separate documents through extensive cross referencing so that their relationships are more evident and their use more likely. After documenting the outputs of the shelter M&E consultancy, the report evaluates the 2002 – 2004 programmes to synthesise key findings and to provide a baseline against which future evaluations can be compared. Using the logical framework approach to provide a methodological structure, the evaluation investigates programme performance and impact (summarised in Figures 2 and 3, respectively). 1.2 Performance Performance is based on output level analysis of the shelter logical framework, stating that the UNHCR shelter programme facilitates construction of shelters in areas of high and potential return. Each year has succeeded in maintaining a high completion rate, and implementation has begun to demonstrate signs of earlier completion each year. The shelter design is appropriate and consistent with Afghan standards and is composed of quality materials that are better than what families would ordinarily be able to purchase themselves. The shelter design similarly meets

1 WFP & MRRD 2004. 2 Consultancy period was 19 July 2004 to 31 January 2005.


UNHCR and Sphere standards for minimum floor space per person for all but the largest beneficiary families. Most families are building and using latrines as part of the programme. Despite effective delivery of the above, most shelters tend to be completed during winter rather than during the ideal construction period of late March to August or September. This can make construction slower and more expensive, which can ultimately weaken household economies during the return and reintegration process. Female participation is also limited in the programme, although not for lack of effort at all levels. Gender inclusion in Afghan programming is bound by cultural and contextual considerations that affect the ability to engage female staff and beneficiaries to the same extent as males. Finally, beneficiary vulnerability is the primary criterion for programme eligibility, but the concept itself is not well defined and not clearly distinguished from poverty. The shelter programme is nonetheless performing according to targets and standards, and except in areas of high insecurity has been able to concentrate allocations in regions of high and potential return. As the programme continues to improve implementation schedules and maintain its achievement rate during 2005 and 2006, families will continue to benefit from secure shelter in their places of return. Figure 2 outlines performance level findings and the corresponding section in which they are discussed in greater detail. Figure 2: Performance analysis

Finding Report Section

• UNHCR has maintained a 99% completion rate for each year of the shelter programme. • The rate of programme implementation is improving for each year. • The shelter design is consistent with Afghan standards. • Shelter materials are of better quality than what most families would otherwise be able to afford.

3.1

• Ongoing technical support maximises the number of beneficiaries who are able to complete their shelters.

• Revised monitoring formats will support the quality of remote control management. • Families will benefit from the new policy to approve handover upon completing the shelter and

latrine, rather than upon occupancy. • Most shelters are built after the ideal construction period (late March – August/September) has

passed. • Beneficiaries prefer to complete their shelters well before the onset of winter because of the

subsequently reduced availability of water and wages.

3.2

• Direct female participation is possible but minimal due to cultural and contextual considerations. • Remote control management further inhibits efforts to facilitate female participation. • Efforts to include more women are sometimes limited to targeting female headed households,

rather than trying to reach female and male members of the same household during assessment and selection.

• Programmatic gender analysis has been limited to date but will increase in subsequent programme years.

3.3

• UNHCR classifications of vulnerability have changed during each year of the shelter programme. • Widows, elderly, and disabled are valid as priority vulnerability groups. • Opportunities to better understand vulnerability dynamics in the shelter programme have been

missed by limiting analysis to widows, elderly, and disabled. • There is no clear distinction between vulnerability and poverty, with no recognition of how the

shelter programme might affect or reduce beneficiary vulnerability. • The selection process is focused and effective in identifying appropriate families for assistance.

3.4

• Approximately 92% of families achieve the performance standard of 3.5m2 per person per shelter.

• Floor space performance achievement has not been assessed in previous years. • Families with 9 members or more will not achieve the floor space standard. • Regional management has the flexibility to determine whether supplemental shelter materials

should be authorised for large families in order to maintain the standard.

3.5

• Families are building and using latrines in most locations. • Requiring completed latrines to qualify for shelter handover is effective to maximise the number

of latrines. 3.6


1.3 Impact Impact is based on objective and goal level analysis of the shelter logical framework, which states that the most vulnerable returnees have access to adequate shelter. UNHCR has on average surpassed its target of providing shelter to 15% of vulnerable returnees during 2002 – 2004. Some of the poorest families, however, have been missed along the way as a result either of their inability to mobilise sufficient human and financial resources for construction or their inability to access land upon which to establish themselves. This slippage may to some extent be unavoidable, although implementing partners and UNHCR alike have so far minimised the number of poor families unable to participate through mobilisation of voluntary community assistance (ashar) and through approval of supplemental cash grants for the poorest families. A distinction is also made between shelter and housing, using the argument that UNHCR is in fact facilitating the construction of permanent socioeconomic bases from which families pursue livelihood strategies. Acknowledging the organisational reasons why the term shelter must prevail, this distinction is nonetheless important when considering overall programme impact. Apparently minimal secondary displacement following shelter occupancy further supports that families are better grounded in their places of return as a result of establishing a shelter. It therefore appears certain that most families who invest in and build a shelter will ultimately inhabit the unit, regardless of whether occupancy takes place within a particular programme year. Once living in their shelters, families can begin the process of social and economic reintegration – although the ability to improve livelihoods over time will be determined in part by family access and control over land, labour, and water. This is an unfortunate irony, as the vulnerability that made families eligible for shelter in the first place is likely to reflect similar issues of access and control of these essential resources. The most lingering question of household impact is whether the apparently high incidence of beneficiary debt for material and construction costs can be absorbed by household economies. Although not quantifiable, it appears that beneficiaries everywhere are borrowing to fulfil their share of programme contribution costs. The programme ethos of self help housing is both constructive and appropriate, but it remains to be seen how debt might undermine household ability to improve livelihoods over time. The shelter selection process varies by region but is nonetheless consistent in its guiding principles (from the Shelter Guidelines) and the effort to include diverse stakeholders in an inclusive, transparent, and informed process. Regional flexibility is essential to maintaining appropriate implementation strategies, and has contributed to an effective selection process that identifies appropriate families and ensures their access to shelter. Figure 3 outlines impact level findings and the corresponding section in which they are discussed in greater detail. Figure 3: Impact analysis

Finding Report Section

• Overall, UNHCR has surpassed its target by reaching approximately 22% of returnees with shelter.

• 2004 achievement was only 8% as a result of the high number of returns. • Staff feel that although usually succeeding to meet targets, annual shelter allocations are

insufficient. • Improved beneficiary profiling by type of return will help determine whether 15% is an appropriate

target for future years.

4.1

• Despite genuine efforts, there is a thin layer of vulnerable families who are too poor to participate in the programme or who cannot mobilise the required resources.

• Where possible, mobilising ashar contributions helps to maximise the number of families who complete their shelters.

• Ashar is not always possible or available, as some families cannot afford to forfeit paid labour opportunities.

• Additional cash grants are also effective to support the poorest families, but not all regions utilise this facility.

4.2


• Although families who might not be able to complete their shelters can sometimes be identified in advance, such cases are sometimes not apparent until implementation has begun.

• Some layer of slippage appears to be unavoidable, but can however be minimised by the BSC, IP, and UNHCR.

• The shelter programme reinforces legal and social reintegration benchmarks such as access to shelter and officially affirmed land ownership.

• Although officially called shelters, UNHCR is facilitating the construction of permanent houses that provide a foundation for family reintegration.

• Even in cases of labour migration, the family remains in the shelter while male members leave in search of work.

• Secondary or reverse displacement appears to be minimal. • Secondary displacement will usually result from insufficient wages or water – not from a lack of

shelter. • If families are further displaced, they nonetheless maintain control over the shelter (or its

materials) as an important household asset. • Families who invest in and construct a shelter will in almost all cases occupy it.

4.3

• Household ability to improve conditions will depend on accessibility of land, labour, and water – or some combination thereof.

• Shelters are facilitating reintegration through provision of a family base from which livelihood strategies can be pursued.

• Family ability to remain rooted supports their power to maintain the social and economic ties that further support reintegration.

• Some families have experienced a decrease in their livelihood status since returning, the result of lower wages and higher commodity prices.

• Debt incurred during shelter construction might be undermining household economies and resulting in less available resources for household consumption.

4.4

• While the selection process demonstrates consideration of environment, accessibility, and security a standard assessment methodology is difficult to standardise due to the regional variety of the shelter programme.

• The ability to identify potential water shortages before they cause construction delays can facilitate more effective implementation.

• More integrated programming with shelter and water could add value to each intervention. • Security assessment has influenced the shelter allocation process – balanced by the need to

target areas of high and potential return. • Accessibility of markets or facilities such as education or health will be part of beneficiaries

decision to return, but should nonetheless remain included in regional assessment of viable shelter locations.

4.5

1.4 Recommendations Based on the experience of 2002 and 2003 beneficiaries met during fieldwork, we can expect that most families will occupy their completed shelters, and that they will continue to invest in maintenance and improvement in the years to come. The shelters offer a permanent socioeconomic base from which livelihood strategies will be pursued. The UNHCR shelter programme is an appropriate and effective intervention that has fulfilled its stated output and objective annually. Programming and beneficiaries alike would however benefit from more consistent attention to M&E processes in the course of ongoing implementation during 2005 and 2006 – possibly the final years of the programme – that will ultimately increase programme effectiveness and impact. Most of the momentum generated during the M&E consultancy has already been woven into practice through revision of the 2005 Shelter Guidelines and corresponding systems for data collection, reporting, and analysis. The remaining recommendations generated during the final reporting period are therefore oriented toward further maintenance and revision of what is already underway, in hope that this will further internalise the process and lessons of shelter monitoring and evaluation. The report concludes by elaborating the following recommendations:


1. Complete shelters as early in the year as possible, and consider whether it is possible to implement in batches that begin with the beneficiary backlog from the previous year.

2. Internally clarify M&E roles and responsibilities at BOK and regional offices. 3. Increase dialogue between the Programme Unit and the Operational Information Unit. 4. Ensure translation and dissemination of Dari and Pashto versions of the 2005 Shelter

Guidelines. 5. Ensure translation and dissemination of Dari and Pashto versions of the Guideline summary

to be used in regional workshops. 6. Ensure regional workshops are convened to improve IP and DORR understanding of revised

Guidelines and reporting formats. 7. Continue to refer to the shelter logical framework and further revise if necessary. 8. Revisit the regional shelter logical frameworks used for 2005 LOI submissions and identify

means of verification for each indicator. 9. Ensure that Monthly Progress Reports are consistently complete and submitted on time. 10. Ensure that Monthly Consolidated Reports are disseminated to all regions on time. 11. Make more routine use of available programme data. 12. Perform evaluations at the end of every programme year.


2. SHELTER MONITORING AND EVALUATION Quantitative data collection during the life of the shelter programme has been both thorough and accessible, with regional offices and the central Operational Information Unit jointly collecting and collating a range of data that are readily accessible and linked to a sophisticated mapping system. Qualitative interpretation of programme data has however been limited, resulting in the decision to contract an M&E specialist to review the shelter programme. Working from UNHCR Branch Office Kabul, the M&E process began with consultations with section heads, counterparts from other agencies, and regional heads of office and programming. Plans and reports from previous programme years were reviewed, in addition to key documents produced by donors and regional staff. Although limited by insecurity and restricted mobility during the Afghan presidential election and later the inauguration, the consultancy included fieldwork in six of the seven programme regions. Fieldwork relied on key informant interviews, focus group discussions, and direct observation. The fieldwork schedule was: East Sub Office Jalalabad 16 – 21 August 2004 South Sub Office Qandahar 23 – 28 August 2004 Southeast Sub Office Gardez 29 August – 9 September 2004 North Sub Office Mazar-e Sharif 4 – 14 December West Sub Office Herat 16 – 22 December Central Field Office Kabul July – December 2004 (intermittently) Central Highlands Field Office Bamyan not visited The M&E consultancy has endeavoured to work within the framework of the existing shelter programme, rather than to create or propose an entirely new direction or methodology. This approach was adopted to maximise the input and experience of staff and counterparts who had close to three years of programme implementation experience and habits behind them. It was also envisioned that revising existing structures rather than building new ones would increase the acceptability and ultimate usability of whatever recommendations were implemented. The M&E consultancy and its outputs seek to bridge the distance between research and practice. Findings have therefore been carried into programmatic revisions, not simply submitted as recommendations that might never be acted upon. The increased dependence on remote control management as a result of increased insecurity in several regions played an important role in the approach to revising the M&E methodology. Because of the challenges inherent in remote control, it became apparent that greater clarity and structure of reporting formats would help to ensure remotely collected data was thorough and reliable. This initiated the revision of the overall shelter monitoring system and reporting formats, and to corresponding changes in the 2005 Shelter Guidelines. More generally, the M&E consultancy sought to build on qualitative findings to improve the framework for implementation, data collection, reporting, and analysis to ensure greater comparability between regions and a subsequently more coherent picture of the national programme as a whole. The logical framework was an important tool in this process, giving shape both to fieldwork and to the final evaluation. This approach will also facilitate future evaluations by providing a baseline for comparison. Each of the consultancy outputs therefore builds on the previous in an iterative process of translating findings into programming. There are five outputs of the M&E consultancy, each of which will be discussed below: 1. M&E Progress Report that identifies and analyses qualitative shelter programme issues. 2. 2005 Shelter Guidelines revised to reflect programme evolution and the changes made to

data collection themes and formats based on Progress Report recommendations. 3. Shelter Logical Framework that gives programmatic structure and focus for M&E.


4. Shelter Compact Disc that contains essential and supporting documents for 2005. 5. M&E Final Report that includes discussion of all the above, in addition to an evaluation that

uses the logical framework as a guide and as a baseline for future evaluation. Despite the concrete outputs, the consultancy demonstrated several limitations. Foremost was that it was an isolated position. Even though the process depended on continuous dialogue, there was no M&E person with whom to work in concert so that the breadth of ideas remains grounded in the organisation. Secondly, interaction with the Ministry for Rural Rehabilitation was limited to provincial and district levels, without engaging directly with the ministry in Kabul. This bias came from the pursuit of keeping the consultancy grounded in fieldwork and the practicalities of implementation, but in hindsight should have nonetheless included Kabul discussions too. Insecurity and subsequently limited field access also affected the consultancy, resulting in nearly two months during which fieldwork was impossible. Even when fieldwork was possible, the specific locations open to UNHCR staff were frequently limited. In some cases official vehicles were accompanied by security convoys, but in other cases certain areas simply could not be visited. Fieldwork findings are therefore representative of those areas open to direct access and to the secondary input of IP, DORR, and UNHCR staff. Despite these limitations, the process and outputs of the consultancy are believed to be valid and relevant, offering concrete progress in the initiative to improve qualitative understanding of the shelter programme and ultimately to increase programme impact. 2.1 M&E progress report The M&E Progress Report was produced and circulated for comments in September 2004, then revised and redistributed in October (in full and summary versions; see Annex B). The report investigates qualitative aspects of the UNHCR shelter programme, concentrating on four main themes: • Beneficiary selection • Monitoring and data collection • Shelter occupancy • Programme impact Despite giving direction to the remainder of the M&E consultancy, the potential benefits of the Progress Report were reduced by several limitations. The report was long and analytic, at times dependent on complicated language in its attempt to identify and elaborate on the qualitative themes listed above. As a result, the report does not appear to have been read or circulated widely, even within UNHCR Afghanistan – although this may also indicate an operational culture inclined toward summaries and bullet points. Readership and distribution to IPs was minimal. Nonetheless, the report provides a detailed qualitative foundation to inform the shelter programme and its M&E, despite the unexpectedly low level of feedback and readership. The findings and recommendations gave focus to the remainder of the consultancy, to all subsequent fieldwork, and to many of the revisions included in the 2005 Shelter Guidelines. It will therefore be helpful to outline the primary findings, by theme: Figure 4: M&E progress report findings Targeting & Selection • The selection process appears to be effective in reaching appropriate families. • Vulnerability criteria are subjective and unstandardised, with an unclear distinction from poverty. • Kuchi are not specified as a possibly vulnerable group. • Beneficiary ‘preselection’ might facilitate the BSC decision making process. • Female participation at all levels of the programme appears minimal. • Among poorest of the poor, some families are unable to benefit from the programme because of landlessness or an

inability to afford construction – even where voluntary labour might be accessible.


Technical Monitoring • Revised and more expansive monitoring formats can help guide implementation at all programme levels. • Monitoring formats designed by individual sub or field offices serve to meet regional needs, but do not allow for

national data comparability or analysis. • Field level IP follow-up is worthwhile, but its technical role might be limited as shelter construction techniques appear

to be widely known in communities. • Remote control management, monitoring, or verification in insecure areas presents programmatic challenges.

Revised monitoring formats can affect a measure of control over the process. • Training to strengthen staff M&E capacity will add value to the programme and validity to its data. Shelter Occupancy • Occupancy of completed shelters is not a problem; nearly all shelters are (eventually) occupied. • Apparent unoccupancy is the result of unfortunate verification timing, not inadequate beneficiary selection. • Empty units are uncommon, and are typically the result of exceptional circumstances. • Seasonal issues also might affect the timing of shelter inhabitation. • Inner floor preparation and compound perimeter wall erection also cause delays to inhabit a ‘completed’ house. • Overall, the timing of all programme phases needs more analysis. Programme Impact • Higher programme impact will result from effective beneficiary targeting and selection. • BSC perceptions that national shelter needs are greater than UNHCR programme supply apparently result in more

scrutiny during the selection process. • It is likely that poorer families would use lower quality building materials if they did not receive programme assistance. • Shelter kits appear to be reaching the appropriate beneficiaries, creating greater programme impact. • Families are building and using latrines. • Hygiene messages and their promotion might need to be strengthened. Building on its thematic findings, the Progress Report identified 25 programmatic recommendations. These recommendations were pursued under supervision of the BOK Programme Unit during the remainder of the M&E consultancy. Most of the recommendations have been taken forward and addressed directly in the recommended formats for selection and verification, in the revised data collection and reporting systems, and in the policy direction of the revised 2005 Shelter Guidelines. Figure 2 outlines the Progress Report recommendations and their subsequent follow up during the consultancy: Figure 5: M&E progress report recommendations and implementation

Recommendation Action Taken 1. Refine vulnerability groups on beneficiary selection

form and include greater detail. Include Kuchi. Vulnerability groups redefined on selection format, monthly report, and consolidated report. After national consultation, decided not to include Kuchi as a specific group.

2. Revised selection and monitoring formats should include expanded information on criteria identified during the selection process that led the BSC to agree on family eligibility for shelter assistance.

Included in the recommended format for beneficiary selection, such as questions on income insecurity, water access, family size, land status, and return status. It was agreed that most of these would not included on monitoring formats.

3. Identify whether other encashment centres are engaging in shelter beneficiary preselection, then determine whether this should be pursued or proscribed as a national strategy. If pursued nationally, develop a set of preselection guidelines to facilitate a more standardised and less subjective approach, ensuring that BSCs remain the ultimate decision making body for determining shelter eligibility.

Preselection is only pursued in the Southeast. Other regions question its usefulness and believe that the entire selection process should be managed by the BSC. A national approach was therefore agreed to be unnecessary.

4. As BSC membership is more broadly based than that of CDCs, BSCs should remain the selection body for the shelter programme. In areas where NSP CDCs are currently functional, maintain dialogue and consider including a CDC member on the BSC, but if possible do not cede decision making responsibility and thereby jeopardise the current accountability and transparency of the BSC.

All regions agree that the BSC should retain control over selection, but that CDC representatives should be invited to join the process – not only during beneficiary selection, but throughout programme implementation. This is included in the revised Guidelines.


5. With guidance from the UNHCR Gender Advisor, establish a gender analysis framework for the shelter programme to identify the different needs and impact of shelter programming for females and males. Once established, identify strategies and methodologies to better access female beneficiaries and to include more women in the selection process.

In addition to revising the Guidelines section on Gender Considerations, three additional steps have been taken to increase gender analysis (see Section 3.3, below): o Self check for respondent sex during selection and

verification o BSC female/male membership will be reported and

tracked by IP, district, province, and region o Beneficiary sex disaggregated data will be reported

and tracked by IP, district, province, and region 6. Monitoring formats should include data on excluded

and ineligible families in order to better understand the dynamics of selection and potentially identify means for either including the poorest families or pursuing other assistance possibilities.

National consultation resulted in not including this question in the revised monitoring formats because it was not needed for selection decision making. Instead, the question is part of the recommended format for beneficiary selection.

7. Include in programme monitoring data on families rejected due to landlessness. Investigate ways that these families might receive additional support in lieu of shelter.

This too was decided not to be included in formal reporting, but to remain on the recommended selection format. Like the above, a distinction was made between interesting information and necessary information.

8. Ongoing monitoring formats should be standardised across all regions for continuity and comparability at each programme level. A database designed in collaboration with the Data Section at Base Office Kabul should be developed to compile the additional information collected and to facilitate further analysis of national data.

The Monthly Progress Report was revised to include new data and to improve the clarity of the format itself, with several unnecessary columns deleted for easier use. The corresponding Consolidated Report and OIU database were similarly changed. Both reports were expanded to include an additional worksheet that provides automatically updated graphs and tables so that regions can more easily track progress and performance.

9. Backstop revised monitoring systems with the training and support needed to ensure they are used properly. This will also give an opportunity to identify M&E point people in each region and to ensure everybody recognises why information is being collected, how it is used to inform the programme, and how to apply the common methodology.

Less time was available for training than hoped. Nonetheless fieldwork in each region included office time reviewing systems and logic with staff responsible for data collection and reporting. The December 2004 shelter workshop was also used as a forum to ensure clarity and continuity among all participating shelter staff, and to review the rationale behind monitoring and Guideline revisions.

10. Develop a common methodology for remote control management and monitoring to ensure that important issues are not overlooked and that necessary data are collected.

An RCM workshop was convened in Kabul in December 2004 with staff from the East, South, Southeast, and West. Regional papers had been prepared and will be consolidated by SOG into a single RCM document. Participants agreed that while issues are similar, regional responses should remain flexible based on context.

11. Ensure that all stakeholders involved in M&E receive the support and training needed to effectively engage in the process, analyse the findings, and respond programmatically.

One of the shelter workshop objectives was to prepare participants to convene similar workshops for IP and DORR counterparts upon returning to their regions. Additional materials and translated documents were circulated to facilitate these workshops.

12. Review additional qualitative themes related to beneficiary selection, vulnerability criteria, technical standards, construction quality, occupancy, and overall impact to determine which should be included in future programme monitoring.

Most of these ideas translated into revisions or clarifications made in the 2005 Shelter Guidelines to ensure that the document matches actual practice. Data collection and reporting systems were similarly revised on the basis of this recommendation, as was the development of recommended formats for beneficiary selection and final verification.

13. Use existing data to analyse seasonal trends in returns and how they compare with the shelter programme cycle. This data can also be cross referenced with seasonal considerations including labour availability, water access, and agricultural cycles.

Section 3.2 below is based on this recommendation, which found that most shelters are completed too late in the year and that beneficiary difficulties are increased as a result. Trend analysis also demonstrates that most returns occur later in the year, so it will not be possible to complete selection earlier as would be ideal – although it is possible to have an initial batch of beneficiaries begin earlier (based on Beneficiary Backlog from the Monthly Progress Report.


14. Selection monitoring forms can include fields for when returned, when selected, when construction began, estimated completion date, actual completion date, estimated occupancy date, actual occupancy date. These data would be easy to collect and would offer better understanding into the questions of decision making and timing of occupancy. This understanding would help to schedule verification and clarify lingering concerns about unoccupancy – and could be cross referenced with seasonal data to better understand occupancy decision making.

The Monthly Report format was revised to include information on when agreements were signed and when final verification should be conducted. Further investigation into occupancy also determined that most families will in fact inhabit their shelter after building. This in turn led to the revised policy for handover, demonstrating how increased understanding can translate into revised policies. Corresponding report formats were updated to reflect this change.

15. Verify the selection calendar for each region, specifying the period of return for each year of beneficiary selection. Ensure that each cycle of selection begins with the month at which it concluded in the previous round, so that returnees are captured throughout the year.

This too was included in analysis of selection and return trends, although less formally. Most regions try to begin selection in one year with returnees who arrived after selection concluded the previous year, however not all regions maintain data on the beneficiary backlog in order to prioritise them in a new programme year.

16. Analyse existing VRF data on average family size of selected beneficiaries to determine whether five is indeed an appropriate figure to use as a programmatic reference.

Family size analysis instead concentrated on families of 8 members or less and families of 9 members or more in order to determine achievement of the minimum floor space performance standard. This also revealed that 8 has been used as an informal basis for defining “large family” in the course of selection.

17. Investigate effects of construction loans on household economy. Also investigate how much people borrow and how much they expect to repay, as some respondents suggested that many loans will never be repaid.

This is discussed in Section 4.4 below. Despite the significance of this issue it was impractical to include debt questions in revised monitoring formats, particularly as data would probably not be reliable. This should nonetheless be monitored informally in the course of implementation.

18. Determine whether annual income data would be possible to collect and whether an income threshold could be used to help target beneficiaries for greater impact.

Like debt data, this was decided too be unrealistically complicated and unreliable in the context of data collection, especially in regions dependent on remote control management.

19. Investigate what strategies beneficiaries in different regions might be using spontaneously to protect shelter materials from insects or other endemic parasites that can weaken materials. When such practices are identified, mainstream them into ongoing technical support and monitoring provided by IPs and UNHCR.

Coating wooden beams with used motor oil appears to be the most prevalent protection method in use throughout the country, although not all IPs have promoted this actively. Iron beams, however, are becoming increasingly prevalent and therefore eliminating the problem of insect infestation of roof beams; all doors and windows are iron.

20. Determine the extent to which families are repairing or reconstructing houses, and see whether current selection modalities address this distinction in all regions.

This was clarified in the revised Monthly Progress Report, with “extension” deleted from the criteria because of redundancy. Not all regions agree with the practicality of partial shelter packages for rehabilitation, in the belief that it complicates implementation, tracking, and logistics – and that they would rather increase the impact of full shelters for fewer beneficiaries than try to stretch the number of partial shelters to as many beneficiaries as possible.

21. Confirm whether there are Afghan government guidelines for agencies funding, implementing, and monitoring rural shelter. If so, ensure that programme design aligns with government standards – particularly for earthquake prone regions.

The 2003 government guidelines have been found, in both English and Dari translations, and the 2005 Guidelines updated to better reflect UNHCR alignment with the government version. Copies of the guidelines are included in the shelter compact disc circulated to all regions in February 2004.

22. Develop intermittent monitoring to capture data on market activity related to the shelter programme – particularly in areas with high concentrations of shelter beneficiaries.

WFP already collects and disseminates data on market activity and commodity price trends, so it is unnecessary to collect it separately. Market data is useful to calculate the costs of additional materials needed for construction and to better understand beneficiary expenses such as labour or water.

23. Develop guidelines for IP hygiene promotion activities and provide training needed to ensure effective implementation and communication of messages.

24. Ensure that IPs and UNHCR include female staff in the training and implementation of hygiene promotion to the maximum extent possible.

These three recommendations are the only ones not brought forward during the consultancy, as they were included erroneously in the M&E Progress Report and reflected an unclear understanding by the consultant on the boundaries between the UNHCR shelter and water programmes. Nonetheless, these issues arose during


25. Revisit the messages and methodology of hygiene promotion activities. Revise if necessary and investigate opportunities for capacity building to support better promotion.

fieldwork and the recommendations are a synthesis of the input received from regional staff. Water and sanitation guidelines are available but appear to be in limited use.

The M&E Progress Report is an important resource for the shelter programme. And while this Final Report revisits or expands on many of the same qualitative issues, the ideas and detail of the Progress Report remain relevant to the ongoing shelter programme. For a summary of Findings and Recommendations, see Annex B. The full and summary versions of the M&E Progress Report are also included on the shelter compact disc (discussed in Section 2.4 below). 2.2 2005 shelter guidelines The original terms of reference for the M&E consultancy did not include revising the shelter policy document, 2004 Guidelines for Rural Shelter. This document was revised extensively prior to the 2004 programme and included narrative explanations for all aspects of implementation, in addition to technical annexes detailing shelter materials and design. As a result of shelter M&E fieldwork it became apparent that implementation practice had begun to diverge from the policy document, suggesting that a Guideline revision would be useful. Similarly, many of the qualitative themes identified in the M&E Progress Report and included in revised data collection systems were not reflected in the Guidelines. Some of the technical annexes also required updating for the 2005 programme. Finally, the 2004 Guidelines were poorly written and of a standard below that required for external dissemination of the document. It was therefore agreed to revise the entire Guidelines in coordination with the BOK Technical Advisor. The 2005 Shelter Guideline revision became more extensive than originally anticipated, and came to represent the concrete outcome of the M&E process initiated in July 2004, in that the Guidelines represent the practical implementation of the findings and recommendations generated from the M&E process. The resulting 2005 Guidelines are a more effective policy and management document that include the following changes: • All programme documents, forms, formats, reports, and specifications are included as

annexes. The complete 2005 Guidelines therefore represent the shelter programme in its entirety.

• All annexes are updated and organised to reflect the 2005 programme. • While the original structure of the document is unchanged, all sections have been updated so

that the natural evolution of implementation strategies and styles is reflected in the document.

• More guidance has been given regarding where policies need to be adhered to directly and where regions can exercise contextual flexibility.

• The language of the Guidelines has been edited to facilitate greater access and usability; particularly by users to whom English is not a first language.

• The cleaner language also will facilitate better translations into Dari and Pashto, further increasing Guideline accessibility and usability.

As much as possible, this report is linked with the 2005 Guidelines and annexes to facilitate cross reference and to increase the overall usability of both. More significantly, the revision (including its Dari and Pashto translations) should provide improved guidance for IP and DORR counterparts, particularly in regions where remote control management is necessary. More clear and more structured guidelines, formats, and reports will help ensure that remotely managed programme areas are consistent with areas UNHCR and IP staff can access. This will also ensure that



necessary programme data remain accessible and comparable with data from all other regions nationally, whether managed remotely or directly. 2.3 Shelter logical framework A shelter logical framework was developed during a national Programme meeting in March 2004. The framework included goal, objective, and output level narratives in addition to indicators of achievement. Means of verification, however, were never identified. Similarly, it appears that the logical framework has never been used systematically to monitor or evaluate shelter programme progress. While the exercise of developing the framework was useful for participating staff to articulate key programmatic components, the fact that the document lacked means of verification meant that the framework had no practical purpose as a management or M&E tool at regional or national levels. At regional levels, a standardised version of the logical framework was distributed to each region in preparation for the 2005 LOIs. Each region’s submission included their own revision of the framework, including revised indicators. Here too it appears that the logical framework was more of an obligation than a tool, in that means of verification were never identified. Here too the logical framework appeared to offer no practical benefit. The national version of the logical framework was therefore revised with these issues in mind. More significantly, the revision was undertaken so that improved and more practical application could be made in structuring programme M&E, in addition to shaping the fieldwork that informed the progress and final M&E reports. Figure 2 presents the revised shelter logical framework.

Figure 6: UNHCR 2005 shelter logical framework

Narrative Indicators VerificationGOAL SUSTAINABILITY The UNHCR shelter programme aims to assist the sustainable reintegration of returned refugees and IDPs into their communities of origin through promotion of gradual, voluntary, safe, and gender sensitive return.

Although officially called shelters, UNHCR Afghanistan is facilitating the construction of permanent houses from which families commence reintegration and economic production upon return and occupancy. One sustainability indicator will be continued investment in house maintenance and improvement. Demonstration over time of minimal secondary displacement and improved livelihoods (e.g. NRVA findings) will also suggest sustainable shelter programme impact.

OBJECTIVE 15% of vulnerable families, primarily returnees, receive shelter assistance.

• Monthly Consolidated Report from BOK OIU* [comparison (%) of returnee and beneficiary numbers]

Beneficiaries identified as too poor to construct their shelter receive community support and/or additional financial support to complete their shelters.

• Final Verification format (#9) • Monthly Consolidated Report from BOK OIU

Intended beneficiaries occupy their shelters 1 and 2 years after completion, with few cases of reverse or secondary displacement due to lack of shelter in places of return.

• Field visits (during selection, implementation, and verification) and qualitative discussions conducted by UNHCR, IP, and DORR staff

Beneficiaries managed to improve their condition without deterioration of their livelihood status.


Impa

ct

1. The most vulnerable returnees have access to adequate shelter.

The shelter selection process includes assessment of the local environment, accessibility (water, markets, health, education), and security.


OUTPUT [#] shelters constructed and handed over according to standards and specifications of UNHCR 2005 Guidelines.

• Monthly Consolidated Report from BOK OIU

[#] beneficiary families completed their shelters & latrines and received cash grant.

• Beneficiary Selection format (#12) • Monthly Consolidated Report from BOK OIU

# / % of women and men participating in BSC decision making and implementation process.


[#] most vulnerable beneficiary families selected by BSC according to established eligibility criteria.

• Final Verification format (#5,6) • Monthly Consolidated Report from BOK OIU

Minimum floor space of 3.5m2 per person per shelter provided.

• Beneficiary Selection format (#5) [% families with 8 members or less, with 9 or more]


Perf

orm

ance

1.1 Shelters constructed and occupied in areas of high and potential return.

1 latrine constructed for each shelter. • Final Verification format (#6) • Monthly Consolidated Report from BOK OIU

The indicators were revised to be more specific and more analytical, and to ensure that they could provide a more effective point of comparison for future programme years. More importantly, means of verification were established for each indicator. Without these it would be impossible to measure the achievement of any of the indicators, and subsequently impossible to make comparative analyses across programme years. The means of verification identify by name the field and reporting formats in which specific data can be found, thereby increasing data accessibility. The revised logical framework then provided an obvious structure for the M&E consultancy, so that the logical framework itself would become more legitimate and therefore more practical, and so that the proposed evaluation would benefit from a more structured approach (because there were no original terms of reference upon which to perform a shelter programme evaluation). The logical framework also gives structure to the revised data collection and reporting systems developed for 2005. Finally, the logical framework shaped the second round of shelter M&E fieldwork (in the Northern and Western regions) and the analytical framework of the Final Report (see Section 2.5, below). The shelter logical framework therefore provides greater programmatic structure than previously. More importantly, it offers an analytical framework to assess achievement of programme performance and impact, and provides a 2004 baseline against which future evaluations can be compared. 2.4 Shelter compact disc During the first week of February 2005 a compact disc was circulated to all regions in order to increase the accessibility of shelter related documents and resources. Some regions already had some of the documents, but no single region (including BOK) had all documents readily available in one location. Distribution of the CD was therefore a simple means of contributing to national continuity in the programme and its approach. The CD contains the following materials: • 2005 UNHCR shelter guidelines and annexes • Afghan government shelter guidelines (English and Dari) • UNHCR standards and indicators • Shelter M&E progress report (October 2004, full and summary versions) • Shelter mission report by Ghassem Fardenesh, UNHCR senior site planner (December

2002). • Shelter reference materials (including papers on housing reconstruction, transitional

settlement, international shelter guidelines, and UNHCR evaluations of its shelter programmes in Rwanda and Bosnia Herzegovina)

• General reference materials (including papers on land, vulnerability, livelihoods, environmental assessment, logical framework analysis, and M&E)

• Sphere standards (2nd edition) Additional copies of the shelter CD can be obtained from the BOK Programme Unit or the Operational Information Unit. 2.5 M&E final report This document is the concluding output of 2004 shelter M&E activities. As such, it documents the process, findings, recommendations, and subsequent follow up actions undertaken during the July – January consultancy so that these remain as accessible and familiar as possible to shelter programme staff and counterparts. This report also documents the rationale behind many of the policy and procedural revisions included in the 2005 Shelter Guidelines. Here too, the intent is to provide as much insight as possible into the reasons for the revisions and the expected benefits these revisions are intended to yield. Extensive cross referencing between this report, the M&E


Progress Report, and the 2005 Shelter Guidelines has similarly been included to keep explicit the linkages and to highlight the relationship between all five consultancy outputs. This report also serves as an analytical baseline against which future evaluations can be compared. As Section 2 discusses, shelter programme analysis previously has concentrated more on quantitative data collection and presentation than on systematic qualitative interpretation – one of the principal reasons why an external M&E advisor was sought. Not only does this report attempt an evaluation of the 2002 – 2004 shelter programmes, but the evaluation itself is structured so that it can be used as a point of comparison for future evaluations. This approach was taken because no such baseline (or terms of reference) were available for the current evaluation and because future evaluations will be more rigorous with the added value of comparative data. The remainder of this document is an evaluative analysis of the UNHCR Afghanistan shelter programme, structured on the logical framework presented in Section 2.3. The evaluation moves upward through the logical framework, looking first at programme output (programme performance, Section 3) and then looking at the programme objective and goal (programme impact, Section 4). Finally, conclusions are drawn and recommendations are outlined for UNHCR Afghanistan action.


3. PROGRAMME PERFORMANCE Performance is based on output level analysis of the shelter logical framework, stating that the UNHCR shelter programme facilitates construction of shelters in areas of high and potential return. Each year has succeeded in maintaining a high completion rate, and implementation has begun to demonstrate signs of earlier completion each year. The shelter design is appropriate and consistent with Afghan standards and is composed of quality materials that are better than what families would ordinarily be able to purchase themselves. The shelter design similarly meets UNHCR and Sphere standards for minimum floor space per person for all but the largest beneficiary families. Most families are building and using latrines as part of the programme. Despite effective delivery of the above, most shelters tend to be completed during winter rather than during the ideal construction period of late March to August or September. This can make construction slower and more expensive, which can ultimately weaken household economies during the return and reintegration process. Female participation is also limited in the programme, although not for lack of effort at all levels. Gender inclusion in Afghan programming is bound by cultural and contextual considerations that affect the ability to engage female staff and beneficiaries to the same extent as males. Finally, beneficiary vulnerability is the primary criterion for programme eligibility but the concept itself is not well defined and not clearly distinguished from poverty. The shelter programme is nonetheless performing according to targets and standards, and except in areas of high insecurity has been able to concentrate allocations in regions of high and potential return. As the programme continues to improve implementation schedules and maintain its achievement rate during 2005 and 2006, families will continue to benefit from secure shelter in their places of return. Based on output indicators of the shelter logical framework, these ideas are explored in more detail in Sections 3.1 – 3.6 below. 3.1 Shelters are constructed, verified, and handed over according to standards and

specifications of the UNHCR Shelter Guidelines. • UNHCR has maintained a 99% completion rate for each year of the shelter programme. • The rate of programme implementation is improving for each year. • The shelter design is consistent with Afghan standards. • Shelter materials are of better quality than what most families would otherwise be able to

afford. UNHCR has achieved an annual completion rate of 99% for each year of the programme, facilitating the construction of 117,305 shelters between 2002 and 2004. This is summarised in Figure 3, below. (For a more detailed breakdown by province and region, see Annex G.)


Figure 7: Completed shelters

2002 2003 2004* Regional

Completed Shelters

Regional Planned Shelters

Regional Achievement 2002 – 2004

North 7,797 9,529 4,913 22,239 22,527 99% South 1,400 4,459 2,302 8,161 8,281 99% Southeast 1,500 4,648 2,887 9,035 9,100 99% East 9,113 6,723 3,808 19,644 20,120 98% West 7,292 9,587 4,960 21,839 21,898 100% Central 12,757 16,339 6,491 35,587 35,655 100% Central Highlands 0 0 800 800 800 100% TOTAL 39,859 51,285 26,161 117,305 118,381 99% Planned 40,301 51,584 26,496 Achievement 99% 99% 99% *Data are current to January 2005 and do not include shelters remaining to be completed during February. In addition to maintaining consistent annual achievement rates, UNHCR Afghanistan has managed to complete the programme earlier in 2004 than in previous years, as follows: • 2002 Completed in July 2003 • 2003 Completed in July 2004 • 2004 Completed in January 2005 This improved completion rate suggests improved implementation capacity at all programme levels, including IPs, DORR, and UNHCR. That is, the fact that the 2004 programme was completed five months earlier than those of previous years probably demonstrates an improvement in managerial and oversight capacity. (Section 3.2 below will demonstrate that earlier completion is to beneficiaries’ advantage as well.) It can therefore be expected that the 2005 programme should exhibit an even better completion rate, ideally handing over all shelters before the end of the 2005 calendar year. The standards and specifications of UNHCR shelters and shelter materials are both appropriate and of high technical quality. The National Risk and Vulnerability Assessment of WFP and MRRD reports that from their sample, approximately 95% of Afghan houses have roofs composed of wooden beams or mud bricks (configures into a dome) – with less than 1% reporting corrugated iron, concrete, or other roofing materials. Similarly, approximately 94% of houses have walls constructed of sun dried bricks or mud (pakhsa) – with only 2% having baked bricks, wood, concrete, or other wall materials. These statistics validate the UNHCR shelter design and demonstrate its alignment with standard Afghan housing. And although data are not available to compare shelter size or area with national averages, direct observation during fieldwork suggests that programme shelters are often larger and better illuminated (from more windows) than what families would otherwise construct for themselves. (Some families, however, prefer smaller or fewer windows so that houses are easier to warm in winter.) The shelter materials themselves are of higher technical quality than what families would typically be able to afford – particularly upon return from exile. The shift in 2003 to using only iron doors and window frames has met universal approval from respondents in all regions, beneficiaries and staff alike. Not only are iron materials more durable and longer lasting, but they eliminate the problem of insect infestation common in wooden materials. The same applies to the iron roof beams that have been used with increasing frequency since 2003. These are everywhere preferred to wooden beams, are much stronger, and are free of pests. Finally, the earthquake portion provided for Standard A shelters constructed in areas of possible seismic activity is further evidence of the appropriateness and technical integrity of the UNHCR shelter design.


Data for this indicator are collected as part of ongoing monthly reporting, and are included in the OIU Monthly Consolidated Report (Guidelines, Annexes 9 & 10). Annual completion rates are juxtaposed in the Monthly Comparison Report produced by the BOK Technical Advisor (Guidelines, Annex 11). 3.2 Beneficiary families are able to complete their shelters and latrines, and therefore to

receive their cash grant upon final verification. • Ongoing technical support maximises the number of beneficiaries who are able to complete

their shelters. • Revised monitoring formats will support the quality of remote control management. • Families will benefit from the new policy to approve handover upon completing the shelter

and latrine, rather than upon occupancy. • Most shelters are built after the ideal construction period (late March – August/September)

has passed. • Beneficiaries prefer to complete their shelters well before the onset of winter because of the

subsequently reduced availability of water and wages. Beneficiary ability to complete their shelters according to specifications begins with the technical support provided by IPs and UNHCR. Because the shelter design is based on standard Afghan shelters, construction techniques in most cases tend to be widely known. Nonetheless respondents everywhere have spoken highly of the ongoing support they received from IPs and UNHCR. Technical support can be particularly useful to ensure that the shelter is completed according to specifications, and that all materials will be correctly utilised – provided that materials are procured and distributed on time. In cases where construction techniques are less well known, IP assistance helps minimise technical errors. Even if construction techniques are well understood, IP and UNHCR follow up has proven effective in monitoring the ability of particular families to complete construction. That is, ongoing field visits ensure that families at risk of not completing their shelters are identified early enough in the process for preemptive action to be taken. This tends to be assurances of community labour assistance (ashar) or additional cash grants to support particularly poor families (discussed in greater detail in Section 4.2, below). IP reliability for continued field support has come into increasing question due to the greater incidence of remote control management (RCM), particularly in the East, Southeast, and South where insecurity defines entire programme areas off limits to UNHCR – and in some cases IP – staff. How can UNHCR be certain that IPs or shura members engaged in RCM are in fact providing the field presence and support required? To help answer this question, initial drafts of the revised 2005 M&E formats included IP reporting on the frequency of their field visits. However, UNHCR staff agreed nationally that it was not worthwhile to collect this data because of the low likelihood of receiving consistently honest reporting from IPs, shura, or BSCs engaging in RCM. Nonetheless, the revised formats themselves are recognised as a step toward mitigating this issue, as the greater specificity required for 2005 should provide structure and focus for all support visits. This can help to ensure that, even in cases of RCM, beneficiaries are not denied the ongoing support required for successful shelter completion. 2005 programme policy has been revised so that a “completed” shelter (including the latrine) qualifies for handover and cash grant payment; occupancy is no longer the criterion that determines handover (see Section 4.3, below). This reflects UNHCR’s increased programmatic understanding that, with few exceptions, all shelters will be occupied – regardless if occupancy occurs prior to final verification. This will ensure that: • Families will no longer be denied cash grants because of not occupying their shelters prior to

final verification.


• More shelters are registered as complete, because more shelters will be able to qualify for handover within the programme year.

More significantly, earlier completion of shelters will increase programme impact. Namely because: • More water is available • Construction periods are shorter • Wage availability is higher • Beneficiaries incur less debt • Warmer weather reduces the drying time of packed earth floors and allows for earlier shelter

occupancy However, in all programme years UNHCR shelters tend to be completed too late in the year to maximise these potential benefits. Figure 4 illustrates the seasonality of beneficiary selection during 2003 and 2004 (2002 data are unavailable), in addition to return trends for the same period: Figure 8: Selection and return trends (2003 & 2004)

Afghanistan Returns 2003 & 2004

010,00020,00030,00040,000

Jan

Feb

Mar

April

May

June

July

Aug

Sept

Oct

Nov

Dec

20032004

Shelter Beneficiary Selection 2003 & 2004

010,00020,00030,00040,000

Jan

Feb

Mar

April

May

June

July

Aug

Sept

Oct

Nov

Dec

20032004

ideal construction period

Note: 2002 monthly data are unavailable. Respondents in all regions agreed that the ideal shelter construction period begins after the Nau Roz holiday in late March, and continues until August or September (depending on region and climate). In all three programme years, however, beneficiary selection has not begun until June or July, more than halfway through the ideal construction period. This means that most families do not begin building their shelters until after the ideal period has passed, with the majority of families completing construction during the early months of winter when less water and less paid labour opportunities are available. The late construction period therefore slows the entire programme. Water shortages, for example, were cited in all regions as the primary reason for construction delays. Unless construction can begin earlier in the year when more water is available, shortages and construction delays can be expected in subsequent programme years. Respondents everywhere are borrowing to construct their shelters. Reduced wage availability during the construction period means that beneficiaries are likely to depend more strongly on loans to fulfil material and construction costs, resulting in a weakened household economy. In all regions, respondents agreed that they would have been better off building their shelters between April and September when they tend to have more money in their pockets. In most cases, this was said to be more important than the missed opportunity of building instead of seeking employment. That is, families usually can better afford to miss labour during the high availability season then they can afford to borrow later in the year when wages are scarcer. Unfortunately, it is not entirely possible to complete selection and begin construction during the ideal March – September period, because return trends demonstrate that more families return later in the year than during the first half. Figure 2, above, illustrates how returns peak later in the year. By necessity beneficiary selection must therefore be weighted toward the second half of the year. Nonetheless it would be possible to maximise water and wage availability by phasing selection and construction so that an initial batch of families in each region is able to begin early. In most cases this would begin with the beneficiary backlog that each region should be monitoring as part of its


monthly data collection and reporting. This initial batch can also include families who returned after selection was completed in the previous programme year. In some cases, BSCs reported that they informally monitored families who were rejected for assistance in previous years based apparent programme ineligibility. Occasionally they identified that some of these families were still without shelter, sometimes one or two years later, and that they were therefore considered eligible for assistance due to their demonstrated inability to manage independently. These cases can be included in the first batch of shelters so that as many families as possible can benefit from the ideal construction period. This too will help ensure that more shelters are completed each year (proportional to planned figures) and that more shelters are completed earlier in the programme year. This indicator is informed in part by questions 5 – 7 of the recommended final verification format (Guidelines, Annex 8). This indicator is also reflects the completion ratio (of total planned) calculated monthly by the BOK Technical Advisor and OIU, in addition to annual and monthly analyses of implementation progress in the Comparison Report (Guidelines, Annex 11). 3.3 Women participate in programme implementation and decision making. • Direct female participation is possible but minimal due to cultural and contextual

considerations. • Remote control management further inhibits efforts to facilitate female participation. • Efforts to include more women are sometimes limited to targeting female headed

households, rather than trying to reach female and male members of the same household during assessment and selection.

• Programmatic gender analysis has been limited to date but will increase in subsequent programme years.

The M&E Progress Report found that direct female participation appeared minimal (page 5). This section therefore will focus specifically on female participation, even though the logical framework indicator adopts a more gendered approach by stating that “women and men” should participate in decision making. Direct female participation is a Guiding Principle of the shelter programme (Guidelines, Section 3). Female participation in Afghan programmes is, however, subject to substantial cultural and contextual considerations – none of which are unique to the shelter programme. Combined with the added limitations of political insecurity, some regions (especially the East, Southeast, and South – already constrained by remote management issues) are severely restricted in their ability to include women in implementation and decision making. Even in regions where this is possible, the process nonetheless depends on staff commitment (to maintain the Guiding Principle) and the identification of appropriate ways to include women, even in fieldwork. In practice, the principle of female participation translates into three stakeholder groups: • Female beneficiaries • Female BSC members • Female IP, DORR, and UNHCR field staff When trying to reach female beneficiaries, there appears to be a tendency for staff to concentrate on potentially vulnerable groups such as widows. Although they are potentially the most vulnerable group,3 a singular focus on female headed households can distract from the need to include female members of male headed households in the programme process – especially during beneficiary selection. Even in areas where female staff are unable to work, male staff recognise the limitations of entirely male selection committees who are typically unable to access females in the community directly (unless, for example, an older shura member meets with an older widow). For the dynamics of beneficiary selection, family vulnerability cannot be assessed with full certainty 3 WFP & MRRD 2004.


if interaction is limited entirely to male household heads. In areas where female staff are able to operate, selection often includes female and male staff coordination to compare findings and agree on family eligibility for assistance. Staff agree that this is preferable and more effective. Staff everywhere recognise the value and need to include women in the programme process, and agree with the approaches outlined in the Shelter Guidelines suggesting ways to approach female participation. Each region has therefore established its own approach to including women at both field and staff levels. Evaluating the achievement of this indicator, however, is limited by an absence of data quantifying female participation at all programme levels. Three solutions have been instituted for the 2005 programme to increase gender analysis capacity: • IPs and sub offices will collect and report beneficiary data disaggregated by sex. • The number of female and male BSC members for each IP and each district will be collected

and reported. • BSC members are recommended to self-audit the number of females and males contacted

directly during selection and verification. Beneficiary data from previous programme years have been well collected and managed, but specific divisions of female and male beneficiary numbers are not possible. Staff and management opinions vary on the need for sex disaggregated beneficiary data, on the grounds that beyond profiling it does not provide information that can be specifically applied to improved implementation or programming. Other staff argue that sex disaggregated data are easy to collect and provide an important insight into the programme and its dynamics. Sex disaggregated data have the additional benefit of meeting international Sphere analysis standards for demographic beneficiary profiling while enhancing UNHCR’s gender analysis capacity. Although data for 2002 – 2004 are not available, it will be possible to analyse this indicator more effectively once 2005 data collection begins. In the meantime, returnee data can be used as a proxy for beneficiary data, with the caveat that beneficiary family composition might not necessarily be the same as returnee family composition. The following figures nonetheless can substitute for beneficiary data and provide baselines against which data from 2005 onward can be compared. Figure 9: Returnee sex by region (proxy for beneficiary sex 2002 – 2004) Female Returns Male Returns Proportion Female Proportion Male North 315,844 382,319 45% 55% South 108,205 121,210 47% 53% Southeast 59,054 77,827 43% 57% East 265,133 274,390 49% 51% West 85,666 116,897 42% 58% Central 594,909 658,113 47% 53% Unknown 50 49 51% 49% TOTAL 1,428,861 1,630,805 47% 53% Source: BOK Operational Information Summary, December 2004. Figure 10: Returnee sex summary (proxy for beneficiary sex 2002 – 2004)

Returnee Sex

Females47% Males

53%


2005 data collection specifically requires regions to report on the composition of each beneficiary selection committee. This will enable quantification and tracking of female/male BSC membership, and therefore another form of measurement for female participation in implementation and decision making. As outlined in the Guidelines (Section 3), the challenges of female participation are accepted as an unavoidable programmatic issue. Recognising these challenges, quantifying female BSC participation may help regional staff to engage more effectively with gender promotion strategies – or at least allow them to speak more specifically about the extent to which women have (or have not) been brought into the process. Finally, the formats recommended for beneficiary selection and final verification both begin with an initial question of whether respondents are female or male (Guidelines, Annexes 3 & 8). These questions are a self-check mechanism for determining the degree to which female community members are reached. The check can be conducted by field staff themselves or by IP/UNHCR staff who review BSC formats to gauge whether or not female respondents appear to be contacted directly during selection. This approach itself does not ensure that more females are included in selection and verification, but rather it allows staff to monitor themselves and their respondents and to determine whether female beneficiaries are being reached. Data for self-checking respondent inclusion are collected in the formats for beneficiary selection and final verification (Guidelines, Annexes 3 and 8, #1 in each). BSC female and male membership are summarised in the monthly and consolidated reports (Guidelines, Annexes 9 & 10), in addition to disaggregated data for female and male beneficiaries (Guidelines, Section 3; also Guiding Principle 2.2). 3.4 Vulnerable families are selected by the BSC according to established eligibility

criteria. • UNHCR classifications of vulnerability have changed during each year of the shelter

programme. • Widows, elderly, and disabled are valid as priority vulnerability groups. • Opportunities to better understand vulnerability dynamics in the shelter programme have

been missed by limiting analysis to widows, elderly, and disabled. • There is no clear distinction between vulnerability and poverty, with no recognition of how the

shelter programme might affect or reduce beneficiary vulnerability. • The selection process is focused and effective in identifying appropriate families for

assistance. UNHCR Afghanistan’s approach to categorising vulnerability for shelter eligibility has changed every year of the programme. Although this might positively reflect an evolving understanding of vulnerability and how it is assessed by BSCs, the annual category changes make difficult a comparison of vulnerability targeting across programme years. Figure 7 outlines the categories used for each year of the shelter programme.


Figure 11: UNHCR Afghanistan vulnerability categories

2002 Shelter

Programme

2003 Shelter

Programme

2004 Shelter

Programme

2005 Shelter

Programme

2004 Operational Info

Summary* • Widow head of

household • Elderly head of

household • Disabled

member of household

• Returnee (refugee)

• Returnee (IDP) • Local

• HoH widow returning refugee

• HoH widow IDP • HoH widow other • Elderly HoH

returning refugee • Elderly HoH IDP • Elderly HoH

other • Disabled family

member returning refugee

• Disabled family member IDP

• Disabled family member other

• Large family with low income returning refugee

• Large family with low income IDP

• Large family with low income other

• HoH widow returning refugee

• HoH widow IDP • HoH widow other • Elderly HoH

returning refugee • Elderly HoH IDP • Elderly HoH

other • Disabled family

member returning refugee

• Disabled family member IDP

• Disabled family member other

• Female headed household

• Elderly headed household

• Disabled headed household

• Very low income • Very large family • Other (staff

examples include orphan, minor, or chronically ill household members)

• Deported • Disabled • Extreme • Missing family

members • Medical, illness,

malnutrition • Orphan under 18

years old • Pregnant early • Pregnant late • Significant

disease • Single parent • Unaccompanied

child • Unaccompanied

elderly • Unaccompanied

female • Other

*This column is from the Operational Information Summary produced from returnee data by the BOK OIU, and is included here to illustrate the diversity of vulnerability categories currently in organisational use. UNHCR shelter vulnerability categories focus on three types of households: widow, elderly, and disabled. During 2002 – 2004 these were variously subdivided by return status, but this distinction has been dropped for 2005 because of unnecessary complexity and because return status has been addressed separately in beneficiary profiling and its reporting. Staff everywhere appear to recognise that widow, elderly, and disabled households are neither vulnerable by definition nor automatically eligible for shelter assistance; the selection process appears to recognise this distinction. It also appears clear that vulnerability is defined by the primary category, recognising that families may in fact fall within multiple categories. Without focusing on a primary category of vulnerability, it would be impossible to compare programme data because of inconsistencies of whether or not families reported multiple criteria. As priority groups, the three categories of widow, elderly, and disabled do in fact demonstrate worse poverty patterns than other groups – especially female headed households – even though not everybody in these groups is by definition poor.4 Despite the appropriateness of the main categories themselves, emphasis on these three categories has so far obscured analysis of the overall approach to vulnerability targeting. For example, Figure 8 below illustrates that in 2004 female, elderly, and disabled households account for only 23% of the national beneficiary profile. The remaining 76% are identified simply as “ordinary vulnerable” without offering any insight into why these families qualified as vulnerable and therefore eligible for shelter. Although land ownership (or access) are prerequisites for eligibility, there is similarly no assessment of whether families own agricultural land or only enough for a house – or whether agricultural land is rain fed or irrigated. It is surprising to see that not a single family in the entire country qualified as “large family with low income”, suggesting that either the category is in appropriate or that data collection has been unreliable – or both. These categories have been revised for 2005 with the expectation that the majority of beneficiary families will have “very low income” or “very large family” as their primary vulnerability group.

4 WFP & MRRD 2004.


Figure 12: 2004 vulnerability data Widow HOH Elderly HOH Disabled

Family Member

Large Family with Low Income

Ordinary Vulnerable

Total Beneficiaries

Selected North 896 240 512 0 3,352 5,000 South 160 204 28 0 1,989 2,381 Southeast 435 612 134 0 1,719 2,900 East 256 90 196 0 3,358 3,900 West 254 949 59 0 3,738 5,000 Central 421 468 173 0 5,453 6,515 Central Highlands 25 168 23 0 584 800 TOTAL 2,447 2,731 1,125 0 20,193 26,496 % of TOTAL 9% 10% 4% 0% 76% 100% Source: BOK Operational Information Unit The distinction between vulnerability and poverty is not clear in the shelter context. That is, there is no explicit definition of what vulnerability means in the context of the shelter programme. Similarly, there is no explanation of to what families are vulnerable. Are there specific risks or threats facing returnee families, or specific livelihood stresses or shocks that the shelter programme seeks to minimise? If families receive shelter on the basis of perceived vulnerability, does permanent shelter ultimately reduce family vulnerability? Although the programme specifically seeks to reach vulnerable beneficiaries, there is no ultimate analysis and no national engagement in the question of how the programme might affect or reduce vulnerability. These questions were raised in the M&E Progress Report for discussion (page 3). Programme staff agree that vulnerability is relative between regions and families, and that perceived vulnerability is one of the essential criteria required for shelter eligibility (Guidelines, Section 2.4). However the difference between vulnerability and poverty remains unaddressed. Nonetheless, it appears that BSCs in all regions are successfully targeting and reaching those families most in need of shelter assistance. Therefore it may be unnecessary to engage in the academic debate of defining criteria more specifically. In terms of the programmatic goal and objective, to facilitate reintegration through shelter assistance for families in need, selection committees appear knowledgeable and confident in their ability to manage an assessment process that identifies appropriate families for assistance and minimises allocations to families that could have established shelters independently. BSC members throughout the country (in addition to IP, DORR, and UNHCR staff) report that shelter allocations are insufficient in comparison to the perceived level of need among returnee communities. The positive outcome of this perception is that BSC members exercise greater scrutiny in selection and become more sensitive to the need to maximise the impact of their apparently low allocations. This results in more effective selection and, consequently, in greater programme impact – even if the concept of vulnerability itself cannot be defined in specific or consistent terms. To enable more accurate and more comparable vulnerability data, the 2005 categories have been revised so that all beneficiaries will meet at least one criterion as their primary vulnerability category – even if simply a family with “very low income”. With different approaches taken to classifying vulnerability in the different regions, and inconsistent quality control over IP data collection and reporting, the submitted data in previous years are unreliable. By ensuring that 100% of all families from all regions are categorised (and correctly reported) by vulnerability group, a more accurate and more reliable picture of vulnerability trends in the shelter programme will be possible. Similarly, in previous years the vulnerability statistics for households headed by widows, elderly, or disabled beneficiaries offered insight for only 10-30% of families nationally – leaving unanswered the questions of who are the other 70% of beneficiaries and why were they selected? Ensuring that vulnerability statistics represent the entire programme population (100% of


beneficiaries), UNHCR will be able to analyse, understand, and report vulnerability trends in more accurate detail. Vulnerability data are collected during beneficiary selection (Guidelines, Annex 3, #13) and are included in the monthly progress and monthly consolidated reports (Guidelines, Annexes 9 & 10) produced by each region and the OIU. 3.5 Beneficiary families construct shelters with a minimum floor space of 3.5m2 per

person per shelter. • Approximately 92% of families achieve the performance standard of 3.5m2 per person per

shelter. • Floor space performance achievement has not been assessed in previous years. • Families with 9 members or more will not achieve the floor space standard. • Regional management has the flexibility to determine whether supplemental shelter materials

should be authorised for large families in order to maintain the standard. UNHCR standards and international Sphere standards both recommend a minimum floor space of 3.5m2 per person per shelter. Although included as a performance indictor in the logical framework, no systematic analysis has been conducted in previous years to determine the extent to which this standard has been achieved. Data on beneficiary family size have been collected by some regions as part of the profiling that accompanies selection. However even in these regions it appears that these data have not been used to assess whether families do in fact have 3.5m2 of floor space per person. Although the usable floor space of a shelter will vary depending on wall thickness (brick walls common for shelter Standards A and B are at least 30cm thick, while mud walls (pakhsa) common for Standard C are usually 60cm thick) or whether the family has altered the shelter design (for example by providing additional roof beams to enlarge one or both rooms), UNHCR staff from all regions have agreed on a calculated average of 29m2 usable floor space per shelter. It is therefore possible to measure this performance indicator by separating family data into those with 8 members or less and those with 9 members or more: • Families with 8 members or less achieve the standard (29m2 ÷ 8 = 3.6m2 per person) • Families with 9 members or more do not achieve the standard (29m2 ÷ 9 = 3.2m2 per person) It is not possible to assess directly this indicator for the years 2002 – 2004 because beneficiary family size data have not been collected or reported by all regions. These data will be included in the revised reporting formats for 2005, after which it will be possible to measure this indicator more effectively (Guidelines, Annexes 9 and 10). In the meantime, it is possible to use returnee family size data as a proxy for beneficiary family size data. The data presented here are therefore useful to approximate achievement of previous years, both to asses it independently and to use as a baseline for comparison with the more accurate data available from 2005 onwards.


Figure 13: Returnee family size (proxy for beneficiary family size 2002 – 2004)

Family Size ≤ 8 members ≥ 9 members Total Returned Families

North 111,886 9,324 121,210 South 35,403 2,222 37,625 Southeast 18,601 2,297 20,898 East 89,020 6,591 95,611 West 34,548 2,904 37,452 Central 200,722 18,273 218,995 Central Highlands 2,679 146 2,825 Unknown 12 3 15 TOTAL 492,871 41,760 534,631 % of TOTAL 92% 8% 100%

Source: BOK Operational Information Unit Figure 14: Families achieving floor space performance standard (return proxy for beneficiary data 2002 – 2004)

Families with 3.5m2 Floor Spaceper Person per Shelter

Yes92%

No8%

Using returnee family size as a proxy for beneficiary family size, it appears that approximately 92% of families have constructed shelters that achieve the performance standard of 3.5m2 per person per shelter. Again, although this is an estimate it is nonetheless useful both to approximate achievement during 2002 – 2004 and to provide a comparative baseline for future analysis. Here it is important to note an inconsistency in the analysis of family size used in the shelter programme – not because it is problematic, but only because it could create possible confusion in the future. Although an 8 member family achieves the 3.5m2 performance indicator, 8 also has become (through habit of practice) the basis of a family qualifying as a “very large family” under the vulnerability categories used for beneficiary selection. These criteria are not contradictory, and are highlighted here simply to avoid possible confusion. This distinction was discussed during the December 2004 shelter workshop, with the conclusion that regional offices have the flexibility to approve additional shelter materials for families of 9 members or more if they prioritise maintaining the performance standard for as many families as possible over providing as many families as possible with shelter. Family size data are collected as a mandatory part of the beneficiary selection (and profiling) process (Guidelines, Annex 3, #5). These data are included in the Monthly Progress Report and Monthly Consolidated Report, summarised as total number of families with 8 members or less and number of families with 9 members or more, to yield “yes/no” analysis (%) of performance indicator achievement (Guidelines, Annexes 9 & 10).


3.6 Latrines are built as part of every completed shelter. • Families are building and using latrines in most locations. • Requiring completed latrines to qualify for shelter handover is effective to maximise the

number of latrines. Latrines are an inherent part of the UNHCR shelter programme. Every beneficiary family is required to build a latrine as part of their shelter, and finishing the latrine is required before receiving the final cash grant. During final verification and handover, “completed” shelters mean that latrines also have been completed. Unfinished latrines – incomplete shelters – are ineligible for handover and cash payment. This appears to be an effective means of ensuring that most latrines are in fact built, although data are unavailable to quantify exactly how many shelters have been reported as unfinished as a result of incomplete latrines. Despite the potentially punitive approach to latrine promotion, beneficiaries in most cases appear receptive to the requirement and are willing to accept this conditionality of shelter assistance. Latrines therefore appear to be an appropriate and acceptable component of the shelter programme, demonstrated by the fact that in nearly every single location visited families were building and using their latrines. For 2005 final verification, latrine completion is part of data collection for completed shelters. That is, all shelters certified as complete will by definition have a completed latrine too. Although not centrally analysed, sub offices will follow up “incomplete” shelters to determine whether delays in shelter construction or latrine completion are the reason for not yet handing over the shelter. Hygiene promotion activities are included in the UNHCR water programme, but not in the shelter programme. There is nonetheless a belief among some staff that many beneficiaries would have undergone various levels of health and hygiene promotion while living in Pakistani or Irani refugee camps. Similarly, many of those who lived in towns and cities during exile would have developed improved hygiene habits. While some staff feel that hygiene promotion would be a desirable component of the shelter programme, they recognise that it is secondary to the primary purpose of the programme. Data for this indicator are collected as a required part of final verification (Guidelines, Annex 8, #6), and are reported monthly as part of implementation Status 5, “shelter and latrine completed”.


4. PROGRAMME IMPACT Impact is based on objective and goal level analysis of the shelter logical framework, which states that the most vulnerable returnees have access to adequate shelter. UNHCR has on average surpassed its target of providing shelter to 15% of vulnerable returnees during 2002 – 2004. Some of the poorest families, however, have been missed along the way as a result either of their inability to mobilise sufficient human and financial resources for construction or their inability to access land upon which to establish themselves. This slippage may to some extent be unavoidable, although implementing partners and UNHCR alike have so far minimised the number of poor families unable to participate through mobilisation of voluntary community assistance (ashar) and through approval of supplemental cash grants for the poorest families. A distinction is also made between shelter and housing, using the argument that UNHCR is in fact facilitating the construction of permanent socioeconomic bases from which families pursue livelihood strategies. Acknowledging the organisational reasons why the term shelter must prevail, this distinction is nonetheless important when considering overall programme impact. Apparently minimal secondary displacement following shelter occupancy further supports that families are indeed better grounded in their places of return as a result of establishing a shelter. It therefore appears certain that most families who invest in and build a shelter will ultimately inhabit the unit, regardless of whether occupancy takes place within a particular programme year. Once living in their shelters, families can begin the process of social and economic reintegration – although the ability to improve livelihoods over time will be determined in part by family access and control over land, labour, and water. This is an unfortunate irony, as the vulnerability that made families eligible for shelter in the first place is likely to reflect similar issues of access and control of these essential resources. The most lingering question of household impact is whether the apparently high incidence of beneficiary debt for material and construction costs can be absorbed by the household economy. Although not quantifiable, it appears that beneficiaries everywhere are borrowing to fulfil their share of programme contribution costs. The programme ethos of self help housing is both constructive and appropriate, but it remains to be seen how debt might undermine household ability to improve livelihoods over time. The shelter selection process varies by region but is nonetheless consistent in its guiding principles (from the Shelter Guidelines) and the effort to include diverse stakeholders in an inclusive, transparent, and informed process. Regional flexibility is essential to maintaining appropriate implementation strategies, and has contributed to an effective selection process that identifies appropriate families and ensures their access to shelter. Based on objective indicators of the shelter logical framework, these ideas are explored in more detail in Sections 4.1 – 4.5 below. 4.1 A minimum of 15% of vulnerable returnees receive shelter assistance from UNHCR. • Overall, UNHCR has surpassed its target by reaching approximately 22% of returnees with

shelter. • 2004 achievement was only 8% as a result of the high number of returns. • Staff feel that although usually succeeding to meet targets, annual shelter allocations are

insufficient. • Improved beneficiary profiling by type of return will help determine whether 15% is an

appropriate target for future years. The following two tables compare the numbers of returns with the number of shelter beneficiaries. Figure 11 presents the data by year; Figure 12 presents the data by region. According to the data,


UNHCR has succeeded in providing 22% of 2002 – 2004 returnees with shelter assistance, thereby surpassing its impact target of 15%. Figure 15: Returns and beneficiaries by year (families)

Returns Beneficiaries Achievement* 2002 134,178 39,859 30% 2003 81,180 51,285 63% 2004 318,633 26,161 8% TOTAL 533,891 117,305 22% Source: BOK OIU. * These proxy figures are based on returnee data only, and are used to approximate a trend of selection achievement. Data do not include IDPs, spontaneous or forced returns, or non-returnee shelter beneficiaries. Figure 16: Returns and beneficiaries by region (2002 – 2004 families)

Returns Beneficiaries Achievement* North 120,973 22,239 18% South 35,091 8,161 23% Southeast 20,883 9,035 43% East 95,452 19,644 21% West 38,510 21,839 57% Central 215,400 35,587 17% Central Highlands 7,622 800 10% TOTAL 533,891 117,305 22% Source: BOK OIU. * These proxy figures are based on returnee data only, and are used to approximate a trend of selection achievement. Data do not include IDPs, spontaneous or forced returns, or non-returnee shelter beneficiaries. Available data for this indicator are misleading and do not represent the actual allocation of shelters to beneficiaries. Firstly, these data are exit data from Pakistan and Iran – not beneficiary data from the shelter programme. Secondly, these data represent only facilitated returns – not IDPs, spontaneous or forced returns, or non-returnee shelter beneficiaries. Nonetheless the data serve to illustrate a trend and to provide a proxy indication of whether or not UNHCR is achieving its target. Twenty-two percent might therefore not be accurate, but provides enough confidence to suggest that the target is indeed met. It is difficult to verify whether or not 15% itself is an appropriate target, or whether it should be adjusted. Staff nationally suggested that shelter allocations are insufficient when compared with the perceived need, and that more shelters should be allocated to every region. These observations could be the result of the beneficiary selection process, when relative comparisons must be made to decide who should receive assistance and who should not, and when staff are forced to control their desire to provide as many shelters as possible to as many families as possible. More likely is that staff recommendations to increase shelter allocations are the result of the apparently insufficient target of 26,000 shelters for 2004. Referring to Figure 11 above, the overall achievement for 2004 was only 8%. While the programme has surpassed its standard of 15% for the whole of 2002 – 2004, it was nonetheless not met during 2004 itself. These data are of course proxies, but nonetheless serve to highlight that 2004 allocations should have been higher in order to maintain consistent annual achievement of this programme standard. Although staff feel that allocations are low, this nonetheless results in a more rigorous selection process because BSC members everywhere feel that they must be extra careful to select appropriate beneficiaries when the number of possible shelters appears low. Therefore even if 15% might be an insufficient target, it nonetheless serves to ensure that the importance of the selection process is respected. From 2005 onwards it will be possible to measure this indicator more precisely as a result of including beneficiary return status data in selection profiling and reporting. Every beneficiary family will be grouped into one of four return categories:


• Assisted refugee return (with VRF) • Non-assisted refugee return (without VRF) • Returned IDP (with or without IDP VRF) • Non-returnee (local population) Dividing all beneficiaries into these categories will enable a precise measurement of which groups receive what portion of the total shelter allocations. In addition to better understanding selection dynamics and beneficiary profiles, more precise data will enable a better analysis of whether 15% is in fact an appropriate target or whether this will need to be revised for the 2006 programme. Data for this indicator are consolidated by the BOK Operational Information Unit. Returnee statistics are the result of exit data from Pakistan and Iran, and are compiled monthly by the OIU. Monthly data for beneficiary numbers are extracted from regional Monthly Progress Reports by the OIU. The final comparison of returnee and beneficiary data is presented as part of the Monthly Consolidated Report (Guidelines, Annex 10). 4.2 Beneficiaries too poor to construct their shelters receive community support (ashar)

and/or additional financial support (from IPs and UNHCR) to complete their shelters. • Despite genuine efforts, there is a thin layer of vulnerable families who are too poor to

participate in the programme or who cannot mobilise the required resources. • Where possible, mobilising ashar contributions helps to maximise the number of families who

complete their shelters. • Ashar is not always possible or available, as some families cannot afford to forfeit paid labour

opportunities. • Additional cash grants are also effective to support the poorest families, but not all regions

utilise this facility. • Although families who might not be able to complete their shelters can sometimes be

identified in advance, such cases are sometimes not apparent until implementation has begun.

• Some layer of slippage appears to be unavoidable, but can however be minimised by the BSC, IP, and UNHCR.

Although the programme seeks to provide shelter for the most vulnerable families, IP and UNHCR staff in all regions agree that there is a thin layer of returnee families who are too poor to participate in the programme or who cannot mobilise the labour or funds needed to construct a shelter. Both the Guidelines and established practice demonstrate an effort to include all such families in the programme, particularly during selection when the BSC must determine whether or not families are eligible – and capable – to participate in the programme. Assistance to families unable to build shelters tends to be on a case by case basis, often managed by the BSC and IP because they are most often in direct contact with beneficiaries and shura. For example a BSC might negotiate with shura members that a particular widow who cannot afford to pay labourers for construction will be assisted by members of the community on a rotational basis of one day per person. Similarly, the BSC might agree with other beneficiaries that nobody in the community will receive roof beams until all families have reached the lintel level of construction, even if that means some families will need the assistance of voluntary ashar labour to ensure that all families reach the lintel level on time. In some cases, ashar labour contributions would happen even if the BSC did not facilitate the process. However earlier programme documentation has presented an idealistic portrait of ashar, suggesting that families always are able to exchange labour freely. This, in turn, has fostered the impression among some staff that ashar is a viable possibility everywhere. Ashar is however quite complex, and raises issues of obligation and reciprocity in communities. More simply, many


beneficiaries explained that they were unable to offer ashar to their neighbours because they could not afford to miss paid wage opportunities. In some cases, ashar was given for one day by each person to minimise lost wage opportunities available elsewhere. In other cases, respondents said they were too poor for ashar and needed to earn for themselves, in which case nobody received ashar. Small groups or clusters of families sometimes agree to help each other. Not only does this ensure that everybody is able to build their shelter, but respondents suggested that collective labour helped to speed the construction period – resulting in less lost wages from outside labour and a faster occupancy period. This cooperation is sometimes, but not always, the result of BSC mobilisation. Focus group discussions during fieldwork often highlighted that ashar labour is never a guarantee, and that some communities are more likely to help one-another than others. IP and UNHCR programme staff cannot assume that the poorest families or those physically unable to construct will necessarily be assisted. Although the incidence cannot be quantified, staff everywhere agreed that despite efforts by the BSC, IP, and UNHCR there were always examples of families who either refused to participate in the programme or who were accepted but later withdrew – all because of an inability to complete their shelter. As a result, the programme guidelines allow for additional cash grants for families who might otherwise be unable to afford construction, labour, or material costs. This is an effective approach to reduce programme dropouts, especially in cases where ashar is either impossible or insufficient. Not all IP and UNHCR staff, however, appeared clear in their familiarity or understanding of this additional cash grant policy. Some staff knew and applied the policy, whereas other regions were apparently unfamiliar with the possibility of allocating additional cash. Finally, still other regional staff knew the policy but felt that giving more cash to particular families would ultimately create conflict in their programme areas and therefore did not apply the policy. Beneficiary focus groups were equally split on whether or not additional grants would be possible. Some said that this would create unmanageable conflict, while others suggested that if negotiated transparently conflict would be avoided. As a result of the different approaches used by different regions, two measures have been included in the 2005 programme to reduce the number of particularly vulnerable families who are unable to benefit from shelter assistance. The first is that the recommended beneficiary selection format includes a question on whether or not ashar will be necessary for each particular family to complete its shelter (Guidelines, Annex 3, #16). Although this is not always obvious at the outset of the programme and becomes clear only once construction has started, adding this question might help to identify some of these cases before construction delays become apparent and problematic. In other cases, the BSC will need to remain alert during implementation to whether particular families are at risk of not completing their shelter. The second approach taken to minimise the number of families who cannot afford construction was to rewrite relevant sections of the Guidelines to ensure that the recommended extra cash grant policy is relevant and clear. The topic was then discussed as part of the shelter workshop held at BOK to ensure staff are clear on the purpose and policy of extra cash grants. Extra grant payment in 2005 will be part of the monthly reporting process, so that managers will have monthly data available to track use of this facility. Despite the efforts of BSC, IP, and UNHCR in mobilising ashar and authorising additional cash grants to ensure that as many families as possible complete their shelters, some of the poorest families still slip through. Staff are unanimous that some degree of slippage is unavoidable and must be accepted as an inherent part of the programme – particularly when trying to work with the poorest returnees. The issue of landlessness will similarly continue to remain problematic when trying to ensure the poorest families can construct shelters. The selection and implementation processes nonetheless are proving effective to minimise the number of families disqualified by


landlessness and the number of selected families unable to complete construction without additional assistance. Data collection for this indicator only quantifies the number of families (per district/province/region) receiving supplemental cash grants – not the incidence, frequency, or dynamics of ashar. UNHCR, IPs, and DORR staff will therefore be responsible to include this in the qualitative observations and discussions conducted during selection, implementation, verification, and management. 4.3 Programme beneficiaries occupy their shelters 1 and 2 years after completion, with

minimum cases of secondary or reverse displacement due to a lack of shelter in places of return.

• The shelter programme reinforces legal and social reintegration benchmarks such as access

to shelter and officially affirmed land ownership. • Although officially called shelters, UNHCR is facilitating the construction of permanent

houses that provide a foundation for family reintegration. • Even in cases of labour migration, the family remains in the shelter while male members

leave in search of work. • Secondary or reverse displacement appears to be minimal. • Secondary displacement will usually result from insufficient wages or water – not from a lack

of shelter. • If families are further displaced, they nonetheless maintain control over the shelter (or its

materials) as an important household asset. • Families who invest in and construct a shelter will in almost all cases occupy it. Legal and social reintegration benchmarks, 5 such as access to shelter or officially affirmed land ownership, are reinforced as part of the programme. Land ownership (or access), for example, is a prerequisite for eligibility (Guidelines, Section 2.3). Beyond this, however, it is difficult to measure or qualify the extent of a particular family’s reintegration. This section therefore concentrates on continued occupancy without onward displacement as an indicator of reintegration. Although officially called shelters, UNHCR Afghanistan in practice is facilitating the construction of houses. In the 1999 UNHCR internal evaluation of the Bosnia Herzegovina housing programme, the writers highlight a similar discrepancy. They emphasise that shelter refers to infrastructures of temporary nature, generally short to medium term (3-8 months), with physical protection from the elements, minimum protection of private belongings, and minimum privacy. In contrast, houses are part of longer term reintegration, enabling returnees to stage their return and to begin the process of economic, social, and physical recovery. It is therefore important to recognise that while called shelters, the Afghanistan programme is providing permanent structures as a foundation for returnee reintegration. Family willingness to invest in establishing their shelter demonstrates their commitment to establish themselves upon return. In addition to the shelter materials and cash grant provided through the programme, families everywhere tend to invest an additional $200 – $400 on materials or labour. Assuming that appropriate families have been selected in the first place, this represents a substantial investment for returning families. Fieldwork in all regions established that although in some cases this additional investment was difficult to mobilise, families recognised this as an essential and worthwhile part of their reintegration. As such, respondents everywhere said that they had no intention to leave the shelter once completed, or to again be displaced to other areas. Even in (frequent) cases where male members left to seek labour opportunities elsewhere, the family unit remained in the shelter, with males often returning for seasonal labour opportunities. This demonstrates how shelters function as the social and economic anchor of families. In other cases where respondents complained of the limited availability or viability of wage opportunities, 5 Pablo Matteu, personal communication (July 2004).


they nonetheless concurred that having the shelter prevented them from having to consider further displacement. That is, without the initial reintegration provided through shelter these families might not have remained in their places of return. An exceptional example of families unable to remain in their place of return is found in Yak Ataz, Faryab – where all 120 returning families received shelter assistance. Secondary displacement was therefore not the result of a lack of shelter in the place of return. Instead, the secondary displacement of at least 15 families was the result of insufficient work and water. Even families who remained were resorting to dismantling parts of their previously completed shelters, selling a window or door to meet immediate cash needs. Two points are important to recognise in the above example. First is that although the indicator specifies that secondary displacement will not result from a lack of shelter in places of return, it is possible that families will resort to secondary displacement for other reasons – usually insufficient water and/or work. Cases of secondary displacement nonetheless appear minimal. The second point to recognise in the Yak Ataz example is that although onward movement is perceived by UNHCR as potentially negative, the few cases found during fieldwork suggested that some of these families were able to sell or rent their shelters. In other cases, families dismantled their shelters and brought the materials with them in order to reconstruct elsewhere. In both scenarios, families pursuing onward movement have displayed their use and control of shelter as a primary household asset, subject to manipulation as the family sees necessary. In terms of programme impact, then, the shelter programme nonetheless enabled families to create and control an asset base, thereby suggesting a continued contribution to reintegration. In early 2004, concerns were raised regarding the apparently low rate of shelter occupancy. Visiting donor representatives (particularly ECHO) saw shelters that appeared to be complete but were nonetheless uninhabited, and therefore questioned the quality of beneficiary selection and the overall appropriateness of the shelter programme. In response to these concerns, the M&E Progress Report analysed occupancy issues and concluded that all shelters will in fact be occupied and that related concerns were probably invalid (Progress Report, pages 11-13). Families will not invest time and resources in shelters that they do not value as part of their reintegration and that they do not intend to occupy. Occupancy therefore appears to be a question of timing. From the perspective of beneficiaries, the question of when to occupy can depend on seasonal issues such as water or wage labour availability. Occupancy can also reflect cultural considerations such as purdah and corresponding shelter boundary walls that provide families with privacy and enable females to move freely within the compound. From the perspective of UNHCR, the timing of occupancy and the perception that shelters were being built but not occupied are the result of final verification bound by programme schedules. Previous verification failed to capture qualitative issues such as families who used all their money during shelter construction and were therefore trying to mobilise more funds to build a boundary wall before moving into the shelter. Verification also did not recognise that packed earth floors can take up to two months to dry properly – particularly when built later in the year when temperatures are lower. As a result of analysing occupancy and reaching national consensus on its dynamics, the 2005 programme was revised to allow cash payment and shelter handover upon shelter completion, not upon occupancy. This recognises the conclusion that nearly all families who build will occupy their shelter, and therefore gives families greater flexibility in the timing of their relocation and ensures that fewer families will be denied cash grants because of not yet inhabiting their completed shelters when final verification is conducted. Families will benefit from the increased flexibility and greater likelihood of receiving their cash grant, while UNHCR will benefit from more timely handover of shelters and completion of programmatic targets. While this indicator therefore appears to be successfully achieved, data beyond the M&E fieldwork to support this conclusion are not being collected or formally monitored. To ensure that future


evaluation of this impact indicator is possible UNHCR, IPs, and DORR staff should be responsible to include the issues outlined above in qualitative observations and discussions made during routine programme selection, implementation, verification, and management. 4.4 Beneficiaries are able to improve their condition after returning and settling, without

deterioration of their livelihood status. • Household ability to improve conditions will depend on accessibility of land, labour, and water

– or some combination thereof. • Shelters are facilitating reintegration through provision of a family base from which livelihood

strategies can be pursued. • Family ability to remain rooted supports their power to maintain the social and economic ties

that further support reintegration. • Some families have experienced a decrease in their livelihood status since returning, the

result of lower wages and higher commodity prices. • Debt incurred during shelter construction might be undermining household economies and

resulting in less available resources for household consumption. Family ability to “improve their condition” means that their livelihood status will increase over time. In the context of the shelter programme, this suggests that establishing a permanent socioeconomic family base – a shelter – will facilitate the pursuit of household livelihoods, understood here as:

The capabilities, assets (including both material and social resources), and activities required for a means of living. A livelihood is sustainable when it can cope with and recover from stresses and shocks and maintain or enhance its capabilities and assets both now and in the future, while not undermining the natural resource base.6

As the UNHCR shelter programme concentrates on rural areas, agriculture will be a primary source of household livelihoods – whether directly for households farming their own land or indirectly for households engaged in agricultural day labour.7 The ability to improve livelihoods will therefore reflect access and availability of land, labour, and water. To some extent, then, improved family conditions will be outside programme control, as these will usually affect why families qualify as ‘vulnerable’ in the first place. In varying degrees this type of assessment is conducted informally during the targeting and selection process (see Section 4.5, below), Nonetheless, UNHCR shelters in most cases are facilitating reintegration through provision of a family base from which livelihood strategies can be pursued. Without access to shelter families would experience continued instability, would remain displaced, or would be more dependent on external assistance to meet household needs. Even in the frequent cases where family members continue to engage in labour migration, a permanent household base allows for (usually male) members to pursue outside work while the family remains. This can also allow families to diversify their livelihood strategies, as one member can seek outside opportunities while another pursues work locally. Fieldwork has also established that labour migrants tend to return to the family base during periods of greater wage availability at home, thereby further increasing family options. The ability of families to remain in the shelter also supports their power to maintain the social and economic ties that further support reintegration and that often provide a network to see them through shocks and stresses. Permanent shelters facilitate these processes and increase family options.

6 Chambers and Conway 1992, cited in Pain and Lautze 2002. 7 WFP & MRRD 2004.


Some families have reported a decrease in their livelihood status since returning. Not only are salaries typically lower in Afghanistan than Pakistan, for example, but basic staples can be more expensive. A family in Faryab reported that one seer (7 kilograms) of wheat locally costs Afs 80 ($1.78) but that in Pakistan the same amount would cost only Rs 30 ($0.46) – nearly one-quarter the cost. And despite a near universal response that beneficiaries preferred to be in their own country rather than living in exile, even if this requires a decline in living standards, families suggested that after returning and constructing a shelter they can no longer afford secondary displacement. That is, even if these families decide to pursue improved livelihoods through secondary displacement, this option is no longer affordable or viable. Families have nonetheless accepted reduced income as part of their desire to return. What is less clear when analysing beneficiary livelihoods is how household debt incurred during the construction process might undermine the household economy, and therefore undermine family ability to improve conditions over time. One of the most consistent themes during fieldwork in all regions was that beneficiaries everywhere are borrowing to meet cash needs for materials and labour, with beneficiary financial contributions typically totalling $200 – $400 depending on context. Of course, the situation is similar throughout the world, that families require credit to establish a home, and is no different in Afghanistan than it is anywhere in Europe for example. What remains unclear in the Afghan shelter context, however, is whether families are in fact able to absorb the debt they incur as an inherent programmatic side effect and whether debt will slow household economic recovery and limit families’ ability to improve their conditions. In many cases it appears that loans will never be repaid, because families might never be able to meet such financial obligations and partly because such loans were never expected to be recovered in the first place (further indication of the importance of social networks, ashar cooperation, and the complexities of community dynamics). Such loans tend to be from relatives who are less likely to exert repayment pressure. Said one respondent: “If I have nothing, how can I repay my loan?” Other families report having already serviced their loan or report making payments – usually during periods of increased food and wage availability. Neither fieldwork nor discussions with IP and UNHCR staff have revealed beneficiaries who were forced to vacate their shelter as a result of unpaid debts incurred during the programme. This suggests that creditworthiness is an indicator of beneficiary viability, that without access to credit families would in many cases be unable to participate in the programme (unless they had access to ashar or additional cash grants; see Section 4.2, above). No matter how able to absorb debt a household might be, it is nonetheless evident that families are borrowing to complete their shelters and that debt servicing ultimately results in less money available for household consumption. Based on a recommendation in the M&E Progress Report (#17), debt monitoring was initially included in the revised data collection system proposed for 2005. During the revision and national consultation process, however, it was ultimately decided that although interesting and worthwhile, debt monitoring was more complicated and less reliable than the more output oriented data already collected. That is, staff suggested that beneficiaries might likely overstate their debt in hopes of receiving additional UNHCR support and that any attempt to assess programme related debt nationally would be therefore invalid. Even if not formally included in programme monitoring, debt incidence is prevalent enough that staff should maintain awareness of the issue. Similarly, improved livelihood status could also be assessed through positive changes in meal frequency, caloric intake, increased assets and savings, and increased household consumption – although these too are outside the remit of the current data collection methodology. Better understanding rural credit, particularly in the context of programmatic borrowing, would help staff and management better understand programme dynamics and impact, and can therefore be included informally in the qualitative observations and discussions conducted during selection, implementation, verification, and management.


4.5 The shelter selection process includes assessment of the local environment, accessibility (water, markets, health, education), and security.

• While the selection process demonstrates consideration of environment, accessibility, and

security a standard assessment methodology is difficult to standardise due to the regional variety of the shelter programme.

• The ability to identify potential water shortages before they cause construction delays can facilitate more effective implementation.

• More integrated programming with shelter and water could add value to each intervention. • Security assessment has influenced the shelter allocation process – balanced by the need to

target areas of high and potential return. • Accessibility of markets or facilities such as education or health will be part of beneficiaries

decision to return, but should nonetheless remain included in regional assessment of viable shelter locations.

Of all the indicators included in the shelter logical framework, this is perhaps the most difficult to assess and the most difficult to establish a comparative baseline for future evaluation. These difficulties are due mainly to the variety of regions and programmatic contexts, and the different approaches adopted by sub offices as a result. The assessment process is therefore more difficult to standardise. And unlike all of the other logical framework indicators, this one was developed after the data collection and reporting systems had been revised and therefore was not included in the methodology. The shelter selection process nonetheless demonstrates some degree of consideration of local environment, accessibility, and security. Water availability is the only one of these themes that is explicitly recognised as part of the recommended format for beneficiary selection, with questions investigating availability of water for construction and for domestic use (Guidelines, Annex 3, #17 & 18). Most cases of construction delays are the result of water shortages. By including this question in beneficiary selection, it is possible that BSCs can identify potential water delays before they occur and that they can identify possible solutions in advance (such as arranging for tankers to deliver water when communities reach particularly water intensive construction phases). Household water availability is also recommended during beneficiary selection so that the BSC can ensure families are establishing shelters in viable locations. Although this is not formally included in the recommended selection format, BSCs everywhere demonstrated attention to this issue. In fact, throughout the country respondents suggested that shelter programming should be linked more closely with water programming. Water availability is a critical factor in whether households are able to remain in their shelters without further displacement. Similarly, the most consistent complaint heard during all fieldwork was that households struggled to find sufficient water. However UNHCR’s water programme is currently independent of the shelter programme, with opportunities for integration not yet explored entirely. IP, DORR, and UNHCR staff everywhere agreed that more integrated programming would add value to water and shelter interventions alike. Insecurity has influenced the locations in which shelters are allocated. While trying to prioritise areas of high return, this has been balanced with consideration of whether or not it will be possible to implement where insecurity limits field access. The Southern region, for example, in 2004 was forced to redistribute shelters designated for Uruzgan province because insecurity meant that neither UNHCR nor IP staff would have been able to access programme areas. The Southeast and East have also included security assessment as part of their shelter planning, with the Southeast in particular dependent on remote control management as a result. Although the other regions are less restricted in their access as compared with the East, Southeast, and South they too remain engaged in ongoing security assessment and monitoring as part of their programme planning and shelter site selection process. It appears that assessment of market accessibility or education and health availability are of less immediate concern than water or security considerations. Beneficiaries are certainly aware of these issues, and during fieldwork are quick to highlight shortcomings. BSCs and field staff are


similarly aware of these issues, despite that they are not explicitly included in the selection format. BSCs tend to be aware of the responsibilities they carry, and in all cases observed appear to take their role seriously. Like IP and UNHCR field staff, everybody engaged in the shelter selection process will need to maintain a personal engagement with these issues. Even if market access, for example, is not included in formats staff should nonetheless be aware of the entire operational environment. Toward this, staff can continue to engage informally in conversation and situational analysis throughout the project cycle, regardless of whether or not specific data are collected to qualify these issues more specifically. How regions decide to respond to particular concerns (water shortages, insecurity, lack of labour opportunities, etc) should remain flexible to ensure that each region can draw upon its collective experience to identify appropriate strategies. Data are not being collected or formally monitored. UNHCR, IP, and DORR staff will therefore be responsible to include these issues in their qualitative observations and discussions conducted during selection, implementation, verification, and management.


5. RECOMMENDATIONS The shelter logical framework (see Figure 2) contains a statement at Goal level on sustainable programme impact:

Although officially called shelters, UNHCR Afghanistan is facilitating the construction of permanent houses from which families commence reintegration and economic production upon return and occupancy. One sustainability indicator will be continued investment in house maintenance and improvement. Demonstration over time of minimal secondary displacement and improved livelihoods (informed by sources such as the National Risk and Vulnerability Assessment conducted annually by WFP and MRRD) will also suggest sustainable shelter programme impact.

Based on the experience of beneficiaries from 2002 and 2003 and the intentions of those from 2004, we can expect that the shelters established through the UNHCR programme will provide a social and economic base for returnee families. As such, families will continue to invest in the maintenance and improvement of their shelters while pursuing livelihood strategies with few cases of onward displacement. The UNHCR shelter programme is an appropriate and effective intervention that assists the sustainable reintegration of returned refugees and IDPs into their communities of origin. In fact, the programme appears to be running more smoothly than the impression given during initial M&E briefing in July 2004. Occupancy in particular was a primary concern at the time, with donors suggesting that an unspecified “large majority” of shelters were unoccupied and that programme relevance and selection effectiveness were questionable as a result. We now know that this is not the case. However the example illustrates how an ‘emergency’ intervention in a post conflict environment can overlook ongoing qualitative analysis in the course of daily and annual implementation demands. Some of the ideas presented in this paper are less the result of original thinking by the M&E consultant than they are the product of consultation and analysis presented in synthesised form. Similarly, some of the conclusions documented here have already been reached by individual beneficiaries or staff, but were never distilled into a larger national perspective – primarily because everybody is thoroughly engaged in ongoing implementation. The M&E process has therefore worked within the framework of existing policies, formats, practices, and habits in order to increase the effectiveness of systems that already worked and to ensure acceptability of revisions. As many recommendations as possible from the M&E Progress Report, for example, were carried forward into updated policies and approaches, revised systems and formats. This approach helped to ensure that M&E lessons were carried forward into practice, rather than risking the all too common outcome of an external consultant leaving behind a set of recommendations that might never be brought into action. Looking forward, the continued investigation of the qualitative and quantitative findings presented in this report have to some degree been systematised already. The 2005 Shelter Guidelines were the first step in putting ideas into practice. The revised data collection and reporting systems, particularly the progress and consolidated reports generated monthly, are an even more direct representation of the ideas contained in this report. Proposed next steps are therefore limited to continued use and refinement of the changes already instituted through a national consultation process and to inculcating within individual staff and stakeholders the habit of ongoing analysis. The recommendations outlined below are therefore minimal, limited primarily to the ongoing engagement and maintenance of M&E processes underway. Priority and responsibility for the recommendations is for the Programme Unit, Operational Information Unit, and regional heads of office, programme, and field to determine. IP and DORR counterparts similarly would benefit from being included in the implementation of recommendations.


1. Complete shelters as early in the year as possible, and consider whether it is possible to implement in batches that begin with the beneficiary backlog from the previous year.

Late March to August/September is the ideal construction period, but most UNHCR shelters are completed later in the year when water and wages are more scarce and when families must therefore borrow more money to complete construction. Earlier completion minimises negative impact on beneficiaries and increases annual programme achievement. 2. Internally clarify M&E roles and responsibilities at BOK and regional offices. Although an external M&E consultant can dedicate time entirely to programme analysis, their departure can create a vacuum in which individual roles in further internalising M&E approaches and lessons are undefined. Although shelter M&E systems have already been revised and implemented, there appears to be uncertainty regarding who will ensure continued quality control and analysis. It would therefore be useful to clarify regional individuals responsible for data collection, reporting, and analysis. Similar clarity would be useful at BOK between the Operational Information Unit and the Programme Unit – specifically the Technical Advisor, Senior Programme Officer, and Assistant Chief of Mission (Programme) roles. More generally, staff and counterparts everywhere can take more responsibility for remaining actively engaged in observation, discussion, and analysis in the course of programme implementation and management. 3. Increase dialogue between the Programme Unit and the Operational Information Unit. It is interesting that consultation with the OIU was not included in the M&E TOR, nor was the OIU ever identified as a key unit with whom to liaise and work. Because the OIU receives, collates, and distributes all programme data they are an important node in the M&E process. OIU tables and graphs are circulated regularly, but it appears that there is little analytical engagement with the content of these outputs. To improve coordination between Programme and the OIU, an important first step was taken in December 2004 by specifying one OIU member who is responsible for all Programme Unit data collection and presentation. 4. Ensure translation and dissemination of Dari and Pashto versions of the 2005 Shelter

Guidelines. Dari and Pashto translations of the 2005 Shelter Guidelines will be instrumental in developing a consistent understanding and ethos across all programme stakeholders. Earlier versions were translated into Dari only and were therefore of less value. Wide dissemination of guidelines in the three operational languages of Afghan programming (including English) will enhance programme continuity and quality. It is possible that these documents have already been translated and circulated at the time of writing. 5. Ensure translation and dissemination of Dari and Pashto versions of the Guideline

summary to be used in regional workshops. At the December 2004 shelter workshop a bullet-point summary of the Guidelines was used to facilitate presentation. SO Qandahar agreed to translate and disseminate a Pashto version, while SO Herat agreed to disseminate a Dari version. These translated presentations will then be used by all regions to guide the workshops planned for IP and DORR counterparts. It is possible that these documents have already been translated and circulated at the time of writing.


6. Ensure regional workshops are convened to improve IP and DORR understanding of revised Guidelines and reporting formats.

The revised Guidelines and reporting formats created a need for regional workshops with IP and DORR counterparts to ensure that all stakeholders are consistently clear with the changes made in 2005 and with the rationale behind the changes. One objective of the December shelter workshop was to ensure that UNHCR participants were able to convene similar workshops in their respective regions. Toward this, additional materials for regional workshops were circulated in January 2005. Failing to convene a regional workshop could result in less effective implementation (particularly if dependent on remote control management) and less reliable data collection. 7. Continue to refer to the shelter logical framework and further revise if necessary. The shelter M&E process has sought to increase legitimacy of the shelter logical framework by using it as the basis of fieldwork, evaluation, and analysis – ideally resulting in a framework that is more a management and analysis tool than an annual report obligation. Just as the Guidelines required revision to update the document to current implementation approaches, the logical framework will require revision to ensure it represents the programme and its priorities as they evolve over time. In its present version, some of the logical framework indicators are more interesting or insightful than others, but they nonetheless represent a spectrum of relevant programmatic issues. Improved indicators might include assessment of beneficiary debt or more specific reference to improved livelihood outcomes. 8. Revisit the regional shelter logical frameworks used for 2005 LOI submissions and

identify means of verification for each indicator. Each region revised the indicators of the standard shelter logical framework to reflect their contextual priorities and interests. Although regional flexibility is enshrined in the Guidelines, the lack of means of verification in all logical frameworks suggests that it will not be possible to track or assess achievement of the indicators. This contradicts the intent of developing a logical framework. 9. Ensure that Monthly Progress Reports are consistently complete and submitted on

time. This is ordinarily a responsibility of the OIU, but should nonetheless have support from the Programme Unit in case repeated follow up is necessary – particularly if heads of section or sub office need to be contacted regarding consistently late or incomplete monthly reports. All regions have agreed to submit monthly progress reports to BOK before the 30th of each month. 10. Ensure that Monthly Consolidated Reports are disseminated to all regions on time. After receiving progress reports from each region, the OIU is responsible for collating data into the Monthly Consolidated Report – after which they are responsible to disseminate this report to the Programme Unit and all regions before the 5th of each subsequent month. Consistent dissemination of programme data can facilitate ongoing engagement with implementation rates and the ongoing achievement of programme standards and indicators. 11. Make more routine use of available programme data. Despite the quantity of programme data and maps available, it appears that data collection receives more attention than data analysis. In addition to the quantitative summaries provided in the monthly and consolidated reports, each report now contains tables and graphs that illustrate


implementation rates and ongoing achievement of programme standards and indicators so that all staff have more interpretive resources immediately available. These data and illustrations can be used during programme or head of sub office meetings, for example, to maintain perspective of national programme progress. Regionally, data can be used to facilitate IP meetings, to plan implementation, or to prioritise individual themes for greater attention or follow up. 12. Perform evaluations at the end of every programme year. Annual programme evaluations can facilitate ongoing engagement with quantitative and qualitative shelter issues, and can provide documented evidence upon which to update programme strategies, systems, or formats. Evaluation styles can range from an informal forum that reviews data and lessons learned to an external assessment that is budgeted into each year. Regardless of style, annual evaluations are a demonstration of commitment to the M&E process and iterative programme cycles.


Annex A: Terms of Reference for UNHCR Shelter M&E Consultancy

I. Position: Management Consultant for Shelter Programme II. Project: Consultant responsible for Monitoring and Evaluation of UNHCR funded shelter projects III. Overview of Shelter Assistance Project Years of conflict and drought have forced an estimated 6 million Afghans to leave their areas of origin. An estimated 5 million people left as refugees, mainly to Pakistan and Iran, and approximately 1 million became Internally Displaced People (IDPs). One of the main effects of conflict and displacement has been the destruction and deterioration of housing throughout the country. There is no systematic and comprehensive assessment undertaken to determine the number and level of housing destruction in the country. However, some indicators (previous returnee monitoring, analysis of displacement statistics, cursory visits by donor community, some interviews, result of district profile, etc.) provided an estimate that over 500,000 homes had been either partially or totally destroyed. The Government has requested UNHCR to focus its shelter assistance for returnees in the rural areas. Upon return, many have no choice but to stay with their relatives or friends in the community during the initial period. Others cannot enjoy such hospitality, having to live in tents or are temporarily accommodated in sub-standard public buildings. In 2002 and 2003, over 2 million people returned to Afghanistan including more than 400,000 IDPs. As part of the initial reintegration package, UNHCR has supported the construction of close to 90,000 rural shelters in 2002/2003. In 2003 alone, approximately 52,000 vulnerable families throughout the country benefited from this assistance. For 2004, the number of returnees is estimated at 1,000,000 individuals (or 200,000 families) and 190,000 IDPs (or 38,000 families). The Government estimates that 60% of these families will return to the rural areas. UNHCR will support 20,000 families to build their shelters. UNOPS will hopefully complement this number by implementing additional 20,000 units with its own funding. As the shelter programme goes into its third year, it is important that a proper monitoring and evaluation mechanisms are set, so that the minimum quality is maintained from the beneficiary selection up to the completion of shelter constructions. This will become even more relevant, as UNHCR’s field presence will be limited with the reduced number of staff in 2004 and beyond. IV. Responsibilities

Under the direct supervision of the Assistant of Chief Mission Programme and in close collaboration with OCM Technical Coordinator, the consultant will undertake the following tasks: • Keep track of all project monitoring and evaluation schedules and work with UNHCR Sub Offices

in the region and UNHCR OCM Program section to design and implement monitoring and evaluation procedures.

• Monitor and evaluate the occupancy rate of UNHCR funded shelter programme in 2002 and 2003 in Afghanistan.

• Evaluate the impact of shelter provided to the returnees, IDPs or other vulnerable population which include selection of beneficiaries.

• Ensure that the 2004 programme has the required quality, including the selection of beneficiaries. • Monitor the capacity building of government/NGO partners. • In line with the budget allocated to IP, analyze effectiveness and efficiency of the program in

2003/2004. • Monitor and evaluate that the assistance is provided to beneficiaries identified, approved and

agreed according to the shelter guideline. • Ensure that the assistance is well documented by IPs.


• Ensure that the IPs visited all the beneficiary households completed shelter. • Ensure that the visibility of the donor is done in the ground by IPs. • Monitor UNOPS’ (and any other operational partners’) shelter programme to ensure consistency

in the implementation. • Monitor and ensure continued coordination between the parties concerned i.e. Government

Officials, IPs, and UNHCR. • Develop evaluation tools for assessing IP capacity (adequate resources, site supervision and

transportation). • In co-ordination with OCM Technical Coordinator, organise workshop on lessons learned with

UNOPS and UNHCR main actors. • To re-assess the value of monitoring with DORR/DRRD and suggest recommendations if any. Information Systems • Work with all departments to follow up on project and programme database. • Collaborate with OCM Technical coordinator and field offices to ensure the quality of current

shelter project implementation. • To review and analyze information collected through audit reports and review how

recommendations were translated into actions. • To document and analyze targeting of shelter assistance in relation to returnee statistics at

district/province level for period 2002-2004. • To liaise with M&E specialist of other main actors in shelter to ensure consistency of approach to

the maximum extent possible. Participatory Appraisal, Monitoring, and Evaluation • Ensure that local partners engaged in appraisal for and/or with UNHCR are trained and advised

in participatory approaches, and other key methodologies for information gathering. • Work with OCM and Sub Office staff to help design and improve adapted participatory appraisal

mechanisms for UNHCR shelter project implementation in Afghanistan. • Develop and provide report every two months. • Provide the final report stating the management against all the key points of the TOR, own

assessment and recommendations, actions already taken during the period of the consultancy and recommendations for future actions in relation to both Afghanistan and future similar operation elsewhere in the world.

V. Qualifications required • University degree preferably in humanitarian assistance, development, and/or related field. • Critical thinking and analytical skills. • Minimum 5 years of professional experience in M&E or related field. • Proven relevant experience in the humanitarian assistance. • Experience with participatory appraisals. • Ability to travel and work in difficult conditions. • Good command of spoken/written English. Knowledge of Dari/Pashto is an asset. • Proven ability to work creatively and independently both in the field and in the office. VI. Other highly desirable characteristics • Experience in working directly with refugees/IDPs and local communities. • Experience in training local NGOs. VII. Reporting to: Assistant Chief of Mission Programme, UNHCR OCM Kabul, Afghanistan. VIII. Support: Water/shelter Technical Coordinator(s); logistics and supply section staff; and Sub

Offices staff as main collaborators. IX. Duration: 9 Months (later revised to 6)


Annex B: Shelter M&E Progress Report (October 2004)

19 July – 19 September 2004

* * * Summary Version * * *

SYNTHESIS This report covers the first two months of a five-and-a-half month consultancy (19 July – 31 December) to investigate qualitative aspects of the UNHCR shelter programme in Afghanistan. Initial analysis of the programme, its progress to date, and the perceived needs for more expansive and rigorous qualitative data suggested a focus on the following four themes, each of which is discussed separately below: • Beneficiary targeting and selection • Technical monitoring8 • Shelter occupancy • Programme impact This report outlines the main findings and recommendations related to each of these four themes. This summary version includes all findings and recommendation from the main report, without the supporting analysis. In the full report version, findings are followed by more detailed explanations of the context from which they emerged. Programmatic and policy recommendations are included in the text as they arise from the analysis. The findings, recommendations, and analysis together give direction to revised M&E systems offering greater quantitative and qualitative breadth, better comparability of national data, more structure to quality control, and deeper insight into programme impact. FINDINGS Targeting & Selection • The selection process appears to be effective in reaching appropriate families. • Vulnerability criteria are subjective and unstandardised, with an unclear distinction from

poverty. • Kuchi are not specified as a possibly vulnerable group. • Beneficiary ‘preselection’ might facilitate the BSC decision making process. • Female participation at all levels of the programme appears minimal. • Among poorest of the poor, some families are unable to benefit from the programme

because of landlessness or an inability to afford construction – even where voluntary labour might be accessible.

Technical Monitoring • Revised and more expansive monitoring formats can help guide implementation at all

programme levels. • Monitoring formats designed by individual sub or field offices serve to meet regional needs,

but do not allow for national data comparability or analysis.

8 The initial focus was on technical quality of shelters. It was then agreed with the ACM/P that this was too limited. By widening the reference to Technical Monitoring, assessing and monitoring technical quality can be included in addition to related issues of data collection, management, and use.


• Field level IP follow-up is worthwhile, but its technical role might be limited as shelter construction techniques appear to be widely known in communities.

• Remote control management, monitoring, or verification in insecure areas presents programmatic challenges. Revised monitoring formats can affect a measure of control over the process.

• Training to strengthen staff M&E capacity will add value to the programme and validity to its data.

Shelter Occupancy • Occupancy of completed shelters is not a problem; nearly all shelters are (eventually)

occupied. • Apparent unoccupancy is the result of unfortunate verification timing, not inadequate

beneficiary selection. • Empty units are uncommon, and are typically the result of exceptional circumstances. • Seasonal issues also might affect the timing of shelter inhabitation. • Inner floor preparation and compound perimeter wall erection also cause delays to inhabit a

‘completed’ house. • Overall, the timing of all programme phases needs more analysis. Programme Impact • Higher programme impact will result from effective beneficiary targeting and selection. • BSC perceptions that national shelter needs are greater than UNHCR programme supply

apparently result in more scrutiny during the selection process. • It is likely that poorer families would use lower quality building materials if they did not

receive programme assistance. • Shelter kits appear to be reaching the appropriate beneficiaries, creating greater programme

impact. • Families are building and using latrines. • Hygiene messages and their promotion might need to be strengthened. RECOMMENDATIONS 1. Refine vulnerability groups on beneficiary selection form and include greater detail. Include

Kuchi. 2. Revised selection and monitoring formats should include expanded information on criteria

identified during the selection process that led the BSC to agree on family eligibility for shelter assistance.

3. Identify whether other encashment centres are engaging in shelter beneficiary preselection,

then determine whether this should be pursued or proscribed as a national strategy. If pursued nationally, develop a set of preselection guidelines to facilitate a more standardised and less subjective approach, ensuring that BSCs remain the ultimate decision making body for determining shelter eligibility.

4. As BSC membership is more broadly based than that of CDCs, BSCs should remain the

selection body for the shelter programme. In areas where NSP CDCs are currently functional, maintain dialogue and consider including a CDC member on the BSC, but if possible do not cede decision making responsibility and thereby jeopardise the current accountability and transparency of the BSC.

5. With guidance from the UNHCR Gender Advisor, establish a gender analysis framework for

the shelter programme to identify the different needs and impact of shelter programming for


females and males. Once established, identify strategies and methodologies to better access female beneficiaries and to include more women in the selection process.

6. Monitoring formats should include data on excluded and ineligible families in order to better

understand the dynamics of selection and potentially identify means for either including the poorest families or pursuing other assistance possibilities.

7. Include in programme monitoring data on families rejected due to landlessness. Investigate

ways that these families might receive additional support in lieu of shelter. 8. Ongoing monitoring formats should be standardised across all regions for continuity and

comparability at each programme level (sub-IP to IP, IP to SO, SO to BOK). A database designed in collaboration with the Data Section at Base Office Kabul should be developed to compile the additional information collected and to facilitate further analysis of national data.

9. Backstop revised monitoring systems with the training and support needed to ensure they

are used properly. This will also give an opportunity to identify M&E point people in each region and to ensure everybody recognises why information is being collected, how it is used to inform the programme, and how to apply the common methodology.

10. Develop a common methodology for remote control management and monitoring to ensure

that important issues are not overlooked and that necessary data are collected. 11. Ensure that all stakeholders involved in M&E receive the support and training needed to

effectively engage in the process, analyse the findings, and programmatically respond. 12. Review additional qualitative themes related to beneficiary selection, vulnerability criteria,

technical standards, construction quality, occupancy, and overall impact to determine which should be included in future programme monitoring. [Please refer to full report format for the complete list of qualitative M&E themes outlined in this recommendation.]9

13. Use existing data to analyse seasonal trends in returns and how they compare with the

shelter programme cycle. This data can also be cross referenced with seasonal considerations including labour availability, water access, and agricultural cycles.

14. Selection monitoring forms can include fields for when returned, when selected, when

construction began, estimated completion date, actual completion date, estimated occupancy date, actual occupancy date. These data would be easy to collect and would offer better understanding into the questions of decision making and timing of occupancy. This understanding would help better time verification and clarify lingering concerns about unoccupancy – and could be cross referenced with seasonal data (c.f. Recommendation 13) to better understand occupancy decision making.

15. Verify the selection calendar for each region, specifying the period of return for each year of

beneficiary selection. Ensure that each cycle of selection begins with the month at which it concluded in the previous round, so that returnees are captured throughout the year.

16. Analyse existing VRF data on average family size of selected beneficiaries to determine

whether five is indeed an appropriate figure to use as a programmatic reference. 17. Investigate effects of construction loans on household economy. Also investigate how much

people borrow and how much they expect to repay, as some respondents suggested that many loans will never be repaid.

9 Note that some of these themes do not require collection of additional data, only a new approach to the analysis of existing data.


18. Determine whether annual income data would be possible to collect and whether an income threshold could be used to help target beneficiaries for greater impact.

19. Investigate what strategies beneficiaries in different regions might be using spontaneously to

protect shelter materials from insects or other endemic parasites that can weaken materials. When such practices are identified, mainstream them into ongoing technical support and monitoring provided by IPs and UNHCR.

20. Determine the extent to which families are repairing or reconstructing houses, and see

whether current selection modalities address this distinction in all regions. 21. Confirm whether there are Afghan government guidelines for agencies funding,

implementing, and monitoring rural shelter. If so, ensure that programme design aligns with government standards – particularly for earthquake prone regions.10

22. Develop intermittent monitoring to capture data on market activity related to the shelter

programme – particularly in areas with high concentrations of shelter beneficiaries. 23. Develop guidelines for IP hygiene promotion activities and provide training needed to ensure

effective implementation and communication of messages. 24. Ensure that IPs and UNHCR include female staff in the training and implementation of

hygiene promotion to the maximum extent possible. 25. Revisit the messages and methodology of hygiene promotion activities. Revise if necessary

and investigate opportunities for capacity building to support better promotion.

10 It has since been established that MRRD produced shelter guidelines dated 29 February 2003, apparently based on those of UNHCR. They outline beneficiary selection, the assistance package, implementation procedures, and stakeholder roles and responsibilities. Although how ‘active’ a document the government guidelines are and the extent to which all agencies implementing shelter adhere to them are unclear, the current UNHCR programme is largely aligned with the guidelines.


Annex C: 2005 Shelter Guidelines – Table of Contents

1. The UNHCR Shelter Programme in 2005........................................................ 4 2. Guiding Principles............................................................................................ 7 2.1 Community based approach ................................................................................. 7 2.2 Women’s direct participation ................................................................................. 7 2.3 Access to land ....................................................................................................... 7 2.4 Focus on vulnerability ........................................................................................... 8 2.5 Environmental concerns........................................................................................ 9 2.6 Preservation of cultural and regional preferences................................................. 9 2.7 Contribution to local economies ............................................................................ 9 2.8 Involvement of local authorities ............................................................................. 9 3. Gender Considerations.................................................................................. 10 4. Implementation............................................................................................... 11 4.1 Project cycle management (Status 0 – Status 6) ................................................ 11 4.2 Identification of implementing partners................................................................ 13 4.3 Mine clearance certificate.................................................................................... 13 4.4 Beneficiary selection .......................................................................................... 14 4.5 Preparation for construction ................................................................................ 15 4.6 Preparation of materials ...................................................................................... 15 4.7 Quality control ..................................................................................................... 16 4.8 Construction ........................................................................................................ 16 4.9 Cash component ................................................................................................. 17 4.10 Handover certificate ............................................................................................ 18 4.11 Monitoring and evaluation ................................................................................... 18 5. Flexibility and Accountability........................................................................ 20 6. Reporting ........................................................................................................ 20 7. Annexes .......................................................................................................... 20

Annex 1: Shelter Logical Framework Annex 2: Beneficiary Selection Procedures Annex 3: Beneficiary Selection Format Annex 4: Letter of Undertaking Annex 5: Mine Clearance Certificate Annex 6: Construction Material Coupon Annex 7: Technical Monitoring Format Annex 8: Final Verification Format Annex 9: Monthly Progress Report Annex 10: Monthly Consolidated Report Annex 11: Monthly Comparison Report Annex 12: Technical Specifications Annex 13: Bill of Quantity (Standards A, B, C) and supply allocations Annex 14: Recommended formula and tabulation rate for IP operation costs Annex 15: Standard A (shelter with wooden beams) Annex 16: Standard B (shelter with iron beams) Annex 17: Standard C (shelter with dome, arch, vault)


Annex D: Fieldwork Guide I

Question Prompts Observation Prompts Name Diagram groupings Returned from Mines/UXO When returned Walk: people, things, quality Number family members Water supplies and sources Number households in community Latrines and hygiene Number of returnees in community Town, market, facilities Number of families in community receiving shelter assistance

Clinics

When did shelter construction begin Schools Approximately when will shelter construction be completed

Average shelter area

How much money invested in the shelter Quality of shelters How much are the anticipated remaining costs Cooking and fuel How was the shelter financed Diet and food supply Land ownership status Daily meals: number and type Time between completion and occupancy Agriculture viability Other unoccupied shelters Livelihood viability Income sources Kitchen gardens Income in exile Other construction Income upon return Specific issues for women Credit necessary for construction or materials Specific issues for children If yes, how much time needed for repayment Kuchi If no shelter assistance, how would family have managed

Why was the decision made to return What would cause family to leave again


Annex E: Fieldwork Guide II

Indicators Verification 15% of most vulnerable families, primarily returnees, receive shelter assistance.

Compare beneficiary # with total returns per district. Data section.

Beneficiaries identified as too poor to construct their shelter receive community support and/or additional financial support to complete their shelters.

Discuss ashar trends during fieldwork. Discuss supplemental grants during fieldwork. SO staff: Supplementary grants disbursed?

Intended beneficiaries are still living in their shelters 12/24 months after handover.

Verification format during fieldwork.

No cases of reverse movements or secondary displacement due to lack of shelter in places of return.

Fieldwork discussions.

Objective

Most vulnerable returnees have access to adequate shelter.

Shelter beneficiaries are able to improve their conditions and not deteriorate into greater vulnerability.

Fieldwork discussions.

# shelters constructed according to standards and specifications of UNHCR 2004 Shelter Guidelines.

SO verification data. Data section.

# most vulnerable beneficiary families selected by BSC according to established eligibility criteria.

Data section vulnerability data (for returns, not for shelter beneficiaries – proxy).

# beneficiary families completed their shelters and latrine and received cash grant.

SO verification data. Data section.

Minimum floor space of 3.5m2 per person per shelter is provided.

Data section return data (proxy indication): data unavailable for shelter beneficiary family size.

1 latrine constructed for each house. SO verification data. Data section.

#/% female and male beneficiaries Not available for 2004. Data section return data (proxy).

Output

Houses constructed and occupied in areas of high and potential return.

#/% women and men participating in BSC decision making and implementation process.

Not quantifiable for 2004. Discussions (very low female participation).

Community ashar contributions • Who participates in ashar? What are the dynamics of making this happen? • What would happen if ashar not mobilized? Do families not receive shelter because of no ashar? Supplemental grants for extremely poor families • Would this create conflict in the community (if different families received different grants)? • Are FO/SOs using this facility? Can IPs authorise? • What about the poorest families? Can they afford to participate? Are we able to identify and include them? Outstanding shelter loans (materials, transport, water, construction) • How much? How does this affect the household economy? • Expected time to repay? Reverse movement or secondary displacement? Seasonal preferences for when is best to construct. Why? Availability of water and availability of other paid labour opportunities? Temporary employment of women for BSC. Female participation throughout the project cycle? Daily labour rates; shelter material costs; wheat cost (seer)? Latrines completed; latrine prevalence in community? Pictorial floor plan distributed to all beneficiaries? Beneficiary selection calendar?


Annex F: Secondary References

Babister, Elizabeth & Kelman, Ilan (2002) The Emergency Shelter Process with Application to

Case Studies in Macedonia and Afghanistan. <www.shelterproject.org> Barakat, Sultan (2003) Housing Reconstruction after Conflict and Disaster (HPN Network Paper

43). Overseas Development Institute: London. Chambers, Robert & Conway, G. (1992) Sustainable Rural Livelihoods: Practical Concepts for the

21st Century (IDS Discussion Paper 296). Institute for Development Studies: Brighton. Fardanesh, Ghassem (2002) Afghanistan Mission Report, 3-31 December (UNHCR internal

report). UNHCR: Geneva. GTZ (2003) Guidelines for Building Measures after Disasters and Conflicts. GTZ: Eschborn. Lister, Sarah & Karaev, Zainiddin (2004) Understanding Markets in Afghanistan: A Case Study of

the Market in Construction Materials. Afghanistan Research and Evaluation Unit: Kabul. March, Candida et al (1999) A Guide to Gender Analysis Frameworks. Oxfam publishing: Oxford. Pain, Adam & Lautze, Sue (2002) Addressing Livelihoods in Afghanistan. Afghanistan Research

and Evaluation Unit: Kabul. Saunders, Graham (2004) ‘Dilemmas and Challenges for the Shelter Sector: Lessons Learned

from the Sphere Revision Process.’ Disasters, XXVIII:2 (June). SIDA (2000) Self-help Housing as Practised by SIDA. SIDA: Stockholm. Sphere Project (2004) Humanitarian Charter and Minimum Standards in Disaster Response (2nd

Edition). Oxfam Publishing: Oxford. Telford, John (1997) Counting and Identification of Beneficiary Populations in Emergencies:

Registration and its Alternatives (Good Practice Review #5). Overseas Development Institute: London.

Twigg, John (2002) Post-Disaster Housing Reconstruction and Livelihood Security. Benfield

Hazard Research Centre: London. UNHCR (undated) Handbook for Emergencies (2nd Edition). UNHCR: Geneva. UNHCR (undated) Planning Rural Settlements for Refugees. UNHCR: Geneva. UNHCR (1993) First International Workshop on Improved Shelter Response and Environment for

Refugees. UNHCR: Geneva. UNHCR (1994) Shelter and Infrastructure: Camp Planning (Programme and Technical Support

Section). UNHCR: Geneva. UNHCR (1998) Review of the UNHCR Housing Programme in Bosnia and Herzegovina

(Inspection and Evaluation Service report). UNHCR: Geneva. UNHCR (2000) External Evaluation of the UNHCR Shelter Program in Rwanda 1994-1999

(Reintegration and Local Settlement Section report). UNHCR: Geneva.


UNHCR (2004) Practical Guide to the Systematic Use of Standards and Indicators in UNHCR

Operations. UNHCR: Geneva. UNHCR (2004) 2004 UNHCR Guidelines for Rural Shelter. Programme Unit: Branch Office Kabul. UNHCR (2005) 2005 Shelter Guidelines. Programme Unit: Branch Office Kabul. USAID (1991) Afghanistan Land Ownership (draft report). Nathan Berger: Virginia. WFP & MRRD (2004) Report on Findings from the 2003 National Risk and Vulnerability

Assessment in Rural Afghanistan. Vulnerability Analysis and Mapping Unit (WFP) and Vulnerability Analysis Unit (MMRD): Kabul.

Wily, Liz Alden (2003) Land Rights in Crisis: Restoring Tenure Security in Afghanistan. Afghanistan

Research and Evaluation Unit: Kabul.


Annex G: Shelter Completion 2002 – 2004

2002 2003 2004 2002 – 2004

Province &

REGION

Planned

Com

pleted July 2003

Achieve-m

ent

Planned

Com

pleted July 2004

Achieve-m

ent

Planned

Com

pleted Jan 2005

Achieve-m

ent

Planned

Com

pleted January

2005

Achieve-m

ent

Badakhshan 0 0 0% 0 0 0% 150 150 100% 150 150 100% Baghlan 1,400 1,400 100% 1,200 1,200 100% 775 775 100% 3,375 3,375 100% Balkh 430 430 100% 761 746 98% 750 750 100% 1,941 1,926 99% Faryab 1,020 945 93% 1,700 1,700 100% 800 800 100% 3,520 3,445 98% Jawzjan 500 500 100% 1,395 1,395 100% 525 525 100% 2,420 2,420 100% Kunduz 1,463 1,463 100% 1,350 1,350 100% 720 711 99% 3,533 3,524 100% Samangan 1,550 1,550 100% 1,038 938 90% 450 450 100% 3,038 2,938 97% Sari Pul 820 809 99% 1,200 1,200 100% 400 400 100% 2,420 2,409 100% Takhar 700 700 100% 1,000 1,000 100% 430 352 82% 2,130 2,052 96% NORTH 7,883 7,797 99% 9,644 9,529 99% 5,000 4,913 98% 22,527 22,239 99% Hilmand 300 300 100% 2,450 2,450 100% 1,400 1,359 97% 4,150 4,109 99% Kandahar 1,100 1,100 100% 1,800 1,759 98% 981 943 96% 3,881 3,802 98% Nimroz 0 0 0% 250 250 100% Under SoH in 2004 250 250 100% Uruzgan 0 0 0% 0 0 0% 0 0 0% 0 0 0% Zabul 0 0 0% 0 0 0% 0 0 0% 0 0 0% SOUTH 1,400 1,400 100% 4,500 4,459 99% 2,381 2,302 97% 8,281 8,161 99% Ghazni 1,500 1,500 100% 1,000 1,000 100% 700 687 98% 3,200 3,187 100% Khost 0 0 0% 1,000 1,000 100% 920 920 100% 1,920 1,920 100% Paktika 0 0 0% 1,080 1,080 100% 0% 1,080 1,080 100% Paktya 0 0 0% 1,620 1,568 97% 1,280 1,280 100% 2,900 2,848 98% SOUTHEAST 1,500 1,500 100% 4,700 4,648 99% 2,900 2,887 100% 9,100 9,035 99% Kunar 500 479 96% 293 287 98% 400 381 95% 1,193 1,147 96% Laghman 2,000 1,893 95% 1,157 1,128 97% 700 685 98% 3,857 3,706 96% Nangarhar 6,920 6,741 97% 5,350 5,308 99% 2,800 2,742 98% 15,070 14,791 98% Nuristan 0 0 0% 0 0 0% 0% 0 0 0% EAST 9,420 9,113 97% 6,800 6,723 99% 3,900 3,808 98% 20,120 19,644 98% Badghis 3,548 3,548 100% 3,650 3,650 100% 900 900 100% 8,098 8,098 100% Farah 1,250 1,244 100% 946 946 100% 700 700 100% 2,896 2,890 100% Ghor 1,300 1,300 100% 1,240 1,240 100% 700 700 100% 3,240 3,240 100% Hirat 1,200 1,200 100% 3,764 3,751 100% 2,400 2,367 99% 7,364 7,318 99% Nimroz Under SO Kandahar in 03 Under SO Kandahar in 03 300 293 98% 300 293 98% WEST 7,298 7,292 100% 9,600 9,587 100% 5,000 4,960 99% 21,898 21,839 100% Bamyan 1,900 1,900 100% 500 500 100% Under FOB in 2004 2,400 2,400 100% Kabul 7,250 7,211 99% 9,880 9,879 100% 4,374 4,350 99% 21,504 21,440 100% Kapisa 900 900 100% 1,450 1,450 100% 401 401 100% 2,751 2,751 100% Logar 900 900 100% 1,000 1,000 100% 840 840 100% 2,740 2,740 100% Parwan 1,450 1,446 100% 2,950 2,950 100% 900 900 100% 5,300 5,296 100% Wardak 400 400 100% 560 560 100% 0 0 0% 960 960 100% CENTRAL 12,800 12,757 100% 16,340 16,339 100% 6,515 6,491 100% 35,655 35,587 100% Bamyan 490 490 100% 490 490 100% Dai Kundi 268 268 100% 268 268 100% Wardak (Behsud 1&2)

New Provinces under FO Central Highlands

New Provinces under FO Central Highlands

42 42 100% 42 42 100% CENT. HIGHLANDS 0 0 0% 0 0 0% 800 800 100% 800 800 100% TOTAL 40,301 39,859 99% 51,584 51,285 99% 26,496 26,161 99% 118,381 117,305 99%


Shelter Programme Monitoring and Evaluation - … · Shelter Programme Monitoring and Evaluation...

Documents

Transcript of Shelter Programme Monitoring and Evaluation - … · Shelter Programme Monitoring and Evaluation...