Conducting disability inclusive baseline assessments for community-level WASH Projects

Clare Hanley and Hanna Goorden, CBM Australia in partnership with World Vision and WaterAid

May 2016

1

Contents

List of Acronyms....................................................................................................................................2

Executive Summary...............................................................................................................................3

Introduction...........................................................................................................................................6

Methodology.........................................................................................................................................7

Partnership between CBM, WaterAid and World Vision.......................................................................8

The WASH projects................................................................................................................................8

Baseline data collection in WASH projects............................................................................................9

Key lessons on disability inclusive data collection...............................................................................12

Discussion............................................................................................................................................23

List of AcronymsCS WASH Fund Civil Society WASH FundDPO Disabled People’s OrganisationDFAT Department of Foreign Affairs and TradeGESI Gender Equity and Social InclusionM&E Monitoring and EvaluationNGO Non-Government OrganisationPNG Papua New GuineaSDGs Sustainable Development GoalsWASH Water, Sanitation and HygieneWGSS Washington Group Short Set of Questions on Disability

Further information

This report has been produced in partnership with WaterAid and World Vision. For further information or to provide feedback please contact:

CBM Australia – Clare HanleyTechnical Advisor – Disability InclusionEmail: CHanley@cbm.org.au

WaterAid Australia - Chelsea HuggettEquity, Inclusion and Rights AdvisorEmail: Chelsea.Huggett@wateraid.org.au

CBM Australia – Hanna GoordenQuality Officer: Monitoring & EvaluationEmail: HGoorden@cbm.org.au

World Vision Australia - Andrew JalanskiCivil Society WASH Program ManagerEmail: Andrew.Jalanski@worldvision.com.au

Cover Photo credit: WaterAid staff conducting an interview with a person with disability as part of the disability situational assessment in PNG. WaterAid/CBM 2015

2

Executive Summary

This report documents the lessons learnt and outcomes of collecting disability data in five water, sanitation and hygiene (WASH) projects, with the aim of learning and therefore strengthening disability inclusive data collection processes in future projects. With the adoption of the Sustainable Development Goals, which aim for universal access by 2030, awareness of the need for WASH programs to reach and benefit everyone, everywhere, is growing. Many programs are now proactively seeking to reach people with disabilities and collecting disability inclusive data is critical in setting the foundations for disability inclusive WASH. Global frameworks such as the SDGs also call for disaggregated data by disability to monitor progress towards the Global Goals and to ensure programs are leaving no one behind. Since 2014, CBM has partnered with World Vision and with WaterAid to support disability inclusion within five Civil Society WASH Fund Projects (two projects in Papua New Guinea (WaterAid & World Vision) and one each in Timor-Leste (WaterAid), Sri Lanka (World Vision) and Zimbabwe (World Vision).

Key Lessons on Disability Inclusive Data Collection in WASH

1. Including people with disabilities in data collection teams resulted in positive outcomes for people with disabilities, their communities & other stakeholders, but needs to be resourced appropriately

The WASH projects all included people with disabilities in data collection teams (rather than merely as subjects) to some degree, generally by establishing partnerships with Disabled People’s Organisations (DPOs). This was found to increase the confidence of people with disabilities, and challenge negative attitudes within communities, government and implementing partners. It also increased the quality of the data collected by encouraging people with disabilities to speak during consultations. It did however require investment of additional time and resources to broker relationships and build trust, strengthen the capacities and confidence of data collectors and plan the logistics. In Zimbabwe for instance, a research team of people with diverse impairments had to

be transported to project sites over rough terrain which proved challenging.

2. Collecting data on people with disabilities in communities needs to be done sensitively

The term ‘disability’ can be understood differently across cultures and communities and in some communities it is not widely used or understood. In the WaterAid projects in PNG and Timor-Leste, project staff found it more useful to talk about difficulties with aspects of functioning (as recommended by the Washington Group) as opposed to talking about ‘disability’. Increasing the visibility of people with disabilities by including them in community consultations for the first time also needs to be closely monitored to ensure that any potential backlash for challenging social norms is addressed.

3. Household surveys need to collect individual level data about access to WASH in order for that data to be disaggregated by disability. However, inclusion of the Washington Group Short Set of (WGSS) questions is also useful for identifying households which include people with disabilities who can then be followed up throughout the project.

The SDGs and some donors require monitoring data to be disaggregated by disability, sex and age to assist in determining who does and does not benefit from development programs. This was one of

3

the common objectives of including the WGSS questions in the baseline surveys of the WASH projects. However, the fact that surveys collected information at a household level only (generally from the head of household) rather than individual level, meant that individual differences in access to WASH within households were not identified, and some data could not be fully disaggregated.

Finding people with disabilities within communities is often a key challenge reported by NGOs undertaking development projects. It is therefore noteworthy that while in most cases including the WGSS questions did not enable disaggregation of data because of the structure of the survey, they were generally effective in identifying households which included people with disabilities. People with disabilities identified could then be invited to participate in qualitative data collection processes or targeted for inclusion throughout the project.

4. There are challenges with but also unexpected findings from using the WGSS questions in surveys

While the WGSS questions are often promoted as being easy to use and proved useful, all NGOs and their local partners reported some challenges in using the questions for the first time as they were intended and with analysing the results. Pressure to complete the baseline quickly and with limited resources meant that in some cases insufficient time was spent explaining to NGOs and data collectors why the WGSS questions were recommended and training them on their use. As a result in some cases the questions were not translated appropriately or were changed without sufficient testing. Deciding on an appropriate ‘cut-off’1 to use to classify people ‘with’ and ‘without’ disabilities and using this consistently was also challenging. For WASH projects, it was also found that there may be advantages in using a more sensitive ‘cut-off’ (e.g. also including people who answer ‘some difficulty’ to at least one question). This is because people with minor difficulties performing tasks in fact reported significant difficulties accessing WASH and were often still expected to undertake WASH work for the household.

5. A combination of quantitative and qualitative data collection processes produces the most useful baseline data on people with disabilities

While the quantitative data helped to identify people with disabilities that faced WASH access issues (who), collection of qualitative data on the situation of people with disabilities was critical to enable an understanding of the type of difficulties people with disabilities were experiencing in accessing WASH (why) and developing strategies to address these (how). Projects which completed qualitative data collection specifically with people with disabilities ended up with richer baseline information about barriers to WASH, than those who only included people with disabilities in general WASH community planning processes. Standard community consultation tools, such as pocket voting and community mapping could be used for disability specific consultations with only minor adaptations.

6. Collecting baseline data on people with disabilities led to more inclusive WASH programming and had a range of positive outcomes

Collecting disability inclusive baseline data prompted a range of changes to project planning and inclusive programming. This have resulted in construction of accessible WASH infrastructure, successful use of the baseline data for advocacy and training purposes with stakeholders including

1 Each question includes four response categories including ‘no-difficulty’, ‘some difficulty’, ‘a lot of difficulty’ and ‘cannot do at all’. The recommended ‘cut-off’ for determining disability status in a census is answering “a lot of difficulty’ or ‘cannot do at all’ to at least one question.

4

government, and involvement of people with disabilities in local government decision-making processes. Baseline data collection was able to inform projects most effectively when it was completed early in project implementation, the implementing team had ownership of the findings, data was analysed effectively, and collection and analysis of data was recognised as a tool to inform project implementation rather than as a stand-alone activity for monitoring and evaluation.

Discussion

The above learnings demonstrate that in many ways, inclusive baseline data collection processes lay the foundations for inclusive practice. They can empower people with disabilities, establish relationships between DPOs and other program partners, raise awareness of access challenges, challenge negative attitudes and identify people with disabilities for direct follow up and inclusion. There are however a few major areas where further analysis is recommended. The first relates to the current design of household surveys used by WASH projects to gather baseline information, which was found to be inadequate to allow full disaggregation of data by disability and therefore understanding the extent to which people with disabilities differed in their access to WASH in comparison to people without disabilities. Further analysis is required to determine the best tools for exploring differences in access to WASH within the household within quantitative surveys.

Secondly, given the time and budget constraints which come with any community development project, program implementers need to carefully consider the balance of qualitative and quantitative data that should be collected, based on an analysis of what information is required to inform disability inclusion within that context. Quantitative data can be used to powerfully highlight inequalities in access and measure progress of some indicators towards inclusion. However, there is a danger that analysis stops at identifying people with disabilities and that not enough attention is given to identifying and understanding the situation of people with disabilities more broadly, and developing strategies and solutions to address this, which can impact on the effectiveness of planned WASH interventions.

5

Introduction

It is estimated that 15 per cent of the world’s population have a disability, with prevalence higher in low income countries.2 Although all people have the right to access water, sanitation and hygiene (WASH), people with disabilities often face additional barriers to accessing WASH, compared to people without disabilities, and have often been unintentionally excluded from WASH projects. To achieve universal and sustained access to WASH, programs must overcome these barriers. Over the last few years and with the focus of the Sustainable Development Goals (SDGs) on universal access to WASH, awareness of the need for WASH programs to reach and benefit everyone has been growing. Many programs are now proactively seeking to reach people with disabilities.

Collecting baseline data in WASH programs can provide critical information about the demographics of a community, existing WASH facilities and practices, and a baseline from which to measure change. Ensuring this data collection process is inclusive of people with disabilities is critical in setting the foundations for a disability inclusive WASH program. It provides a systematic way to identify people with disabilities in communities (a precursor for inclusion), to identify potential difficulties experienced in accessing WASH services and strategies to address these, and to enable evaluation over time as to whether inclusion strategies have been successful.

The focus of the SDGs on the need for poverty reduction initiatives to reach all people also has implications for data collection. It increases the need for greater disaggregation of data (e.g. by sex, age and disability) to measure intra-community and intra-household level inequalities. It has also resulted in increased pressure on WASH actors to capture information that helps monitor elimination of inequalities in access to WASH. Collecting data on people with disabilities presents some unique challenges. Unlike determining age or sex, identifying disability is complex. Due to stigma and different cultural understandings of the term ‘disability’, directly asking ‘do you have a disability?’ has been shown to result in very low and likely underreported rates of identification.

CBM Australia is an international Christian development organisation committed to improving the quality of life of people with disabilities in the poorest countries of the world. CBM has built considerable expertise in supporting mainstream development organisations to strengthen approaches to disability inclusive programming. Since 2014, CBM has been partnering with both World Vision and WaterAid to support disability inclusion within their five Civil Society WASH Fund 3 Projects in partnership with local Disabled People’s Organisations (DPOs) (two projects in Papua New Guinea and one each in Timor-Leste, Sri Lanka and Zimbabwe). While the support provided has focused on disability inclusion across the project as a whole, a key initial component of this has been supporting the baseline data collection process to ensure it is disability inclusive. This has involved providing guidance on data collection methodologies, developing survey tools and designing and co-facilitating disability-specific situational analyses where these are planned, as well as raising awareness, building capacities of and facilitating connections between partners, people with

2 WHO and World Bank (2011) World Report on Disability, http://www.who.int/disabilities/world_report/2011/en/.3 The Civil Society WASH Fund ($103m over five years (2013-18)) is an initiative of the Australian Aid Program and aims to increase access to safe water, sanitation and hygiene in developing countries. It funds 13 Civil Society Organisations involved in implementing 29 projects.

6

disabilities, DPOs and data collectors (with and without disability) in disability inclusive data collection and analysis.

To learn from these experiences, and particularly given a focus on disability data in these settings is relatively new, CBM has led the process of documenting the experiences of collecting disability data in these five project settings. This report shares the main findings, with the aim of learning from the approaches and therefore strengthening disability inclusive data collection processes in future programming.

Methodology

Information for this analysis was gathered from a range of sources. Firstly, a workshop was held during a Civil Society WASH Fund learning event in July 2015 to reflect on the disability inclusive baseline data collection approaches. This was facilitated by CBM with WaterAid, Plan, World Vision and the Australian WASH consultant involved in some of the baseline processes and enabled organisations to share their experiences and lessons learnt. After this event, questions to further reflect on these experiences were defined, and semi-structured interviews were completed for each project with CBM’s Technical Advisors, the Gender and Inclusion Officer at WaterAid Australia and the Civil Society WASH Program Manager of World Vision Australia. Input was also sought from key staff members and partners of WaterAid and World Vision in-country who were involved in the data collection processes in the four countries, such as M&E, Gender and Social Inclusion Officers within the projects, and DPO members. Finally some desk-based analysis was completed, which involved reviewing key documents, data collection tools and guidance notes as well as the baseline finding reports. Based on the individual interview notes for each project, key themes/lessons and good practices were identified and summarised.

7

Partnership between CBM, WaterAid and World Vision

In 2014, CBM Australia developed two separate partnerships to support disability inclusion within Civil Society WASH Fund Projects – one with WaterAid (a WASH-specific NGO) to support WASH projects in Papua New Guinea and Timor-Leste, and another with World Vision Australia (a child-focused NGO) to support WASH projects in PNG, Sri Lanka and Zimbabwe. Within these two partnerships, World Vision and WaterAid are the lead implementing agencies, bringing their extensive WASH expertise and a partnership approach to working with local organisations, communities and governments. CBM Australia has played the role of disability advisor, supporting the NGOs to develop disability inclusive approaches and practices within their WASH projects including facilitating involvement and partnership with local DPOs. CBM has also facilitated knowledge sharing and learning on disability inclusion across organisations involved in the CS WASH Fund, for instance through facilitating a CS WASH Fund Learning Event.

Both WaterAid and World Vision commenced this work building on previous successful programs and (some shared) experiences with regards to disability inclusion, for instance through working together with organisations including CBM on the Australian Disability and Development Consortium, disability inclusive pilot programs, and training for staff.

The WASH projects

This report analyses the experiences of these organisations in incorporating disability into baseline data collection processes within five WASH projects implemented in urban, peri-urban and rural settings. A brief background and description of each project is provided below.

WaterAid Projects

In PNG, WaterAid aims to improve the health and quality of life of the poor and vulnerable in the Sepik region (East and West Sepik provinces) through accessible water, clean sanitation and improved hygiene practices. Program implementation is strengthened through advocacy and sector capacity building activities in Port Moresby and with the provincial and lower level government and stakeholders.

In Timor-Leste, WaterAid has led a rural WASH program for ten years, and has a strong commitment to piloting and implementing disability inclusive approaches in WASH. This project is focussed on delivering inclusive and sustainable WASH to 36 additional communities in rural and remote areas of Liquica and Manufahi districts. As in PNG, the project in Timor-Leste also includes significant advocacy and sector capacity building activities in Dili and with the district government and stakeholders.

8

World Vision Projects

In Northern Sri Lanka, the World Vision Project is targeting the growing population and most vulnerable groups (including female headed households and people with disabilities) in two Divisions and 10 villages. It aims to improve coordination, capacity and governance amongst WASH actors to sustain services, knowledge and adoption of safe, contextually and environmentally appropriate hygiene practices and equitable access to safe water and sanitation. Peri-urban and rural areas within the post-conflict Northern Province of Sri Lanka have been severely impacted by prolonged infrastructure underinvestment and have, as a result, limited and distant safe water sources, as well as a high degree of water contamination.

In Zimbabwe, the World Vision project improves equitable access to safe water and sanitation in the rapidly growing, high density urban and peri-urban areas of the Municipality of Gwanda, and Cowdray Park and Robert Sinoyka in Bulawayo City Council. These areas are typically underserviced by WASH facilities. Residents continue to experience poor water access, limited sanitary facilities and display poor sanitation practices. World Vision, in collaboration with local government and other stakeholders, will sustainably improve WASH infrastructure and practices through delivery of enhanced water and sanitation facilities as well as hygiene awareness and education campaigns to drive behaviour change.

In PNG, World Vision works with remote subsistence villages/communities which are generally accessible only by river, in Western Province of Papua New Guinea. These communities have been adversely impacted by prolonged underinvestment in WASH, resulting in very low access to safe WASH facilities. Hygiene is a particular concern and diseases such as cholera and tuberculosis have had significant and ongoing impact. Civil society is under-developed, representing an opportunity to build governance, capacity and other social capital, for sustainable improvement in WASH. The project aims to increase access to safe water and improve sanitation and hygiene behaviours for target communities, schools and health clinics.

Baseline data collection in WASH projects

Collecting baseline data is an activity undertaken by many WASH programs to inform program planning, implementation and for monitoring and evaluation purposes. Some WASH organisations use household surveys to collect quantitative information regarding household demographics and current WASH access and practices. This is often complemented by qualitative information collected through community meetings, focus groups and key informant interviews.

Given the particular challenges described above with identifying people with disabilities, the UN Washington City Group on Disability Statistics has developed and tested a set of six questions (called the Washington Group Short Set of Questions on Disability (WGSS)) that ask about difficulties with performing basic functions, rather than about disability directly. Whilst the WGSS Questions were originally designed for use in national censuses, in the absence of alternative methods, they are also increasingly being integrated into other data collection processes. Thus far however, there is limited

9

documented evidence as to the usefulness of the questions in these contexts, or the most appropriate ways to use them.

The Washington Group Short Set (WGSS) of Questions on Disability

Due to stigma and different cultural understandings of the term disability, identifying disability is complex and cannot reliably be identified through asking questions such as ‘do you have a disability?’

The UN Washington City Group on Disability Statistics developed and tested a group of six questions (known as the Washington Group Short Set (WGSS)) that can be used in surveys to identify people at risk of disability. The questions ask how much difficulty a person has performing basic actions (walking, seeing, hearing, understanding, self-care and communication). Response categories range from ‘no difficulty’ to ‘cannot do at all’.

The WGSS questions and further information are available at www.washingtongroup-disability.com

Methodologies of collecting baseline information about people with disabilitiesAll five WASH projects conducted baseline household surveys and included the WGSS questions in these to some extent. Organisations also incorporated disability-specific questions into other data collections tools and processes, such as interviews, focus groups and infrastructure accessibility audits. In some cases, a disability-specific situational analysis was completed. The following provides a brief overview of the processes used in each project.

WaterAid PNGIn PNG, WaterAid conducted a household survey which included the WGSS questions. WaterAid and implementing partners then visited project sites and worked with local water committees to confirm and finalise a list of people with disabilities. WaterAid then invited people with disabilities to participate in a disability-specific situational analysis, conducted in partnership with CBM and the DPO. During this two-day workshop, modified versions of WaterAid’s standard participatory tools were used to identify challenges people with disabilities in the project areas experience in accessing WASH.

WaterAid Timor-LesteIn Timor-Leste, WaterAid conducts a household survey each time they commence activities in a new community. Initially this directly asked if a household member had a disability. This process was revised in July-August 2014 to instead include WGSS questions. If a person with disability is identified, WaterAid then conducts a follow up visit to the family to refer to disability services. The WGSS questions have also been used in engagement with key informants (i.e. village Chiefs).

World Vision Sri LankaIn Sri Lanka, World Vision conducted a household survey which included the WGSS Questions. If these questions identified a person with disability, additional follow up questions specific to water, sanitation and hygiene practices were asked for each person with disability identified. Additional questions were also included to identify who in the household had a disability (e.g. sex, age). A

10

qualitative disability assessment is also being undertaken during 2016 using a combination of focus groups and interviews to gain further information on WASH practices.

World Vision ZimbabweIn Zimbabwe, World Vision conducted a household survey which included the WGSS questions and an additional disability-specific question regarding whether adaptations had been made to WASH facilities to make them more accessible to those identified at risk of disability. Disability questions were also included in some key informant interviews. A qualitative disability assessment (focus groups and interviews) was then undertaken with people with and without disabilities identified during the household survey and using DPO contacts. This was done in partnership with CBM and the DPO. Following this, the DPO also completed accessibility audits of existing community infrastructure and made recommendations for improvements. World Vision PNGIn PNG, World Vision conducted a household survey which included the WGSS questions and an additional disability-specific question regarding whether adaptations had been made to WASH facilities to make them more accessible. When a household member with a disability was identified, they were asked whether they could be contacted again, however follow-up didn’t necessarily take place. WASH Assessments were completed for schools, which included questions about the number of children with disabilities in schools and an assessment of whether infrastructure was physically accessible. Questions about disability were also asked during community discussions to gain an understanding of community attitudes towards people with disabilities. A monitoring visit in 2016 included focus group discussions with people with disabilities in several communities. In 2016, interviews will be conducted with people with disabilities identified during the baseline survey to gain further information about WASH needs.

11

Key lessons on disability inclusive data collection

1. Including people with disabilities in data collection teams resulted in positive outcomes for people with disabilities, their communities and other stakeholders, but needs to be resourced appropriately

The WASH projects all included people with disabilities as active participants (rather than merely as subjects) in data collection processes to some degree, generally by establishing partnerships with DPOs. In Zimbabwe and Sri Lanka, people with disabilities were recruited and trained as data collectors to conduct household surveys alongside people without disabilities. In Timor-Leste, WaterAid also employed people with disabilities in their core staff. In PNG, World Vision partnered with a DPO who provided input into the household survey questions and a DPO representative participated in the process of piloting the questions and training data collectors. WaterAid in PNG involved DPO representatives in qualitative data collection by co-facilitating the disability situational analysis workshop.

Including people with disabilities in data collection increases their confidence and can have immediate outcomes in communities by challenging negative attitudes about their capacity

All projects reported positive outcomes from having people with disabilities involved in conducting baseline data collection, particularly with regards to challenging negative attitudes and raising awareness about disability (evident in Timor-Leste, Zimbabwe and PNG). For people with disabilities, it increased their sense of self-worth, confidence, and knowledge of WASH and development projects more generally. This also increased their standing in the community. For example, in Sri Lanka, people with disabilities recruited as data collectors reported that prior to being involved in the project they had lost their confidence and thought they had nothing to offer their communities. Being involved as data collectors gave them back their dignity.

Involving Disabled People’s Organisations resulted in new forms of collaboration with NGOs and governments, helping to change their attitudes and approaches towards people with disabilities

In Sri Lanka, involving the DPO strengthened its relationship with the government and changed attitudes of government employees as they witnessed people with disabilities conducting their own research and presenting findings confidently. This also helped start to change the government’s attitude towards people with disabilities, seeing them as active change agents. In Zimbabwe, being involved in the baseline data collection helped strengthen the DPO’s relationship with World Vision Zimbabwe. This helped staff members see the potential of involving people with disabilities in the project and led to a more formal partnership arrangement. DPOs have also used the qualitative data collected for the baseline to provide training to senior municipal staff and other stakeholders, thereby also influencing their attitudes.

Involving people with disabilities in data collection also increased the quality of data collected

In the WaterAid PNG project the DPO representative led introductory activities during workshops, which helped participants with disabilities have confidence to introduce themselves and discuss

12

issues related to WASH access. She also helped NGO staff become familiar with how to interact with people with disabilities, and was able to assist with data analysis by providing a unique interpretation of the results from her perspective as a person with disability. In Timor-Leste, a WaterAid Timor-Leste staff member with disability was able to model inclusion by effectively supporting households with inclusion strategies and making referrals to disability services. Including people with disabilities in data collection and analysis in Zimbabwe meant that the findings could be tested and contextualised for accuracy.

Including people with disabilities as data collectors also requires additional time and resources

While including people with disabilities clearly had significant benefits, it also presented logistical challenges and required additional time and resources. In particular, logistics associated with transporting a team with diverse impairments over rough terrain was reported as a challenge in both Zimbabwe and Sri Lanka. In the World Vision PNG project, this restricted the extent to which people with disabilities were directly involved in data collection as the communities visited were very remote and difficult to access.

Sufficient time and resources also needed to be dedicated to brokering relationships, building trust and confidence among organisations, training DPO representatives in data collection methodologies and ensuring they understand why certain methodologies are being used. While this was also the case for other NGO staff, in many cases people with disabilities were participating in data collection for the first time and required additional support.

It was also sometimes a challenge for DPOs to navigate undertaking this work in addition to their existing work priorities, particularly given this was a new area of work which required investment of time and energy into learning new concepts and skills. Implementing partners can assist DPOs to consider their role in data collection by instigating a discussion on how they want to be involved and encouraging them to also be mindful and realistic about existing workloads.

Recommendation 1: Involve people with disabilities/DPO representatives in data collection processes as much as possible, not just as ‘subjects’ of data collection.

Recommendation 2: Allow sufficient time to discuss and sensitise DPOs to the methodologies used and provide training for data collectors to undertake their role.

Recommendation 3: Think through how data collectors with disabilities will travel to project sites, taking account of time needed to collect and drop off people, and additional space required in vehicles for support people and assistive devices. Note that even in contexts where public transport is available, this may not be accessible to enumerators with disabilities and transport may need to be provided.

Recommendation 4: Work with DPOs to develop a realistic and clear understanding of their involvement, role and responsibilities. Where needed formalise this through a clear Memorandum of Understanding.

13

2. Collecting data on people with disabilities in communities needs to be done sensitively

Disability is a complex and evolving concept4 which is understood differently across cultures and communities. In many communities, people with disabilities are not ’classified’ as a separate group and the word ‘disability’ is not widely used or understood. Instead, people may have difficulty doing certain tasks because they are ‘elderly’ or for another reason. In the WaterAid PNG project and in Timor-Leste, project staff found it more useful to talk about difficulties with aspects of functioning (as recommended by the Washington Group) when working with the community as opposed to talking about ‘disability’, which would potentially create a whole new group in the communities which previously did not exist.

Increasing the visibility of people with disabilities by including them in community consultations for the first time also needs to be closely monitored to ensure that any backlash from communities for challenging social norms is addressed. For example, in Zimbabwe, community members wrongly blamed people with disabilities as a reason for a project changing direction and 'punished' them by excluding them from community funeral contributions. In another example, DPOs warned that parents bringing children with disabilities to a meeting for the first time may be ridiculed for being 'cruel' by bringing children out in public. Such issues can be addressed through adequate awareness raising, but it is important to identify these issues through strong monitoring. This will help to ensure that people with disabilities will not be further marginalised within their communities as a result of the inclusive WASH project.

Recommendation 5: Be aware of the language you are using when talking about disability within communities, to avoid creating new terminology or grouping people in stigmatising ways. Referring to ‘difficulties’ with doing certain tasks rather than using the word ‘disability’ when engaging with communities may help.

Recommendation 6: Aim to consult both people with disabilities as well as the wider community and be clear about the purposes of consultations. In doing so, it is important to provide “safe spaces” for people with and without disabilities to bring up and discuss issues and community dynamics or beliefs around WASH use which may impact on the project. Close monitoring of consultation processes are vital to identifying issues as they arise.

4 UN Convention on the Rights of Persons with Disabilities.

14

3. Household surveys need to collect individual level data about access to WASH in order for that data to be disaggregated by disability. However, inclusion of the WGSS questions is also useful for identifying households which include people with disabilities who can then be followed up throughout the project.

All projects included the WGSS questions on disability in their household surveys. Most projects also added an additional question regarding difficulty using hands, as this was seen as being of particular significance to WASH tasks such as collecting water and hand washing and is not included in the standard WGSS questions.

The WGSS questions were generally effective in identifying people with disabilities in communities, although there were variations.5 In Timor-Leste, implementing partners reported they were able to find more people with disabilities using this method than previous methods which involved directly asking about disability. However, they also found that trust is a key factor and this needs time to be established. In some cases families did not disclose having a family member with a disability (even when the WGSS questions were asked) until during project implementation once the organisation had demonstrated what the program was doing and that it would deliver results. In PNG, World Vision found very low rates of disability using the recommended cut-off, so decided to also include those who identified as having ‘some difficulty’ performing certain tasks. In contrast, WaterAid in PNG found very high rates of disability however identified that this may have been due to a misconception that community members would receive some benefit in the project if they identified as having a disability.

The SDGs and some donors require data to be disaggregated by disability, sex and age wherever possible, to assist in determining who does and does not benefit from development programs and encourage a focus on universal access. This was one of the common objectives of including the WGSS in the baseline surveys of the WASH projects. However, the fact that surveys collected information about households (generally with the head of household) rather than collecting individual-level data about household members, meant that individual differences in access to WASH within households were not identified and data could not be fully disaggregated by disability (or age and sex). The surveys could generally only identify whether a household included a person with disability and whether a household had access to WASH, rather than identifying whether a particular person with disability within the household had access to WASH. In some cases, surveys also did not identify how many people within a household had a particular functional difficulty, or the age/sex of those people which prevented data disaggregation.

5 WGSS questions were designed to be asked at an individual level. When asked to the head of the household this requires someone else to make a judgement about whether another person has difficulty seeing/hearing/ remembering etc. Further testing and analysis is required to determine the potential impact this may have on the reliability of the questions.

15

Sri Lanka: Capturing individual level data on disability and access to WASH within a household survey

The baseline survey in Sri Lanka successfully captured information on disability and access to WASH by including some individual level questions in the survey. In this questionnaire, an initial screening question was asked to the head of the household ‘does anyone in this household have any difficulties in seeing, hearing, walking, remembering, self-care, communicating, or in using hands?’ Respondents were then asked how many people in the household experienced any difficulties (in total)6 and were then asked the WGSS questions about each person, along with their age and gender. Additional questions were also included as to whether and how people with difficulties accessed water and sanitation facilities and if they required assistance. While this methodology still did not enable full disaggregation of survey data by disability (as the remainder of the survey was only asked at household level) it did capture a far greater level of detail on the WASH needs of people with disabilities within the household and provided a solid baseline on which to monitor disability inclusion.

Recommendation 7: With the Sustainable Development Goals’ renewed focus on access to water and sanitation for all, WASH implementing organisations should reconsider their approach to collecting baseline information to better identify and seek to monitor inequalities to accessing WASH within households, rather than only collecting household level information.

Household surveys were an important mechanism to identify people with disabilities for inclusion in qualitative data collection processes

Finding people with disabilities within communities is often a key challenge reported by NGOs undertaking development projects, as social norms, stigma and the physical environment often mean that people with disabilities remain hidden within households. While using the WGSS questions alone did not enable full disaggregation of data, the fact that they were generally effective in identifying households which included people with disabilities was very useful for projects and opened up a range of opportunities for their inclusion in the WASH projects. For example, in most projects, people with disabilities identified during household surveys were then invited to participate in other data collection processes such as interviews, focus groups and community meetings. One key challenge in this regard was using an identifier that enabled the household to be located again after the survey. In Zimbabwe, following up people with disabilities identified during the initial survey proved to be a challenge due to the urban/peri-urban context where there was no systematised numbering of households. In other settings this was not a significant concern.

People with disabilities were also recruited to be involved in qualitative processes through DPO networks, however this sometimes led to a skewed representation of different impairment types (if for example DPOs had more networks amongst people with mobility and vision impairment rather than psychosocial impairments). This further highlights the benefits of using the WGSS questions to identify households with people with disabilities in addition to DPO networks to get a more representative sample of people with disabilities for inclusion in qualitative processes.

6 This question may result in ‘over-reporting’ disability as it refers to those with ‘any difficulty’ in functioning rather than just those with ‘a lot of difficulty’ or that ‘cannot (see/hear etc) at all’.

16

Recommendation 8: Even where disaggregation of data will not be possible, the WGSS questions should be included in household surveys for the purposes of identifying households with people with disabilities for future follow-up. In doing so, ensure processes are used during surveys to enable identification of households where people with disabilities have been identified. Undertaking follow-up analysis at the same /similar time as when the survey is being undertaken will help to avoid difficulties in finding households.

Recommendation 9: If relying primarily on DPO contacts to find people with disabilities within communities, consider what impairment types are/are not represented and aim for a broad representation. This may require contacting a range of DPOs or using additional key informants.

4. There were challenges with but also unexpected findings from using the WGSS questions in surveys

In most countries, this was the first time the NGOs and implementing partners had used the WGSS questions in a household survey, and while they are often promoted as being easy to use, in reality it turned out to be quite complex. All NGOs reported some challenges in using the questions as they were intended and in analysing the results. An overview of some of the challenges that NGOs encountered is provided below.

Finding the time and resources to build understanding of the rationale of the WGSS questions and follow testing and translating protocols

The WGSS questions have been developed to overcome particular challenges and cultural sensitivities in gathering data on people with disabilities, and are the result of extensive testing to ensure they reliably identify people at risk of disability. This means the exact wording of each question has been chosen very carefully. As words and concepts are understood differently across cultures and can be difficult to translate, the Washington Group recommends the WGSS questions are translated by a qualified/experienced translator and cognitively tested prior to their use. While this is best practice, in reality, development organisations are often pressed for time and funding and are under pressure to get their baselines completed quickly. For the WASH projects, this meant translation and testing protocols could not always be followed.

Additional time also needed to be invested in ensuring organisations using the questions understood the rationale behind their development, as well as the risks of changing the questions without proper testing. In some cases when this was not addressed sufficiently, changes were made to the wording of the questions without adequate testing or additional questions were added. In Sri Lanka, the DPO completed a parallel disability survey using an alternative approach to identifying disability as they did not believe the WGSS questions were effective. Spending additional time with the DPO to build their understanding of the approach could have prevented this duplication of effort.

Balancing the need to ensure consistent use of the WGSS questions with the preferences of data collectors regarding survey tools and guidance materials

While the Washington Group has recommended the WGSS questions are used in a certain way to maximise their effectiveness, this needed to be balanced with the preferences of data collectors. For

17

example, in Zimbabwe, data collectors expressed a strong preference for the survey tool to be provided in English for them to translate on the spot, as they were not accustomed to reading in the local language. While this was not ideal (as it is likely to have produced many inconsistencies in the way questions were translated), it was important to provide survey tools to the data collectors in a format they were comfortable with. Similarly, in Timor-Leste, it was not considered appropriate for data collectors to carry large amounts of paper given the distances travelled within communities. In this circumstance it was difficult to provide sufficient guidance to help data collectors implement the WGSS questions effectively.

Selecting and using a ‘cut-off’ to determine disability status

The baseline surveys of the WASH projects were all structured differently. Some surveys were designed so that the responses to the WGSS questions would trigger asking additional questions once a person had been identified as having a disability. While this enabled more specific information to be captured around the situation of people with disabilities, it also required data collectors to interpret responses to the WGSS during data collection, which in some cases became a point of confusion. This is because the WGSS questions are not ‘yes/no’ but instead include four response categories (no difficulty, some difficulty, a lot of difficulty, cannot do at all). Those using the questions are required to select a ‘cut-off’ to determine disability status. For large/national level surveys this is generally recommended to be when a person answers ‘yes - a lot of difficulty’ or ‘cannot do at all’ to at least one question; however it can also be when a person answers ‘yes-some difficulty’ if programs are interested to identify people with more minor functional difficulties.

While the cut-off can generally be selected during data analysis, it must be clear to data collectors when additional disability specific questions should be ‘triggered’ during the survey. In one project for example data collectors were not clearly instructed on what cut-off to use and unintentionally used different cut-offs in different project locations, which meant the additional questions on disability were not asked consistently.

Benefits of using a more sensitive cut-off rate for the WGSS questions

The projects found that there may be a need to change the ’cut off’ to be more sensitive in WASH projects as people with minor difficulties performing tasks in fact reported significant difficulties accessing WASH. In PNG for example, WaterAid unintentionally classified people who reported ‘some difficulty’ to one of the WGSS questions as people with disabilities, meaning that people with relatively “minor” disabilities were included in their disability analysis. While this was unplanned, it lead to revealing findings as people who would not traditionally have been thought of as having a disability reported a range of substantial difficulties accessing WASH. This was particularly the case for women who were still culturally expected to undertake WASH work, and undertake caring and other household duties despite the difficulties they faced. These women reported spending significantly longer undertaking WASH tasks (particularly collecting water) because they were not able to carry the same quantity of water as others, took longer to travel to the water source and had to make more return trips. Similarly, by also consulting with carers of people with disabilities, new and unique insights were gathered on their specific needs which can inform project design and implementation.

18

Using the language of the WGSS questions to talk about disability in communities also helps capture difficulties faced by elderly people

In Timor-Leste, using the WGSS questions enabled implementing partners to also capture information about elderly people who were facing difficulties accessing WASH. Previously their needs may have been missed as they may not have identified as people with disabilities, instead considering their impairments to be simply a consequence of ageing. This highlights the benefits of focusing on difficulties with functioning rather than talking directly about disability.

Recommendation 10: Ensure adequate time is spent discussing and explaining why the WGSS questions are recommended for use in baseline surveys, to increase buy-in of local DPOs, implementing partners and data collectors. This should highlight the importance of not changing the questions unnecessarily and of appropriate translation and testing.

Recommendation 11: Prior to use, provide face to face training for data collectors on how to ask the WGSS questions, why they are being asked and how they will be expected to interpret the questions (if relevant).

Recommendation 12: If data collectors are required to interpret the WGSS questions during the survey in order to ‘trigger’ additional questions, ensure the more sensitive cut-off is used (e.g. answering ‘some difficulty’ to at least one question) so that people with minor functional difficulties are also asked about their access to WASH. Where possible, also state this cut-off on survey templates to remind data collectors.

5. A combination of quantitative and qualitative data collection processes produces the most useful baseline data on people with disabilities

Even in Sri Lanka, where additional questions were included in the household survey and the WGSS questions were asked at an individual level, this only resulted in a clear picture of who did/did not have access to WASH rather than what, why, or how best to address the barriers faced. Collection of qualitative information on the experiences of people with disabilities in accessing WASH facilities was critical to enable an understanding of the type of difficulties people with disabilities were experiencing in accessing WASH (and therefore strategies to address these).

All projects completed (or plan to complete) some type of qualitative disability assessment as part of the baseline assessment. In Zimbabwe, this included interviews and focus groups with people with disabilities and accessibility audits of WASH infrastructure. In the PNG World Vision project, disability questions were included in community mapping activities and in WASH infrastructure audits. In Timor-Leste, a follow up is done with people with disabilities identified through the survey to understand what support they need and refer them to services where possible and to assist with further monitoring. In the PNG WaterAid project, a participatory community workshop was held for people with disabilities.

19

Projects which completed qualitative data collection specifically with people with disabilities ended up with richer baseline information about barriers to WASH

Where most projects completed specific (separate) qualitative processes for people with disabilities, World Vision PNG included questions about disability within standard community mapping processes. These aimed to gather information about community attitudes towards people with disabilities and access to WASH. The data gathered through these discussions was useful in understanding broad community attitudes but did not provide sufficient information on barriers to accessing WASH on which to adapt project designs. World Vision PNG therefore also held additional interviews directly with people with disabilities in some communities to supplement this information.

Standard community consultation and planning tools can be used with people with disabilities with minor adaptations

In PNG, WaterAid used a slightly modified version of their existing participatory WASH planning tools which partners and communities were already familiar with for their disability situational analysis. While some additional disability-specific questions were added, overall, these were found to be appropriate to use with men, women and children with disabilities and were successfully combined with focus group discussions, interviews and observation techniques (to go into more depth on the identified issues). The exception to this was for some image-based participatory tools used for children with disabilities where some further adjustments would need to be made to fully cater for children with vision impairments.

In Zimbabwe and PNG, World Vision adapted standard infrastructure audits of school WASH facilities to include a question on accessibility. While these did not assess accessibility in great detail, they provided an initial indication of accessibility on which to base further analysis. In Zimbabwe, this was supplemented with a targeted accessibility audit which was undertaken of community infrastructure by the DPO. This successfully raised awareness of what was lacking in school and as a result, two schools have now completed refurbishments beyond just WASH facilities, and it is expected that more children with mobility issues can now attend school. While this produced a good outcome, it is recommended that this type of activity is included in the initial baseline visit to schools instead of as a separate activity.

20

Participatory tools used during the disability situational analysis with Water Aid in PNG

Community mapping: brought out particular issues for people with disabilities with regard to access to water holes, location of toilets etc and practices around sanitation and hygiene.

Timeline exercise: gave insights in how a typical day looks like, and what challenges people with disabilities may typically experience during the day to access and participate in WASH activities.

Story telling for children using standard WASH picture cards : brought out issues related to access to WASH for children with disabilities, WASH practices of the children themselves and their parents, gender roles, their own attitudes towards these roles and how they perceived themselves as children with disabilities compared to other children. It also brought out their personal priorities in relation to WASH and community participation.

6. Collecting baseline data on people with disabilities led to more inclusive WASH programming and had a range of positive outcomes

In addition to being a source of data to monitor progress, the process of collecting baseline data has in itself been a catalyst for projects to increase their commitment to disability inclusion and has led to a range of outcomes. For example:

In the PNG World Vision project, some accessible toilets have been constructed; while as a result of implementing the Healthy Islands approach in their communities, people have began to consider the needs of people with disabilities in broader community activities addressing hygiene behaviour.

In Timor-Leste, people with disabilities have been involved in deciding where community WASH infrastructure will be located, support has been given to households to problem solve accessibility issues, and materials have been distributed to assist households to construct accessible latrines. WaterAid has now been asked to share its experience using the WGSS questions by speaking at the working group on the National Census data process. It has also been able to draw on this experience to influence the Government’s national rural WASH program (BESIK).

In the PNG WaterAid project, an advocacy booklet is currently being prepared to share with water committees.

In Sri Lanka, the baseline helped to identify which people with disabilities had no access to latrines and enabled the project to assist those people to apply for them.

21

Zimbabwe: Baseline data prompts a disability inclusive approach

In Zimbabwe, data from the baseline and the accessibility audits have prompted the construction of accessible public toilets, and two schools have refurbished latrines with accessibility features. Information from the baseline assessment is also being used in training workshops with council workers and WASH sector staff to influence policy and planning. Furthermore, Gender and Social Inclusion (GESI) Champions have been established in each community to strengthen representation at community meetings and identify households who have a person with a disability to facilitate access to disability services and advocate for government support. GESI Champions are now being consulted by the Local Authority on disability and gender issues beyond the WASH project. The Local Authority is even planning to replicate the GESI model to other communities beyond the project area, as they have found it an excellent way to consult with people with disabilities.

While all projects have seen benefits from conducting disability inclusive baselines, a range of factors have influenced the extent to which baseline information informed project design and implementation.

Baseline data collection needs to be conducted early and seen as a tool to guide project implementation

Timing of baseline data collection was a big factor in the extent to which it could inform project design. For some projects, delays in conducting the baseline data collection (sometimes beyond the project’s control), analysing the data and sharing the findings with implementing partners limited the extent to which it has informed project implementation. For example, in Sri Lanka there was a need to ensure local government personnel were given the opportunity to contribute to the tools and methodologies used. In some cases, baseline surveys were also considered stand-alone activities rather being seen as a source of data to inform projects. Encouraging and actively supporting projects to conduct baselines data collection early and see the value of baseline data as a project planning tool would assist with strengthening its use in the future.

Analysis of baseline data related to disability needs to be resourced appropriately

Analysis of both quantitative and qualitative data related to disability is complex and was also reported to be a challenge. In some cases this was because the actual questions asked did not provide the type of information sought from the baseline (e.g. the WGSS questions were not coupled with other questions in a way that allowed for data disaggregation) whereas in others, further training and support was needed to analyse and interpret the information effectively.

Recommendation 13: Conduct baseline data collection and share findings as early in the project as possible, and support implementing partners to think through what the findings mean for project implementation.Recommendation 14: Plan for how baseline data will be analysed before it is collected and ensure adequate training and support is provided to implementing partners and DPOs to facilitate this.

22

Discussion

Benefits of disability inclusive baseline data collection

Overall, conducting disability inclusive baseline data collection produced a broad range of benefits to projects on multiple levels. The action learning approach used for the baseline processes led to greater disability awareness and capacity among NGO and partner staff members. This led to increased confidence to talk about disability rights in a more open manner, particularly when meeting with community leaders, and awareness of the need to consider people with disabilities in WASH programming. At a community level, the baselines helped to identify people with disabilities, including groups that may have been left out previously, such as people with minor disabilities and older people, but who turned out to face significant access issues. Baseline data collection processes also helped to challenge negative attitudes of community members, NGOs and government stakeholders towards people with disabilities.

Finally, use of the findings resulted in more inclusive programming and improved project and development outcomes. These included the construction of accessible WASH infrastructure, successful use of the baseline data for advocacy and training purposes with stakeholders including government, and involvement of people with disabilities in local government decision-making processes.

Despite these positive outcomes, conducting disability inclusive data collection was not without its challenges and required additional resourcing in terms of time and budget, particularly as organisations learned new tools and approaches. Furthermore, there are a few major areas where further analysis is recommended.

Need to explore alternative methods for measuring intra-household inequalities

The design of household surveys used by the WASH implementing partners collected information primarily at a household level (e.g. whether the household has a latrine). This was found to be inadequate to disaggregate data on access to WASH by people with disabilities and therefore understand the extent to which inequity may occur within households. Capturing only household-level data assumes that everybody within a household experiences poverty in the same way and has access to the same resources, whereas we know that gender, disability, age and other family dynamics impact on this. If development partners are serious about ensuring WASH programming reaches everyone, it is vital to reflect more on how household surveys can better collect information on individuals within households and differences in access to WASH. Further analysis is required to determine the best methods of achieving this, acknowledging the significant expense associated with undertaking large surveys or other data collection processes. This should include exploration of other measures such as the Individual Deprivation Measure (IDM) developed by the International Women’s Development Agency7 and the Rapid Assessment of Disability (RAD) developed by the University of Melbourne’s Nossal Institute for Global Health and Centre for Eye Research Australia8.

7 See https://www.iwda.org.au/assets/files/IDM-Poster_digital.pdf 8 The RAD survey consists of household and individual questionnaires. Each head of household is invited to complete the household questionnaire, while individuals residing in the household are then invited to complete the individual questionnaire with the interviewer that explores the situation of people with disabilities that can be included in household surveys when a person with disability is identified.

23

Balancing qualitative and quantitative methods

Secondly, given the time and budget constraints which come with community development projects, program implementers need to carefully consider the balance of qualitative and quantitative data that should be collected, based on an analysis of what information is required to inform disability inclusion within that context. Quantitative data can be used to powerfully highlight inequalities in access and measure progress of some indicators towards inclusion. However, there is a danger that analysis stops at identifying people with disabilities and too much emphasis is placed on ‘classifying people’ as with or without disability which may create further separation and marginalisation of people with disabilities within communities by highlighting difference. Sufficient attention needs to be given to identifying and understanding the situation of people with disabilities more broadly, and the barriers they may face in accessing WASH services. A strong analysis of the situation of people with disabilities will enable development of strategies and solutions to address challenges in accessing WASH that are contextually appropriate.

24