1

Micro-data access: Micro-data access:

Data Data Reconciliation Reconciliation

forforRobust Robust

EstimationsEstimations Expert Group Meeting on MDG Indicators Astana, 5-8 October 2009

2

Presentation overview:

IHSN tools ADP task 1 leading

to Task 2 Partnership

ADP/WHO Preparations for a

new workshop Implications of

greater data availability

3

A network of international agencies ($0.75-1m pa DGF)

A coordinating mechanism to:◦ Improve quality and use of household survey data in developing

countries /MDG monitoring◦ Harmonise international recommendations for survey design,

data analysis, etc◦ Produce and disseminate international good practices

…

A mechanism is needed to bring survey sponsors and survey users together. The recommendations propose creation of a Household Survey (HHS) Network, comprising the major sponsors of the global household survey programs... MAPS

4

In developing countries, survey data are produced mostly by official agencies for monitoring / assessing / targeting purposes

High dependency on external financial and technical assistance

Poor coordination creates problems:◦Accessibility and use (no re-purposing of data)◦Reliability and relevance◦Comparability across sources and over-time

5

Better use of survey dataLots of data are collected, but various issues must be

addressed to make them more useful/used: Limited information on what’s available (no catalogs of

surveys) Limited accessibility by secondary users (for legal,

technical, psychological and political reasons) Data often poorly documented, hence difficult or risky to use Reliability, comparability, relevance, timeliness, periodicity Loss of many datasets/metadata

Work program on data documentation, dissemination and preservation

6

Cataloguing of over 3,500 surveys by country, forerunner to NADA

1. Search by country

2. List all surveys

www.surveynetwork.org

3. Details on the surveys

7

Input-Metadata Editor: Input of data and metadata and includes bundling of reports, manuals and all Documents: Output to PDF and DDI

Output-CD-ROM Builder: Input of data and metadata in DDI: Output to HTML for CD-ROM or web dissemination

Catalogue-NADA (National Data Archive): Catalogue and link all the survey data and metadata and facilitate data access through on-line request forms

The IHSN Microdata Management Toolkit using the power of the DDI standard for metadata exchange:

8

Better quality of future surveys◦ Timing and sequencing (part of NSDS)◦ Comparability (over-time, across countries),

and consistency across data sources◦ Reliability (use of improved methods)

Work program on coordination and harmonization- Guidelines for dissemination in NSS- International Question Bank

9

An Accelerated Data Program (ADP): Country Implementation of IHSN Tools

the Accelerated Data Program (ADP) supports statistical data relevant for policy design, monitoring and evaluation, by making better use of existing data and aligning survey programs and statistical outputs to priority data needs. This goal will be achieved by:

Task 1: Building national capacity in micro-data preservation, analysis, anonymization, and dissemination;  

Task 2: Working with national data producers and secondary users on the production of updated estimates of key indicators, by further exploiting existing datasets and collecting new data.

At the country’s request,

10

ADP active ADP Inactive Expressed interest in ADP IHSN Tools used without ADP

Includes countries where ADP is implemented by at least one official data producer (NSO, line ministry).

11

ADP

NBS

2006 2007 2008 2009

2003 FOS starts Statistical Master Plan

2005 Merger of FOS and National Data Bank to NBS

2006 NBS attends regional data archiving workshop in Ghana

2006 Request to enter ADP pilot program

March 2007ADP Project document

July 2007ADP Documentation Workshop

October 2007 Inventory of Data

June 2008 National Archiving Workshop

Sept 2008 NADA Launch

12

Task 1: Achievements in Nigeria

• Over 20 surveys documented• Inventory reports over 100 data sets in various ministries• Review process before publishing• Draft data access policy• NADA with 14 reviewed surveys• Microdata available for many surveys on the NADA

See: http://www.nigerianstat.gov.ng/nada/

13

Task 1 Primary Outcome: Measuring Data Demand in Nigeria

Nigeria on-line survey databank - Number of downloads of data (two surveys) in 3 months, by origin of request

University of IbadanInteSOLCentre for Management Development (CMD)BUKGISKONSULTHidden WisdomObafemi Awolowo UniversityUniversity of Agriculture, AbeokutaAbu ZariaCBNF.U.T MinnaFederal Ministry of Agriculture & Water ResourcesFMAWRIMBLInternational Food Policy Research InstituteKogi State UniversityNational Bureau of Statistics, Calabar

Data users in Nigeria

Total: 89 downloads

14

Task 2: Going further in Nigeria

The ADP Task 2 provides support for the assessment of existing data in the country by specialized national and international researchers. The expected outputs are not only analytical work and policy briefs, but also and mostly a detailed assessment of the strengths and weaknesses of the data (in terms of reliability, relevance, comparability, timeliness, accessibility), and recommendations for improving data collection and analysis in the country.

With Task 1 completed, the ADP was looking for a sector and a partner to validate the work and extend the process further…and this….…is what the presentation is about…a partnership & the results of data access

15

JMP/ADP Start of a JMP/ADP Start of a partnershippartnership

The JMP/ADP made a perfect match!

The JMP was in search of New Data:

•JMP sought help from the IHSN

•Through the World Bank Development Data Group the JMP wasProvided with WAT SAN datasets for 100 countries

Nigeria:

•The JMP had done a traditional workshop in 2007

•Was moving toward a workshop focusing on reconciliation and harmonization

•Through the IHSN and the ADP the JMP found a partner

16

Total populationAccess to improved

sanitation technologies

Toilets flushing to sewerage systems

Affordable

Access to improved sanitation technologies ensuring privacy (not

shared)

Sustainability

Well Well regulated regulated systemssystems

Toilets flushing to sewerage with

effective wastewater treatment

What is measured and what What is measured and what not?not?

17

Major reasons for discrepanciesMajor reasons for discrepancies

Differing sources of data◦ NSO – user based data◦ Sector – provider based data

Differing methodologies◦ Even with user based data

Countries sometimes use single data pointDiffering population estimates

◦ Most recent census vs. UNPD estimatesDiffering definitions of urban/rural

◦ NSO and Sectors not always agree Differing definitions

18

Progress in Sanitation

0%

20%

40%

60%

80%

100%

1990 2006Year

Open defecation Unimproved

Shared Improved pit

Septic tank Sewer

National Definition

With Access Sewer connection Septic tank VIP/improved pit Shared facilities Unimproved facilities

Without Access Open defecation

52%

Differences due to differing definitions

80%MDG Definition With access

Sewer connection Septic tank VIP/improved pit

Without access Shared facilities Unimproved facilities Open defecation

Comparing JMP and national definitions Comparing JMP and national definitions (after closing data gaps)(after closing data gaps)

19

Can JMP and country Can JMP and country

authorities authorities

agree on water and agree on water and

sanitation ladder?sanitation ladder?

20

o Once the partnership was defined after meeting with JMP and Nigeria oIdentify the national agencies monitoring the sector and build consensus

UNICEF, Active Water Sanitation Sector, Water Aid, JMP (Building on the work of JMP reconciliation in 2007)oReview any previous harmonization work done in the sector UNICEF Harmonization work and indicator publication

o Identify and prepare the data setsoReview the quality of the datao Review published survey reports and questionnaires

oOrganization of the workshopoIdentify technical resources

21

ADP identified six more surveys with national Water and Sanitation data and added these to the 3 DHS surveys.

Old JMP Estimates based on:

•DHS 1990•DHS 1999•DHS 2003

New JMP Estimates added thedata available in the NADA

•Child Labour Force Survey 2000•Nigeria Living Standard Survey 2004•General Household Survey 2006•Core Welfare Indicator Survey 2006•Multiple Indicator Cluster Survey 2007•Sector Baseline Survey 2008

22

Survey Year Sample Size (data file)

Producer

DHS 1999 7,647 NPopC/Macro

CLFS 2000 20,830 NBS

DHS 2003 7,225 NPopC/Macro

NLSS 2003 19,158 NBS

CWIQ 2006 77,062 NBS/World Bank

GHS 2006 18,826 NBS

MICS 2007 28,603 NBS/UNICEF

Baseline 2008 192,575 Ministry of Water Resources

23

Survey Definitions Improved water

DHS 1999 Assumes that water from pipes, private wells, boreholes, and springsare not contaminated.

53.1%

CLFS2000 Safe is not defined. Unsafe sources are defined as ponds, stream or river and rainwater.

69.3%

DHS 2003 Sources of water expected to be relatively free of disease are piped water and water drawn from protected wells and deep boreholes.

42%

NLSS 2004 Pipe water, untreated pipe, borehole and protected well are considered safe

60%

CWIQ 2006 Safe water source is defined for households using treatedpiped water, bore hole/hand pump or protected well.

51.4%

GHS 2006 Safe water is constituted by those who use:piped treated water, piped untreated water, borehole with hand pump and well/spring protected water.

46.4%

MICS 2007 Piped into dwelling, Piped into yard or plot, Public tap/standpipe, Tubewell/borehole, Protected well, Protected spring, Rainwater collection, Bottled Water are considered safe

49.1%

BASE 2008 Safe water include: Household Connections, Boreholes with hand pump, Motorized borehole, Protected Dug well, Public Standpipe, Rain water harvesting, Protected Spring

24

Survey Water QuestionDHS 1999 What is the main source of drinking water for members of your

household?CLFS2000* Major Source of Drinking Water

DHS 2003 What is the main source of drinking water for members of your household?

NLSS 2004 What is the main source of drinking water for this household?

CWIQ 2006 What is the main source of drinking water?

GHS 2006 Major source of water for drinking and cooking

MICS 2007 What is the main source of drinking water for members of your household?

2006 Census What is the household’s main source of water for domestic use?

BASE 2008 What is your family’s main source of drinking water?

25

DHS 199911 Piped Into Residence/Yard/Plot12 Public Tap21 Well in Residence/Yard/Plot22 Public Well31 Spring 32 Stream33 Pond/Lake 34 Dam 41 Rainwater 51 Tanker (Truck)52 Tanker Vendor61 Bottled Water71 Borehole96 Other

CLFS 20001 Tap/pipe Inside house2 Tap/Pipe Outside House3 Tube/well4 Manual Well Protected5. Handpump 6.Ponds/Stream/River/Rainwater

DHS 200311 Piped Into Dwelling12 Piped Into Yard/Plot13 Public Tap21 Open Well In Dwelling22 Open Well In Yard/Plot23 Open Public Well31 Protected Well/Borehole In Dwelling32 Protected Well/Borehole In Yard/Plot33 Protected Public Well/Borehole41 Spring42 River/Stream43 Pond/Lake44 Dam51 Rainwater61 Tanker Truck71 Bottled Water96 Other

NLSS 20041 Pipe borne water treated2 Pipe borne water untreated3 Borehole/hand pump4 Protected Well5 Unprotected

CWIQ 20061 Pipe borne water treated2 Pipe borne water untreated3 Bore hole/hand pump4 Protected well5 Unprotected well6 Rain water7 River, lake or pond8 Vendor, truck9 Other

GHS 20061 Pipe borne water treated2 Pipe borne water untreated3 Borehole/hand pump4 Well/Spring Protected5 Well/Spring Unprotected6 Rain Water7 Streams/Pond/River8 Tanker/Truck/Vendor9 Other

MICS 200711 Piped into dwelling 12 Piped into yard or plot 13 Public tap/standpipe21 Tubewell/borehole31 Dug well/Protected well 32 Unprotected well 41 Protected spring42 Unprotected spring51 Rainwater collection61 Tanker-truck71 Cart with small tank/drum81 Surface water (river, stream, dam, lake, pond, canal, irrigation channel) 91 Bottled water 96 Other (specify)

2006 Census1 Pipe borne inside dwelling2 Pipe borne outside dwelling3 Tanker Supply/Water Vendor’4 Well5 Bore-hole6 Rain water7 River/Stream/Spring8 Dugout/Pond/Dam/pool9 Other

Sector Baseline 2008Protecteda. Household Connectionsb. Boreholes with hand pumpc. Motorized boreholed. Protected Dug welle. Public Standpipef. Rain water harvestingg. Protected SpringUnprotecteda. Unprotected Traditional hand dug wellsb. Unprotected wellsc. Vendor provided waterd. Bottled/sachets watere. Tanker truck provided waterf. Streamsg. Riverh. Pondi. Broken pipes

CompareCategories

26

27

Harmonized response categories between surveys (i.e. recoded and re-compared)

Confidence intervals on all recoded dataNew trend in access to improved water and

sanitationsCommon definitions (JMP based)Core Questions Defined

28

DHS 199911 Piped Into Residence/Yard/Plot12 Public Tap21 Well in Residence/Yard/Plot22 Public Well31 Spring 32 Stream33 Pond/Lake 34 Dam 41 Rainwater 51 Tanker (Truck)52 Tanker Vendor61 Bottled Water71 Borehole96 Other

CLFS 20001 Tap/pipe Inside house2 Tap/Pipe Outside House3 Tube/well4 Manual Well Protected5. Handpump 6.Ponds/Stream/River/Rainwater

DHS 200311 Piped Into Dwelling12 Piped Into Yard/Plot13 Public Tap21 Open Well In Dwelling22 Open Well In Yard/Plot23 Open Public Well31 Protected Well/Borehole In Dwelling32 Protected Well/Borehole In Yard/Plot33 Protected Public Well/Borehole41 Spring42 River/Stream43 Pond/Lake44 Dam51 Rainwater61 Tanker Truck71 Bottled Water96 Other

NLSS 20041 Pipe borne water treated2 Pipe borne water untreated3 Borehole/hand pump4 Protected Well5 Unprotected

CWIQ 20061 Pipe borne water treated2 Pipe borne water untreated3 Bore hole/hand pump4 Protected well5 Unprotected well6 Rain water7 River, lake or pond8 Vendor, truck9 Other

GHS 20061 Pipe borne water treated2 Pipe borne water untreated3 Borehole/hand pump4 Well/Spring Protected5 Well/Spring Unprotected6 Rain Water7 Streams/Pond/River8 Tanker/Truck/Vendor9 Other

MICS 200711 Piped into dwelling 12 Piped into yard or plot 13 Public tap/standpipe21 Tubewell/borehole31 Dug well/Protected well 32 Unprotected well 41 Protected spring42 Unprotected spring51 Rainwater collection61 Tanker-truck71 Cart with small tank/drum81 Surface water (river, stream, dam, lake, pond, canal, irrigation channel) 91 Bottled water 96 Other (specify)

2006 Census1 Pipe borne inside dwelling2 Pipe borne outside dwelling3 Tanker Supply/Water Vendor’4 Well5 Bore-hole6 Rain water7 River/Stream/Spring8 Dugout/Pond/Dam/pool9 Other

Sector Baseline 2008Protecteda. Household Connectionsb. Boreholes with hand pumpc. Motorized boreholed. Protected Dug welle. Public Standpipef. Rain water harvestingg. Protected SpringUnprotecteda. Unprotected Traditional hand dug wellsb. Unprotected wellsc. Vendor provided waterd. Bottled/sachets watere. Tanker truck provided waterf. Streamsg. Riverh. Pondi. Broken pipes

Recoded as Improved or Unimproved

IMPROVED50/50 SPLIT

29

Significant differences to be explained are indicated in light blue.

•NLSS may be a problem with weighting coefficients.•DHS2003 may be a sampling issue or a recoding issue due to a high level of ambiguous categories for reclassification.•MISC2007 takes into account shared toilet facility as unimproved. This cannot be properly measured in other surveys.

IMPROVED WATER, NIGERIA

SurveyRatio Estimate 95% Confidence Interval

MICS07 0.504 0.481 0.527CWIQ06 0.529 0.519 0.539GHS06 0.483 0.461 0.505NLSS04 0.607 0.593 0.620DHS03 0.413 0.370 0.457

CLFS2000 0.508 0.486 0.530

IMPROVED TOILET, NIGERIA

SurveyRatio Estimate 95% Confidence Interval

MICS07 0.443 0.422 0.464CWIQ06 0.567 0.559 0.575GHS06 0.496 0.475 0.517NLSS04 0.601 0.589 0.614

CLFS2000 0.619 0.600 0.639

30

31

San Rural Urban

Old New Old New

1990 22% 26% 33% 28%

2006 25% 33% 35% 32%

Water Rural Urban

Old New Old New

1990 34% 31% 80% 80%

2006 30% 40% 65% 74%

32

Sub-Saharan Africa water

1990 coverage (%)

2006 Coverage (%)

Projected coverage (%)

Target for 2015 (%)

Proximity to target (%)

On/off-track

JMP 2008 estimate

49 58 63 75 12 Off-target

New estimate (unofficial)

48 60 67 74 7 Not off-target

33

What is the household’s main source of DRINKING water?11 Piped into dwelling12 Piped into yard or plot 13 Public tap/standpipe21 Tubewell/borehole 31 Protected well 41 Protected spring 51 Rainwater harvesting 32 Unprotected well 42 Unprotected spring 52 Rainwater collection 61 Tanker-truck 71 Cart with small tank/drum 81 Surface water (river, stream, dam, lake, pond, canal, irrigation channel) 91 Bottled water 92 Sachet 96 Other (specify)

34

The fundamental lesson can be boiled down to two questionsFor the National Statistical Office:

• Is there a National Core Survey Questionnaire available on-line?

•Is there a National Survey Catalogue (Archive) available on-line?

If the answer to the any of the above is no, then please ask why?

Open Open Questions Questions

35

Is there interest at the country level for this activity?

What countries and what sectors ?Is National Data Archive of interest?Is there an interest in developing a core

National questionnaire?Developing training modules/training

materialFunding Mechanisms for

implementation at the country and supporting development of training material/technical support.

Open Open Questions Questions