Multiple Indicator Cluster Surveys Data Interpretation, Further Analysis and

Dissemination Workshop

Tables on Sample and Survey Characteristics, Data Quality and

Sampling Error

2

Sample and Survey Characteristics

Response rates and background characteristics:

• Set of 8 tables that:

• Presents sample coverage and characteristics of households and respondents

• Age and sex distribution of survey population

• Characteristics of respondents

• Household characteristics and wealth quintiles

3

Table HH.1: Results of household, women's, men's and under-5 interviewsNumber of households, women, men, and children under 5 by results of the household, women's, men's and under-5's interviews, and household, women's, men's and under-5's response rates, Country, Year

  Residence   Region  

  Urban Rural   Region 1 Region 2 Region 3 Region 4 Region 5 TotalHouseholds                  

Sampled  

Occupied  

Interviewed  

Household response rate  

Women  

Eligible  

Interviewed  

Women's response rate  

Women's overall response rate  

Men  

Eligible  

Interviewed  

Men's response rate  

Men's overall response rate  

Children under 5  

Eligible  

Mothers/caretakers interviewed  

Under-5's response rate  

Under-5's overall response rate                  

Overall response rates are calculated for women, men and under-5's by multiplying the household response rate by the women's, men's and under-5's response rates, respectively.

4

Table HH.2: Household age distribution by sexPercent and frequency distribution of the household population by five-year age groups, dependency age groups, and by child (age 0-17 years) and adult populations (age 18 or more), by sex, Country, Year

 Males   Females   Total

Number Percent   Number Percent   Number PercentTotal   100.0     100.0     100.0

0-4  5-9  10-14  15-19  20-24  25-29  30-34  35-39  40-44  45-49  50-54  55-59  60-64  

65-69  70-74  75-79  80-84  85+  Missing/DK  

Dependency age groups  

0-14  15-64  65+  Missing/DK  

Child and adult populations  Children age 0-17 years  

Adults age 18+ years  Missing/DK  

   

Missing information on sex is normally not expected; in the event that few household members have missing sex in the final data set, this should be indicated in the final report in a footnote to the table, and such cases should be excluded from the table.

5

Table HH.3: Household compositionPercent and frequency distribution of households by selected characteristics, Country, Year 

Weighted percentNumber of households

  Weighted Unweighted

Total weighted and unweighted numbers of households should be equal when normalized sample weights are used.

Tables HH.3, HH.4, HH.4M and HH.5 present main background characteristics of the household, women's, men's and under-5 samples, and should be produced and finalized before the rest of tables are produced, to ensure that the categories adopted for presentation in the tables will include sufficiently sized denominators.

Religion/Language/Ethnicity of household head should be constructed from information collected in the Household Questionnaire, in questions HC1A, HC1B, and HC1C. In most surveys, some combination of these three questions will be used as the final variable that best describes the main socio-cultural or ethnic groups in the country.

Table HH.4 /HH.4M/HH5: Women's/Men's/Under-5's background characteristicsPercent and frequency distribution of women / men / children under 5 by selected background characteristics, Country, Year 

Weighted percentNumber

  Weighted Unweighted

Table HH.6: Housing characteristicsPercent distribution of households by selected housing characteristics, according to area of residence and regions, Country, Year

  TotalArea   Region

Urban Rural   Region 1 Region 2 Region 3 Region 4 Region 5   

Electricity  Yes  No  Missing/DK  

Flooring  Natural floor  Rudimentary floor  Finished floor  Other  Missing/DK  

Roof  Natural roofing  Rudimentary roofing  Finished roofing  Other  Missing/DK  

Exterior walls  Natural walls  Rudimentary walls  Finished walls  Other  Missing/DK  

Rooms used for sleeping  1  2  3 or more  Missing/DK  

   Total 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0   Number of households     Mean number of persons per room used for sleeping

                 

Information on housing characteristics are obtained in the Household Characteristics module of the Household Questionnaire: Electricity (HC8A), flooring (HC3), roof (HC4), exterior walls (HC5) and rooms used for sleeping (HC2).

To limit the size of the table, detailed floor, roof, and exterior wall categories are not shown. If needed, these categories may be indicated in a footnote below the table, in the final report.

Additional relevant housing characteristics may be added to the table if included in the household questionnaire.

Most of the information collected on these housing characteristics are used in the construction of the wealth index.

Table HH.7: Household and personal assetsPercentage of households by ownership of selected household and personal assets, and percent distribution by ownership of dwelling, according to area of residence and regions, Country, Year

Information on household and personal assets are obtained in the Household Characteristics module of the Household Questionnaire: Radio (HC8B), television (HC8C), Non-mobile telephone (HC8D), refrigerator (HC8E), agricultural land (HC11), farm animals/livestock (HC13), watch (HC9A), mobile telephone (HC9B).bicycle (HC9C), motorcycle or scooter (HC9D), animal-drawn cart (HC9E), car or truck (HC9F), and boat with a motor (HC9G). Ownership of dwelling is based on responses to HC10.

Additional household and personal assets should to be added to the questionnaires (for wealth index construction) and shown in this table.

Missing/DK values are included in the denominators and households with missing information are considered not to own or have these assets. However, a careful examination of the extent of missing values needs to be undertaken prior to the construction of this table. If Missing/DK cases exceed 5 percent, this should be shown in the table.

Most of the information collected on household and personal assets are used in the construction of the wealth index.

Table HH.8: Wealth quintilesPercent distribution of the household population by wealth index quintiles, according to area of residence and regions, Country, Year

 Wealth index quintiles

TotalNumber of household

membersPoorest Second Middle Fourth Richest   

Total 100.0     

Area  Urban 100.0  Rural 100.0  

Region  Region 1 100.0  Region 2 100.0  Region 3 100.0  Region 4 100.0  Region 5           100.0  

Wealth index quintiles are constructed by using data on housing characteristics, household and personal assets, and on water and sanitation via principal components analysis.

Household members should be equally distributed to the five wealth index quintiles for the total sample, in the first row of the table (percentages that deviate from the equal distribution of 20 percent per quintile by 0.1 - 0.2 percent are permissible).

Other background characteristics (such as Religion/Language/Ethnicity, education and sex of household head) may be added to the table, if needed.

9

Data Quality Tables

Before producing tabulations and writing the report narrative, 28 tables are produced for assessment of data quality

Intended to check distributions, heaping, understatement or overstatement, sex ratios, eligibility and coverage, out-transference of eligible persons, the extent of missing information, outliers, sex ratios, quality of anthropometric measurements

Useful for understanding quality issues, familiarity with issues in data sets, indicative of the quality of training and implementation

10

DQ.1: Age distribution of household populationSingle-year age distribution of household population by sex, Country, Year

Males Females

Males Females

Number Percent Number Percent Number Percent Number Percent Age Age

0 45 1 46 2 47 3 48 4 49 5 50 6 51 7 52 8 53 9 54 10 55 11 56 12 57 13 58 If age reporting is good, the distribution should be smooth.

The table should also provide insights into overreporting or underreporting at certain age groups or intervals, and the extent of missing information on age.

Deficits at ages 4, 15, and 49, excesses at ages 5 and 6, 14, and 50 might be indicative of out-transference of ages to avoid administering individual questionnaires.

11

Distribution of household members by single age

12

Table DQ.2: Age distribution of eligible and interviewed women

DQ.2: Age distribution of eligible and interviewed womenHousehold population of women age 10-54 years, interviewed women age 15-49 years, and percentage of eligible women who were interviewed, by five-year age groups, Country, Year

 

Household population of women age 10-54 years Interviewed women age 15-49 years

Percentage of eligible women

interviewed (Completion rate)Number   Number Percent  

   Age  

10-14 na na na15-19  20-24  25-29  30-34  35-39  40-44  45-49  50-54 na na na

   Total (15-49) 100.0     Ratio of 50-54 to 45-49     na na   nana: not applicable

The purpose of these tables is to detect both displacement of respondents out of the eligible age range and differential response rates by age.

13

Completion rates - women, men & under-5s (DQ2, DQ3, DQ4)

Fieldwork performance – re-visits, good planning

Completion rates need to be high, but also uniform by age and background characteristics

Low completion rates for certain age groups are likely to bias results

14

Birth date and age reporting (DQ5, DQ6, DQ7, DQ8, DQ9, DQ10)

Surveys always have cases with missing information

The extent of missing information is important, because it can result in biased results if such proportions are high

Particularly informative about the quality of survey is the extent of missing information on measurements, ages, and dates of events

15

DQ.5: Birth date reporting: Household population

DQ.5: Birth date reporting: Household populationPercent distribution of household population by completeness of date of birth information, Country, Year

 

Completeness of reporting of month and year of birth

Total

Number of household members

Year and month of birth Year of birth only Month of birth only Both missing

Total 100.0  

   

Age  

0-4 100.0  

5-14 100.0  

15-24 100.0  

25-49 100.0  

50-64 100.0  

65-84 100.0  

85+ 100.0  

DK/Missing na na 100.0  

Region  

Region 1 100.0  

Region 2 100.0  

Region 3 100.0  

Region 4 100.0  

Region 5 100.0  

Area  

Urban 100.0  

Rural         100.0  

na: not applicable

16

Completeness of reporting (DQ11)DQ.11: Completeness of reportingPercentage of observations that are missing information for selected questions and indicators, Country, Year

Questionnaire and type of missing information Reference group

Percent with missing/incomplete

informationa

Number of cases

   Household  

Salt test result All households interviewed that have salt  Starting time of interview All households interviewed  Ending time of interview All households interviewed  

   Women  

Date of first marriage/union All ever married women age 15-49  Only month  Both month and year  

Age at first marriage/unionAll ever married women age 15-49 with year of first marriage not known

 

Age at first intercourse All women age 15-24 who have ever had sex  Time since last intercourse All women age 15-24 who have ever had sex  Starting time of interview All women interviewed  Ending time of interview All women interviewed  

   Men  

Date of first marriage/union All ever married men age 15-49  Only month  Both month and year  

Age at first marriage/union All ever married men age 15-49 with year of first marriage not known  

Age at first intercourse All men age 15-24 who have ever had sex  Time since last intercourse All men age 15-24 who have ever had sex  Starting time of interview All men interviewed  Ending time of interview All men interviewed  

   Under-5  

Starting time of interview All under-5 children  Ending time of interview All under-5 children  

a Includes "Don't know" responses

The purpose is to examine the amount of missing information for certain key indicators.

High levels of missing data may indicate that the non-missing data are biased or of poor quality.

17

Completeness of anthropometric data (DQ12, DQ13, DQ14)

Many tools have been developed for assessing data quality of anthropometric indicators

Completeness of anthropometric data influenced by Birth date reporting Children not weighed, measured Bad quality measurements

Expected completeness should be above 90 percent, preferably 95

18

Completeness of anthropometric data (DQ12) - Underweight

DQ.12: Completeness of information for anthropometric indicators: UnderweightPercent distribution of children under 5 by completeness of information on date of birth and weight, Country, Year

 

Valid weight and

date of birth

Reason for exclusion from analysis

Total

Percent of children excluded

from analysis

Number of children under 5

Weight not

measured

Incomplete date of birth

Weight not measured and

incomplete date of birth

Flagged cases

(outliers)   Total 100.0     Age  

<6 months 100.0  6-11 months 100.0  12-23 months 100.0  24-35 months 100.0  36-47 months 100.0  48-59 months           100.0    

19

Heaping in anthropometric data (DQ15)

Under normal circumstances, approximately 10 percent of anthropometric measurements should be reported for each of the digits for the decimals.

Significant excesses over 10 percent are indicative of heaping, and therefore quality problems in anthropometric measurements, either due to truncation or rounding.

Typically, more heaping is expected in height/length than weight measurements.

20

Heaping in anthropometric data (DQ15)

DQ.15: Heaping in anthropometric measurementsDistribution of weight and height/length measurements by digits reported for the decimal points, Country, Year

 Weight Height or length

Number Percent   Number Percent           Total 100.0 100.0   Digits  

0  1  2  3  4  5  6  7  8  9  

   0 or 5          

The table includes all children with weight and height/length measurements, regardless of the completeness of date of birth information, and flagged cases, which may not be included in the anthropometric analysis.

21

Observation of documents (DQ16-DQ18) and observation of bednets and places for handwashing

(DQ19)

Interviewers are required to ask and see the specific documents and copy relevant information on the questionnaire

This is important for the completion of the several modules in women and under-5 questionnaire, and may also be useful for obtaining accurate information on children's dates of birth and ages

22

DQ.17: Observation of vaccination cardsPercent distribution of children age 0-35 months by presence of a vaccination card, and the percentage of vaccination cards seen by the interviewers, Country, Year

 

Child does not have vaccination card Child has vaccination card

DK/Missing Total

Percentage of vaccination cards

seen by the interviewer

(1)/(1+2)*100

Number of children age 0-35 months

Had vaccination card previously

Never had vaccination

card  Seen by the interviewer

(1)

Not seen by the

interviewer (2)

                   

Total 100.0  

   

Region  

Region 1 100.0  

Region 2 100.0  

Region 3 100.0  

Region 4 100.0  

Region 5 100.0  

Area  

Urban 100.0  

Rural 100.0  

Child's age  

0-5 months 100.0  

6-11 months 100.0  

12-23 months 100.0  

24-35 months             100.0    

23

DQ20: Respondent to under-5 questionnaire

Presence of mother in the household and the person interviewed for the under-5 questionnaire:

The under-5 questionnaire should be administered to the mother, if the mother is listed the household roster

DQ.20: Presence of mother in the household and the person interviewed for the under-5 questionnaireDistribution of children under five by whether the mother lives in the same household, and the person who was interviewed for the under-5 questionnaire, Country, Year

 

Mother in the household Mother not in the household

Total

Number of children under 5

Mother interviewed

Father interviewed

Other adult

female interview

ed

Other adult male interviewe

d  Father

interviewed

Other adult female

interviewed

Other adult male

interviewed

24

DQ21: Correct selection for child labour and child discipline modules

Selection of children age 1-17 years for the child labour and child discipline modules

In households where 2 or more children age 1-17 years live, interviewers are required to select, according to pre-determined random selection procedures, one child for the child discipline module

Percentages with correct selection should be close to 100.0

DQ.21: Selection of children age 1-17 years for the child labour and child discipline modulesPercent distribution of households by the number of children age 1-17 years, and the percentage of households with at least two children age 1-17 years where where correct selection of one child for the child labour and child discipline modules was performed, Country, Year

 

Number of children age 1-17 years

TotalNumber of households

Percentage of households where

correct selection was performed

Number of households with 2 or more children age 1-17 yearsNone One

Two or more

               

Total 100.0  

   

Region  

Region 1 100.0  

Region 2 100.0  

Region 3 100.0  

Region 4 100.0  

Region 5 100.0  

Area  

Urban 100.0  

Rural 100.0  

Wealth index quintiles  

Poorest 100.0  

Second 100.0  

Middle 100.0  

Fourth 100.0  

Richest       100.0      

26

DQ.22: School attendance by single age 

Not attending

school

Currently attending

DK/Missing Total

Number of

household

membersPreschool

Primary schoolGrade

Secondary schoolGrade

Higher than

secondary1 2 3 4 5 6   1 2 3 4 5 6

   Age at beginning of school year

 

5 100.0  6 100.0  7 100.0  8 100.0  9 100.0  10 100.0  11 100.0  12 100.0  

Age at the beginning of the school year is calculated from dates of birth of household members or by rejuvenating household members based on the date of the survey and current age. Levels and grades refer to the current school year, or the most recent school year if data collection was completed between school years.

Many cases outside the diagonal would be indicative of both poor fieldwork supervision, as well as poor data entry and (lack of) verification.

27

Child mortality related (DQ23-DQ26)

DQ.23: Sex ratio at birth among children ever born and living

DQ.24: Births by calendar years

DQ.25: Reporting of age at death in days

DQ.26: Reporting of age at death in months

28

DQ.23: Sex ratio at birth among children ever born and living

Children Ever Born Children Living Children DeceasedNumber

of womenSons Daugthers

Sex ratio at

birth Sons DaugthersSex

ratio Sons DaugthersSex

ratio Total Age

15-19 20-24 25-29 30-34 35-39 40-44 45-49

29

DQ.24: Births by calendar yearsNumber of births, percentage with complete birth date, sex ratio at birth, and calendar year ratio by calendar year, according to living, deceased, and total children (weighted, unimputed), as reported in the birth histories, Country, Year

 Number of births Percent with complete birth dateb Sex ratio at birthc Calendar year ratiod

Living Deceased Total   Living Deceased Total   Living Deceased Total   Living Deceased Total   

Total na na na   Year of birth  

2013a na na na2012 na na na2011  2010  2009  1999-2003 na na na

1994-1998 na na na

<1994 na na naDK/missing na na na

na: not applicable a Interviews were conducted from [Month] to [Month], 2013b Both month and year of birth givenc (Bm/Bf) x 100, where Bm and Bf are the numbers of male and female births, respectivelyd (2 x Bt/(Bt-1 + Bt+1)) x 100, where Bt is the number of births in calendar year t

The purpose is to examine the impact of omission of births in the five years preceding the survey.

If large amounts of omission are suspected, then careful interpretation of current fertility and mortality levels and trends is needed.

Graphic presentation of these data can provide good visual appreciation of omission and transference.

DQ.25: Reporting of age at death in daysDistribution of reported deaths under one month of age by age at death in days and the percentage of neonatal deaths reported to occur at ages 0–6 days, by 5-year periods preceding the survey (weighted, imputed), Country, Year

 Number of years preceding the survey Total

(0–19)0–4 5–9 10–14 15–19Age at death (days)

DQ.26: Reporting of age at death in monthsDistribution of reported deaths under two years of age by age at death in months and the percentage of infant deaths reported to occur at age under one month, for the 5-year periods of birth preceding the survey (weighted, imputed), Country, Year

Number of years preceding the survey Total(0-19)0–4 5–9 10–14 15–19

Age at death (months)

The purposes of tables DQ25 and DQ26 are to examine the possible omission of neonatal and early neonatal deaths; and the effects of age at death heaping. 

31

Maternal mortality related (DQ27 and DQ28)

DQ.27: Completeness of information on siblings

DQ.28: Sibship size and sex ratio of siblings

32

Sampling error tables

33

Sampling Error Tables: Background

The sample selected in a survey is one of the many samples that could have been selected (with same design and size)

Sampling errors are measures of the variability between all possible samples, which can be estimated from survey results

34

Sampling Error Tables: Background

Calculation of sampling errors is very important

Provides information on the reliability of your results

Tells you the ranges within which your estimates most probably fall

Provides clues as to the sample sizes (and designs) to be selected in forthcoming surveys

35

Sampling Error Tables: Background

MICS sample designs are complex designs, usually based on stratified, multi-stage, cluster samples

It is not possible to use straightforward formula for the calculation of sampling errors. Sophisticated approaches have to be used

36

Sampling Error Tables: Background

Versions 13 and above of SPSS are used for this purpose

SPSS uses Taylor linearization method of variance estimation for survey estimates that are means or proportions

This approach is used by most other package programs: Wesvar, Sudaan, Systat, EpiInfo, SAS

37

Sampling Error Tables: Background

In MICS, the objective is to calculate sampling errors for a selection of variables, for the national sample, as well as for each of the reported domains

Sampling error tabulation plan includes separate excel worksheets for: total sample, urban, rural, and 6 regions. SE tables can be produced for other domains such as ethnicity and wealth quintiles

38

 

MICS Indicator

MDG Indicator

Value (r)

Standard error (se)

Coefficient of variation

(se/r)Design

effect (deff)

Square root of design

effect (deft)Weighted

countUnweighte

d count

Confidence limits

Lower boundr - 2se

Upper boundr + 2se

Household members                      

Use of improved drinking water sources 4.1 7.8 0.000 0.000

Use of improved sanitation 4.3 7.9 0.000 0.000

Primary school net attendance ratio (adjusted) 7.4 2.1 0.000 0.000

Women  

Infant mortality rate 1.2 4.2 0.000 0.000

Under five mortality rate 1.5 4.1 0.000 0.000

Adolescent birth rate 5.1 5.4 0.000 0.000

Contraceptive prevalence rate 5.3 5.3 0.000 0.000

Unmet need 5.4 5.6 0.000 0.000

Antenatal care coverage (1+ times, skilled provider)

5.5a 5.5 0.000 0.000

Antenatal care coverage (4+ times, any provider) 5.5b 5.5 0.000 0.000

Skilled attendant at delivery 5.7 5.2 0.000 0.000

Maternal mortality ratio 5.13 5.1 0.000 0.000

Literacy rate (young women) 7.1 2.3 0.000 0.000

Knowledge about HIV prevention (young women)

9.1 6.3 0.000 0.000

Condom use with non-regular partners 9.15 6.2 0.000 0.000

Men  

Literacy rate (young men) 7.1 2.3 0.000 0.000

Knowledge about HIV prevention (young men) 9.1 6.3 0.000 0.000

Condom use with non-regular partners 9.15 6.2 0.000 0.000

Under-5s  

Underweight prevalence (moderate and severe) 2.1a 1.8 0.000 0.000

Underweight prevalence (severe) 2.1b 1.8 0.000 0.000

Children under age 5 who slept under an ITN 3.18 6.7 0.000 0.000

Anti-malarial treatment of children under age 5 3.22 6.8               0.000 0.000

Note that mortality SEs can only be calculated for results based on birth history with the existing and separate SPSS syntax.

Also note that SEs for the maternal mortality ratio can be calculated only through the CS Pro application.

The indicators listed in SE tab plan represent the MDG indicators for which SEs can be calculated. SEs can easily be produced for most other MICS indicators and included if desired.

39

Comprehensive knowledge about HIVprevention among young people

40

Thank You