Post on 20-Sep-2020
Estimating Comparable Poverty Counts from Incomparable
Surveys: Measuring Poverty in India
Alessandro TarozziPrinceton University∗
First draft September 2001 - This version, May 2002
Abstract
We develop a procedure to estimate poverty counts in India from the 55th Round of the Na-tional Sample Survey (NSS), a large household survey run in 1999-2000. The evidence suggeststhat a change in the survey design caused the reports on household expenditure to change toan extent that it is impossible, without adjustments, to compare poverty estimates from thissurvey with those obtained from previous NSS Rounds. More generally, the paper addresses theproblem of comparing the distribution of a variable across differently designed surveys, when thedifferent design causes the respondents� reports about the variable to be incomparable acrossthe surveys. The proposed procedure requires only the existence of a set of auxiliary variableswhose reports are not affected by the different survey design, and whose relation with the mainvariable of interest is stable across the surveys. The estimator, instead, does not require spe-ciÞc functional form assumptions on the relation between the main variable of interest and theauxiliary variable. In the context of NSS data, we identify a set of variables whose reports arenot systematically affected by the changes implemented in the survey design, and we provideevidence of the stability over time of the distribution of per capita total expenditure conditionalon these variables. We describe an experiment to evaluate the performance of the estimator,showing that it provides satisfactory results, both in the estimation of poverty counts and in theestimation of the density of per capita expenditure. Finally, we use our estimator to calculateadjusted estimates for poverty in India using data from the 1999-2000 NSS Survey. The resultsshow a sharp reduction in poverty in the nineties, even if in rural areas the reduction is not aslarge as that implied by the unadjusted Þgures.
∗I am grateful to Angus Deaton and Bo Honoré for comments, support, and conversations that led to this project.
I would also like to thank Elie Tamer for his help, and for comments on an earlier draft, and Dwayne Benjamin,
Debopam Bhattacharya, Phil Cross, Peter Lanjouw, Aprajit Mahajan, Robert McMillan, Jonathan Morduch, and
Barbara Rossi for comments and useful conversations, as well as seminar participants at Duke, Georgetown, Princeton,
Toronto, and the World Bank. All errors are mine.
1 Introduction
The assessment of poverty in India has been based for decades on simple poverty counts (or headcount
poverty ratios), which estimate the proportion of individuals living in households whose monthly per capita
total expenditure is below a given threshold, known as poverty line.1 The Government of India periodically
publishes �official� lines, representing the estimated per capita monthly expenditure associated, on average,
with the consumption of a given minimum caloric intake.2 Data on expenditure are typically obtained from
the Indian National Sample Survey (NSS hereafter), a household survey which, almost every year, collects
detailed information on the consumption of an exhaustive list of items. Even if consumption is not the main
object of most NSS Rounds,3 approximately every Þve years a larger (quinquennial) survey is run, and the
data thereby recorded are used to study household consumption and to estimate poverty counts in every
Indian state.
Poverty estimates represent an extremely relevant issue in political and economic discussions in India,
and they became even more frequently debated after a process of economic liberalization started in the
early nineties.4 Those opposing the process used the increasing gap between the growth in GDP showed by
National Accounts Statistics (NAS) and the stagnant levels of per capita expenditure revealed by NSS, to
argue that the liberalization process was not bringing beneÞts to poor people in India, especially in rural
areas.5 Several pro-liberalization observers also started to question the ability of the NSS surveys to pick
up correctly consumption data. In particular, the reference (or recall) period adopted in the surveys started
to be criticized. Historically, the respondent was asked to recall household consumption during the 30 days
before the interview for every item listed in the questionnaire. Such choice of recall period is not typical
in expenditure surveys, where it is more common to observe shorter recall periods for items purchased
frequently (like food), and longer periods for durables.6 The choice of the reference period was inßuenced by
an experimental study completed in West Bengal in the Þfties (Mahalanobis and Sen, 1954), which showed
that a shorter recall period led to overestimate consumption of food. Even if the subsequent literature
has not yet reached a consensus as to which recall period gives, on average, the most correct picture of1The use of headcount ratios to evaluate poverty is often criticized, since they are not sensitive to changes in the distribution
of income below the poverty line. For a review of the literature on poverty measures and their properties see Deaton (1997, Ch.
3), or Ravallion (1993).2For a historical and methodological overview of the construction of the lines see Government of India, Planning Commission
(1993). Deaton and Tarozzi (2000) investigate the appropriateness of the price indexes routinely used to inßate the poverty
lines, and propose alternatives. For another alternative set of poverty estimates see also Datt (1999).3Examples of other issues analyzed by NSS surveys are schooling, health, employment, small enterprises, and the Indian
Public Distribution System.4The liberalization process started after a Balance of Payment crisis in the summer of 1991. For an overview of the economic
reforms and its effects see Sachs, Varshney, and Bajpai (1999), and references therein.5 Sen (2000) shows that the increasing gap between NSS and NAS estimates during the nineties is actually a statistical artifact,
due to differences in the deßators used in the two different databases, and to changes introduced in the NAS methodology.6For a recent survey of the literature on the choice of the recall period in consumption surveys, see Deaton and Grosh (2000)
and references therein.
2
household consumption, there is ample evidence that longer recall periods typically imply lower reports for
items purchased frequently. Since high-frequency purchases like food and beverages in India account for
a very large share of the budget, it would have been possible to explain at least part of the gap between
NAS and NSS results arguing that the 30-day recall period had ceased, for some reason, to be adequate.7
For this reason, after the 50th Round of the NSS (July 1993-June 1994), which was at that time the latest
large quinquennial survey, it was decided to start experimenting with different recall periods, using the four
relatively smaller surveys that followed.8
From the 51st to the 54th NSS Round, households were randomly assigned to two different subsamples.
Respondents in one group were assigned the standard questionnaire (called schedule 1, S1 hereafter), with
a 30-day recall period for all items. Respondents in the second group were instead given schedule 2 (S2), in
which the recall period was shortened to 7 days for food, beverages and some other items generally bought
frequently, while for durables, clothing, footwear and some other low-frequency purchases the reference
period was brought to 365 days (in what follows I will use the notation S1 and S2 broadly, indicating the two
questionnaires, or the two different populations, or samples, or surveys they refer to). For what will follow
in the paper, it is important to stress that the 30-day recall period was instead kept in both schedules for a
list of items accounting for a substantial share of the budget. This list includes fuel and light, miscellaneous
goods and services, rents and consumer taxes, and non-institutional medical expenses.9 We will generally
refer to this list of items as �30-day items�.
As a result, the estimated average per capita total monthly expenditure (PCE hereafter) turned out to be
strikingly different across the two different schedules. These results are shown in the second row of Table 1,
from which it can be seen that PCE as reported in S2 is systematically 15-20% higher than the corresponding
Þgure for S1. This is mostly due to the fact that S2 typically produced much higher estimates of average
monthly consumption of food than S1.10 The last row of Table 1 shows that if one uses the same poverty
line for both subsamples, the proportion of people in poverty is cut approximately by a half if S2 is used.
Probably to achieve a better understanding of the reasons underlying these differences, the National
Sample Survey Organization decided to give both schedules to all households in the next large quinquennial
survey (the 55th Round), carried out from July 1999 to June 2000. The preliminary aggregate results7Experimental evidence suggests that for items purchased infrequently, the recall bias might be due less to forgetting than
to the opposite process known as telescoping, by which the respondent includes expenditures incurred before the reference
period (see Neter and Waksburg (1964). To explain these Þndings, Rubin and Baddeley (1989), and Bradburn, Huttenlocher,
and Hedges (1994), propose models of the cognitive mechanisms underlying the recall process). Note that, in India, durables
account on average for a small fraction of total budget, so the effects of changes in the recall period are typically dominated by
the induced changes in reported consumption of food.8 It is also widely believed that the NSS systematically undersamples richer households. Since the late eighties the sampling
scheme has been modiÞed in an attempt to resolve the issue (see G.o.I., 1988, section 1.4).9The distinction between institutional and non-institutional medical expenses lies in whether the expenses were incurred for
medical treatment as an in-patient at a medical institution or otherwise.10Deaton (2001) and Sen (2000) analyze in detail how the differences between the two schedules affect the measurement of
consumption, poverty, and inequality. See also Visaria (2000).
3
estimated by the Indian Planning Commission showed an impressive reduction in poverty with respect to
the early nineties: using the data collected with a 30-day reports for food, in the rural sector the counts
dropped from 37.2 in 1993-94 to 27.1 six years later, while in urban areas the proportion dropped from 32.6
percent to 23.6. In both cases this amounts to a reduction of one third in poverty rates, in less than a decade.
Such results, though, puzzled many observers, since they also showed that the change in the survey
design reduced the discrepancy in average PCE between the two schedules to an implausible extent: the
gap between the two estimates dropped to less than 5 percent, both in the rural and the urban sector
(Government of India, 2001). The most plausible explanation of the reduction of the gap is that asking to
report consumption with both recall periods prompted the respondents to reconcile the two consumption
levels they were reporting. This interpretation is made more plausible by the fact that the two different
reports had to be recorded on two parallel columns printed next to each other on the questionnaire. But
then we cannot use any of the two sets of reports to build poverty counts comparable with those obtained
from previous Rounds. In fact, consumption of food reported with the traditional 30-day recall period is
likely to be disproportionately high (since the respondent will tend to avoid large discrepancies with the 7-day
reports, which are typically higher), and the corresponding reports based on a 7-day recall period are likely
to be disproportionately low (by a symmetric argument). The comparability of these latest poverty counts
is further hindered by the fact that consumption data on durables, clothing and footwear, and educational
and institutional medical expenditure have been collected exclusively with a 365-day recall period.11
The main objective of this paper is to propose a method to estimate PCE-based poverty counts from the
55th NSS Round without using data on PCE from such survey. The procedure is based on the assumption
that the 55th Round and previous NSS Rounds characterized by the standard survey design share the same
distribution of PCE conditional on a set of other variables for which the changes in the questionnaire did
not cause systematic changes in the reports. If this assumption holds, it will be possible to use observations
on these other variables from the 55th Round, together with information on the structure of the conditional
distribution from previous Rounds, to recover the marginal distribution of PCE in the 55th Round. Formally,
the headcount ratio H55 for the population sampled in the 55th Round can be written as
H55 ≡ P (y < z | 55) =ZP (y < z | v, 55)f (v | 55) dv (1)
where z is the poverty line, y is reported PCE when the respondent is given the standard questionnaire, v is
a vector of variables whose reports have not been affected by the change in survey design, P (y < z | v, 55)denotes the conditional probability of being poor given v, and f (v | 55) is the marginal density of v inthe 55th Round. A sample equivalent of the above expression cannot be directly computed, since y is not
observed. But if the sampling process identiÞes f (v | 55), and if P (y < z | v, 55) = P (y < z | v, a) for someother auxiliary survey a in which both y and v are reported using a standard questionnaire, the poverty11For other papers dealing with the consequences of differences in survey design on poverty estimation, see Gibson (1999a,
1999b, 2001), and Lanjouw and Lanjouw (2001).
4
count becomes
H55 =
ZP (y < z | v, a)f (v | 55) dv (2)
where both terms inside the integral are now identiÞed.12 More generally, we show that one can use analogous
arguments to estimate the marginal density of y in the 55th Round without using observations on PCE
recorded in that survey.
A natural choice for a variable to be included in v is the set of �30-day items� listed above, for several
reasons. First, the recall period for these items stayed the same in all NSS surveys, including those with
experimental design, so that it is reasonable to expect that questionnaire changes did not affect consumption
reports for these items. In this respect, it is suggestive that in the smaller experimental surveys run after
the 50th Round, while the average PCE changes considerably across different schedules, mean per capita
expenditure of 30-day items does not show any such difference across the two schedule types (see Table
1). Second, expenditure on 30-day items is strongly correlated with total expenditure, so that knowing the
Þrst will allow one to make inferences about the second.13 Third, an important component of the relation
between y and v will depend upon households� preferences, so that such relation is likely to be rather stable
over time. Note, though, that movements in relative prices or other factors might undermine the validity
of this assumption, whose credibility should be scrutinized as carefully as possible. We will include other
household-speciÞc covariates in v, but our results show that per capita expenditure on 30-day items is by
far the most important variable one has to consider when implementing the adjustment we propose.
It should be stressed that the scope of our procedure goes beyond the calculation of poverty counts or
densities of PCE in India. The same tools developed here might be used in any situation where the researcher
needs to build comparable statistics using surveys which are only partly comparable, if the conditions outlined
above hold. It is also important to note that such conditions cannot be formally tested. In our context, for
example, they relate to reports as they would be observed if the questionnaire design had remained the same
as in previous Rounds, which it has not. In this sense, they are assumptions necessary for identiÞcation.
Indirect evidence of the validity of the assumptions can be obtained from the smaller NSS Rounds run
between 1994 and 1998, which also offer an interesting opportunity to evaluate how well the procedure
performs, which is of course an important element to consider if one wants to implement it using data
from the 55th NSS Round. In each of these smaller surveys a subset of households received the standard
questionnaire (S1) based on a 30-day recall period for all items, while another subset received an experimental
questionnaire (S2) with different recall periods for different groups of items. Since households were assigned
randomly to the two subsamples, the sampling process for each of these Rounds identiÞes the distribution12A different approach would be to estimate a regression of y on v from an auxiliary survey, and then use the estimated
coefficients, together with observations on v from the target survey, to extrapolate the distribution of y in the latter survey.
Note, though, that such approach would require the choice of a functional form for the relationship between y and the chosen
covariates, together with assumptions on the distribution of the error in the estimated regression (see Elbers, Lanjouw, and
Lanjouw, 2001).13 In all surveys considered here, the correlation coefficient is about 0.8 in the rural sector, and around 0.85 for urban areas.
5
of all variables for both questionnaire types. On one hand this allows one to test formally whether the
existing differences in the questionnaire affect the distribution of the variables that we want to include in
v. On the other hand, for each of these surveys, one can compare the headcount ratio estimated using our
procedure from S2 (whose data on PCE are non-comparable with those recorded in a standard survey) with
a �benchmark� represented by the poverty count estimated through standard methods using PCE from S1.
The latter estimate of the poverty count can be used as benchmark since it is computed using observations
recorded using a standard questionnaire, so that the result should be comparable with the poverty counts
estimated using the previous quinquennial surveys.
As we mentioned before, our procedure requires the use of an auxiliary survey in which both y and v are
recorded in a standard questionnaire. In our experiment this role is played by the 50th NSS Round, the last
quinquennial survey run before the 55th Round. This allows us to evaluate whether the assumptions become
less plausible, and/or whether the performance of the estimator worsens, when the survey for which we want
to estimate the poverty count (the target survey) becomes more distant in time from the auxiliary survey.
Our results are encouraging. The hypothesis that the distribution of the covariates in v is the same
across the two different schedules is almost never rejected. Also, the evidence suggests that the distribution
of y given v is rather stable over time. While using data on PCE recorded with the experimental set of
recall periods would produce poverty counts generally Þfty percent lower than the benchmark, our estimator
produces results that differ from the latter by about 10 percent in the rural sector, and even less in the urban
sector. Even if this does not imply that our estimator will give sensible results once applied to data from
the 55th Round, it does suggest that it will do so.
When we adjust poverty counts in the 55th Round using our procedure, our results show that the official
Þgures overstate (as expected) the reduction in poverty in the rural sector, while the corresponding counts
for urban areas are left virtually unchanged. Even in the rural sector, though, our headcount ratios are only
about 1.5 percentage points higher than the official ones. This suggests that the effect of the questionnaire
change on the pattern of 30-day reports for food is not large. These should be welcomed as good news,
since the reduction in poverty in the nineties remains impressively large even after the adjustment that we
propose.
The paper is organized as follows: the next section provides further information about the National
Sample Survey, and its importance in the evaluation of poverty in India. In Section 3 we describe formally
the theoretical problem we face, and lay out the notation we will use throughout the rest of the paper.
Section 4 describes the estimators we use (with the asymptotic properties studied in the appendix). Section
5 describes the empirical experiment we use to evaluate the credibility of the assumptions needed for our
procedure to work, and to evaluate its performance. In Section 6 we apply our adjusted procedure to the
55th NSS Round, and in Section 7 we conclude.
6
2 The National Sample Survey and its Use in Poverty Estimation
Since the early seventies, the Planning Commission has regularly published �official� poverty estimates
separately for rural and urban areas of every Indian State and Union Territory. These estimates are based
on the large household consumption surveys run by the National Sample Survey Organization approximately
every Þve years. At the time of writing official poverty estimates are available for 1973-74 (28th NSS Round),
1977-78 (32nd), 1983-84 (38th), 1987-88 (43rd), 1993-94 (50th), and 1999-2000 (55th). The NSS collects
information on a wide spectrum of variables, but a large section of the questionnaire is devoted to record
household consumption of an exhaustive and detailed list of items.14 All individuals living in households
where the total consumption per person is below the poverty line are then counted as poor, and no adjustment
is made to take into account family composition. Note that when we compute PCE we always consider a
measure of total household expenditure that, besides purchased items, also includes commodities received
in exchange of goods and services, home-produced items, goods received as gifts, loans or charities, and free
collection.
The original poverty lines have been computed estimating calorie Engel curves in the early seventies,
using NSS data from the 28th Round. The lines represent the PCE associated, on average, with a sector-
speciÞc minimum calorie intake recommended by the Indian National Institute of Nutrition.15 These lines
are price-inßated using the Consumer Price Index for Agricultural Labourers (CPIAL) for rural areas, and
the Consumer Price Index for Industrial Workers for the urban sector.
Between quinquennial Rounds, smaller surveys are run almost every year, but their main focus is generally
different from consumption. With a very few exceptions, NSS surveys are carried out over a one year period,
and the interviews are conducted evenly across four quarters.16
All NSS surveys are characterized by a complex survey design, which has not always remained unchanged
across different Rounds, even if all surveys are stratiÞed and clustered. The whole population is Þrst divided
into separate strata, typically geographically deÞned, and then a number of primary stage units (villages in
the rural sector, and blocks in urban areas) are selected from every stratum. Once a primary stage unit (PSU
hereafter) is included in the sample, a Þxed number of households are selected from it. The NSS sampling
scheme is such that every household does not have, ex-ante, the same probability of being selected, so inßation
factors (or probability weights), are necessary to recover population estimates.17 The complex survey design
also has consequences for the standard errors of most statistics of interest: stratiÞcation, making all possible
samples more homogeneous, tends to decrease the variability of the estimates, while clustering typically
increases considerably the standard errors. The latter result is due to the fact that several households are14The item list is very detailed. There are, for example, approximately 200 different food items.15 See GOI, Planning Commission (1993).16The larger surveys are designed to give representative pictures of consumption patterns in every state, sector (rural or
urban) and sub-Round (the quarter in which the interview takes place).17 If weights are not used, households with a lower probability of selection would be underrepresented. For a short textbook
treatment of sampling weights see Deaton (1997, Ch. 1).
7
selected from the same PSU, and variables collected in the same PSU generally display positive correlation.18
Besides the changes in the consumption questionnaire that we already outlined above, the smaller surveys
we use in our empirical experiment partly differ in their sampling scheme. In particular, even if consumption
data were collected in all smaller surveys, the 51st (July 1994 - June 1995), 53rd (January - December 1997)
and 54th Round (January - June 1998) were designed to have a major focus on enterprises, so that the frame
from which PSU�s have been selected is not a census of households, but rather a census of enterprises. For
this reason, it is not obvious that these three surveys on one side, and the 50th, 52nd (July 1995 - June 1996),
and 55th Round on the other side are representative of the same population considered at different points
in time. We will show, though, that there is no evidence that these differences in frame affect the stability
of the relation between PCE and per capita expenditure on 30-day items, which is crucial for our procedure
to work. Since the 54th Round was run over a six-months period, while all the other surveys are carried out
over a one year time span, we do not use this Round, to avoid seasonality issues.
3 The Theoretical Problem
In this section, we describe the theoretical issues involved in the comparison of poverty counts across dif-
ferently designed surveys, when the differences in design cause systematic discrepancies in reports between
the two surveys. Then we show how, and under what conditions, such incomparability can be solved using
other variables, whose distribution has not been affected by changes in the survey design. More generally,
we also show how analogous procedures can be used in order to estimate comparable densities of variables
measured differently due to changes in the survey design.
Let z be the poverty line, so that all members in household j will be counted as poor if yj is below z,
where y is the variable chosen by the researcher to discriminate between poor and non-poor households in
a given population s. In our analysis, yj will be (log) PCE in household j. We are interested in estimating
the proportion of individuals below the poverty line. Letting f(y | s) be the density of y in population s,the headcount ratio Hs can then be written as follows:
Hs =
Z z
−∞f(y | s)dy (3)
If survey s contains observations on yj , one can simply estimate the poverty count using
�Hs =
Pj∈S(s)wjnj1 (yj < z)P
j∈S(s)wjnj(4)
18Clustering reduces the monetary costs of collecting data, since more observations are sampled from the same geographic
area, but it also typically reduces the amount of information contained in a given sample. Intuitively, and using an extreme
case, if all households inside the same PSU were identical, having more than one household from each PSU would not increase
the precision of the estimates. For an intuitive introduction to the effects of stratiÞcation and clustering on standard errors,
see Deaton (1997, Ch. 1), or Howes and Lanjouw (1995). Howes and Lanjow (1998) analyze the effect that taking into account
sampling design has on standard errors for poverty measures.
8
where 1 (E) is the indicator function equal to one when event E is true (and zero otherwise), wj is the
inßation factor (or weight) provided in the survey for household j,19 nj is household size, and S(s) is deÞned
as the set of indexes related to households sampled from population s. In what follows, we will use the same
index to denote both the population and the sample drawn from it in a household survey.
Suppose that we already have estimates of �H computed by (4) using a series of surveys run in different
years, but characterized by the same standard survey design. Suppose also that we want to compute a
comparable poverty count using data from a new target survey t, but that a change in the survey design is
such that a different quantity �y, and not y, is observed in survey t. So, for example, y is the reported PCE
with a 30-day recall period for all items, while �y is the reported PCE when different recall periods are used
for different item categories. Since y is not observed in the target survey t, the estimator in (4) is infeasible
for such survey. The objective is then to estimate Ht without using observations on y from survey t.
Let τ j be a binary variable equal to one when household j belongs to the target population t, and
equal to zero when the household belongs to the auxiliary population a. In what follows we will consider all
observations (yj ,vj , τ j) as being generated by a data generating process described by a joint distribution
f (yj ,vj , τ j) . To avoid excessive notation, we will use the same notation f for all marginal and conditional
distributions derived from f (yj ,vj , τ j) , and we will use differences in the arguments of these distributions
to differentiate them from one another. Similarly, we use the notation P to denote all probability masses.
So, for example, f (y | τ = 0) will be the marginal distribution of y in the auxiliary population a, andP (y < z | v, τ = 1) will be the probability of being poor once v is observed from a household belonging to
the target population. The y-based headcount ratio for the target population can be rewritten as
Ht =
ZP (y < z | v, τ = 1)f (v | τ = 1) dv (5)
In general, this quantity is not identiÞed since the change in survey design is such that the target survey t
only identiÞes P (�y < z | �v, τ = 1) and f (�v | τ = 1). Still, Ht can be estimated if the following assumptionshold:
Assumption 1 f (�v | τ = 1) = f (v | τ = 1)
Assumption 2 P (y < z | v, τ = 1) = P (y < z | v, τ = 0)
Assumption 1 states that the distribution of the reports for variables included in v is unaffected by the
changes in the questionnaire. Assumption 2 requires that the probability of being poor is the same in the
target and in the auxiliary survey once we condition on the vector of variables v. Then, using Assumptions
1 and 2 one can rewrite equation (5) as follows:
Ht =
ZP (y < z | v, τ = 0)f (v | τ = 1) dv (6)
19The inßation factor wj indicates the number of households in the population represented by the j-th household included
in the sample.
9
where all terms are now identiÞed.20
One can think of estimating (6) by numerical integration, once the two terms inside the integral have
been estimated nonparametrically. Even if the dimension of v is large, so that the curse of dimensionality
would not allow the precise estimation of the two terms, precision in the estimate of Ht would be recovered
through the integration over v. However, one can rewrite (6) in a way which is simpler to estimate, using
steps similar to those followed by DiNardo, Fortin, and Lemieux (1996). Let P (τ = 1) be the unconditional
probability that a household belongs to the target population, when the household is randomly selected from
the joint population encompassing both the target and the auxiliary population. Similarly, let P (τ = 1 | v)be the probability that a household belongs to the target population when the observed household speciÞc
covariates are equal to v. Then:
Ht =
Zf(y < z,v | τ = 0)f (v | τ = 0) f (v | τ = 1) dv (7)
=
Zf(v | y < z, τ = 0)P (y < z | τ = 0)f (v | τ = 1)
f (v | τ = 0)dv
= P (y < z | τ = 0)ZP (τ = 1 | v)P (τ = 0)P (τ = 0 | v)P (τ = 1)f (v | y < z, τ = 0) dv
= HaE [R (v) | y < z, τ = 0] (8)
where
R (v) =f (v | τ = 1)f (v | τ = 0) =
P (τ = 1 | v)P (τ = 0)P (τ = 0 | v)P (τ = 1) (9)
R (v) is the reweighting function analogous to that in DiNardo et al. (1996). Intuitively, in our context,
this function delivers the poverty count in the target survey reweighting every joint probability P (y < z,v |τ = 0) in (7), taking into account the fact that, by assumption, the two surveys share the same conditional
probability of being poor, but have potentially different distributions of the conditioning variable v. Note
that now, in (8), once a consistent estimator for the reweighting function is available, the headcount ratio
can be simply computed with a univariate nonparametric regression. However, one still has to deal with
a multivariate nonparametric regression if v is multidimensional and one is not willing to make parametric
assumptions on the functional form of the conditional probability P (τ = 1 | v), which has to be estimatedto compute the reweighting function.
Note that the above framework would also allow the estimation of the marginal density f (y | τ = 1) ,again without the use of information on y from the target survey, when Assumptions 2 and 3 are strengthened
using f (y | v, τ) instead of P (y < z | v, τ). These stronger conditions are actually extremely desirable, sincethey would allow the researcher to estimate not only poverty counts for any poverty line, but also other
poverty and inequality measures based on the marginal distribution of y.
The following proposition summarizes all the population-related results identiÞed so far:20Note that it�s important that the variables in v be not independent on y, or else (6) would mechanically return the headcount
ratio in population a.
10
Proposition 1: Suppose that Assumptions 1 and 2 hold. Then the population headcount poverty
ratio in the target survey t is
Ht = HaE [R (v) | y < z, τ = 0] . (10)
Moreover, when Assumptions 2 is replaced by
Assumption 2a f(y | v, τ = 1) = f(y | v, τ = 0)then
f (y | τ = 1) = f (y | τ = 0)E [R (v) | y, τ = 0] . (11)
In the context of this paper we will use the fact that for a set of items (fuel and light, miscellaneous goods
and services, rents, consumer taxes and non-institutional medical expenses) the recall period is equal to the
thirty days before the interview, in all questionnaire types used in NSS surveys. Then reports on household
consumption of these items are likely to be left unaffected by the different reference periods introduced for
other items, which is what we need for Assumption 1 to be satisÞed.
A necessary condition for the validity of Assumptions 2 and 2a is the stability across the auxiliary and
the target population of the Engel curve linking consumption of 30-day items to total expenditure. Suppose
that the Engel curve has the form m = φ (y) + ε, where φ (.) is monotone increasing, and ε is the error
term (independent of y) distributed according to a c.d.f. z. Then one can write the conditional distribution
function of y as:
P (y ≤ ξ | m) = 1−z (m− φ (ξ))
Such conditional distribution will be constant over time as long as φ (.) and z do not change.21 Note that,
in principle, movements in relative prices, or demand shocks, can undermine the validity of this assumption,
which should therefore be pondered carefully. Actually, the smaller experimental surveys carried out after
1993-94 offer the opportunity to provide indirect evidence about both the stability of the conditional distri-
bution, and about the irrelevance of the implemented questionnaire changes on the reported consumption
of 30-day items. In fact, in each of these surveys the sampling process identiÞes both (�y, �v) (using reports
from experimental questionnaires) and (y, v) (using reports from standard questionnaires). The issue of the
credibility of the identifying assumptions will be analyzed in depth in Section 5 below. It must be stressed,
though, that once one wants to implement our proposed adjustment procedure to estimate poverty counts
(or densities) from a target survey for which only a non-standard questionnaire has been used, it will be
no more possible to test the assumptions, which will have merely to be used as conditions necessary for
identiÞcation.
Since the Engel curve linking m to y might also depend upon household characteristics, we will also
analyze the results obtained including in the vector v other household speciÞc variables X that might help
recovering information about the distribution of y. The variables we will use are household size and a set21Note that the stability of the Engel curve does not imply that the budget share of 30-day items has to remain constant
over time.
11
of dummies for education of the household head, main economic activity of the household, land holding,
and whether the household belongs to scheduled castes or tribes. Note Þnally that, even if our adjustment
procedure relies on the underlying stability of the Engel curve, it does not require its estimation, so that no
functional form assumption about its structure will be necessary. The crucial assumption is that the relation
between y and m is stable, but we do not need to know explicitly what is the form of such relation, as the
next section will make clear.
4 Estimation
In this section, we describe the estimators for the poverty count in (10) and for the marginal density in (11).
We present the analysis of the asymptotic properties of the estimators in the appendix.
The population poverty count in (7) can be rewritten as
Ht =
Zf (y < z,v | τ = 0)R (v) dv. (12)
The estimator for the poverty count in the target survey is then
�Ht =Xj∈S(a)
ωj �R (vj) 1 (yj < z) (13)
where
ωj =wjnjP
i∈S(a)wini
So the ωj �s are the individual inßation factors normalized so that they sum up to one. The estimator in (13)
is more easily understood if one consider the simple case where v is univariate and assumes only r values.
Then (12) becomes
Ht =Xr
i=1P (y < z, v = vi | τ = 0)R (vi)
where the conditional probability can be estimated using the (weighted) proportion of individuals from the
auxiliary population living in households whose PCE is below the poverty line, and whose vj is equal to each
speciÞc value. So:
�P (y < z, vi | τ = 0) =Pj∈S(a)wjnj1(yj < z, vj = vi)P
l∈S(a)wlnl
Then, once an estimate for the reweighting function is available, the estimator for the poverty count becomes
�Ht =
Pri=1
Pj∈S(a)wjnj1 (yj < z) 1(vj = vi) �R (vi)P
l∈S(a)wlnl
which in turn, rearranging the terms, can be rewritten as (13).
Note that in (13) the reweighting function has to be estimated in a Þrst step. By deÞnition (see equation
(9) above)R (vj) depends upon P (τ j = 1 | vj) and P (τ j = 1). This latter (unconditional) probability, which
12
is the same for all households, represents the proportion of individuals belonging to the target population,
and can be estimated using the ratio between the estimated size of the target population, and the sum of
the estimated size of both the target and the auxiliary population. So
�P (τ = 1) =
Pj∈S(t)wjnjP
j∈S(t)wjnj +Pj∈S(a)wjnj
(14)
Since τ j is a binary variable, P (τ j = 1 | vj) is the regression of τ j on a vector of covariates vj. Such aconditional expectation can be estimated using a simple logit or probit model or, if the dimensionality of vj
allows it, using nonparametric techniques. In the present context, if one chooses to use a parametric binary
variable model, a logit model might be preferred to a probit model, since the former describes the probability
with a distribution with fatter tails than the latter, and this reduces the probability of having �P (τ j = 0 | vj)close to zero. In this way, since the estimated conditional probability appears in the denominator of �R (vj) ,
extreme values of such ratio become less likely.22
The marginal density in (11) can be estimated using the following kernel estimator23
�f (y | τ = 1) =Xj∈S(a)
ωj �R (vj)K
µyj − yh
¶(15)
where K (.) is the kernel, and h is the bandwidth, that shrinks to zero when the number of observations
grows to inÞnity. An alternative estimator of the poverty count can be obtained by numerical integration,
once the marginal density has been evaluated at a grid of points. Integrating (15) and using the change of
variable ψ = h−1 (yj − y) the estimator becomes
�Hdt =
Xj∈S(a)
ωj �R (vj)
"Z ∞
yj−zh
K (ψ) dψ
#(16)
where the index d indicates that the estimate uses explicitly the estimated marginal density. Note that the
lower bound of the integral approaches ∞sign (yj − z) when the number of observations goes to inÞnity,since h→ 0. But then, since the kernel K (.) is a density
limna→∞
Z ∞
yi−zh
K (ψ) dψ =
Z ∞
∞sign(yj−z)K (ψ) dψ = 1 (yj < z)
where na is the number of observations from the auxiliary survey. Then (16) approaches (13) when the
sample size grows. In our empirical application, we use both (16) and (13), obtaining estimates which differ
only marginally between the two estimators. Note though that using (16) requires the choice of a bandwidth,
which is not necessary if one uses (13).
22 In our empirical application we will show, though, that the choice of the Þrst step estimator affects only marginally the
estimates.23The estimator for the density is completely analogous to eq. 5 in DiNardo et al. (1996).
13
5 The Estimators at Work: an Experiment
Our adjustment procedure relies on identifying assumptions that, as such, cannot be directly tested. In our
context, however, the experimental surveys carried out after the 50th Round can help to indirectly provide
support for those assumptions. In these four rounds, the sample was randomly split into two subsamples.
Households in the Þrst subsample were assigned the standard questionnaire, with a 30-day recall period
for all items consumed, while the module assigned to households in the second subsample used different
recall periods for different groups of items. Even in the latter questionnaire type, though, the 30-day recall
period was kept for fuel and light, miscellaneous consumer goods and services, rents, consumer taxes and
non-institutional medical expenses. So, in every smaller Round, the sampling process identiÞes both the
distribution of (y,v), for households in sample S1, and the distribution of (�y, �v), for households belonging
to sample S2. This means that we can test Assumption 1, comparing the distribution of v to that of �v in
each smaller survey. Also, it will be possible to examine the validity of Assumptions 2 and 2a, comparing
the conditional distribution of y given v in the chosen auxiliary survey to the one estimated using S1 from
each smaller survey. Finally, since we are interested in estimating poverty counts comparable to those
one can obtain with the standard questionnaire, the problem of estimating poverty ratios from the 55th
Round is analogous to the problem of estimating poverty ratios using only sample S2 in any of the smaller
surveys.24 But since for all these surveys we also have S1, which identiÞes (y,v) , it will be possible to
compare the results obtained through our procedure, using S2 and an auxiliary survey, to the benchmarks
straightforwardly derived using y from S1 in the same survey.
In what follows, we use the 50th Round as the auxiliary survey a for all experiments. In turn, we use
the next three Rounds as target surveys t.We do not use the 54th Round since this was a six-month survey,
while all other databases were collected during a one-year period. We deliberately keep the same auxiliary
survey while considering target surveys separated from a by progressively longer time spans, since we are
interested in verifying whether increasing the time span worsens the performance of our estimator and/or
weakens the credibility of the assumptions needed for its use. One might worry, for example, that changes
in preferences, or in the relative price of m (the logarithm of per capita expenditure on 30-day items), might
produce changes in the conditional distribution of y given m.We do not Þnd evidence that this is a problem
with the surveys we use.
In order to express all values involved into the same monetary units, we deßate values from the smaller
Rounds using state-sector speciÞc official price indexes. For the rural sector, we use the Consumer Price
Index for Agricultural Laborer (CPIAL), while for urban areas we use the CPI for Industrial Workers.25 The
poverty lines we use when computing poverty counts are the all-India official ones for 1993-94 (205.7 Rupees24Note that the problem is analogous, but not identical, since the questionnaire in the 55th Round was not identical to S2 in
the smaller surveys.25These indexes are available through a number of publications issued by the Government of India (for example the Statistical
Abstract).
14
per head per month for the rural sector, and 283.4 Rupees for urban areas). In our analysis, we include
only households living in states for which both the price indexes and the official poverty lines are available:
these states, which account for about 95 percent of the total Indian population, are Andhra Pradesh, Assam,
Bihar, Gujarat, Karnataka, Kerala, Madhya Pradesh, Maharashtra, Orissa, Punjab, Haryana (urban only),
Rajasthan, Tamil Nadu, Uttar Pradesh, West Bengal, and urban Delhi.
In what follows, we Þrst test the validity of Assumptions 1, 2, and 2a, and then we apply the reweighting
procedure to the experimental problem described above.
5.1 Testing the Assumptions
In the present context, Assumption 1 requires that a change in the recall period for some consumption items
does not affect the pattern of reports for the variables we include in v. The crucial variable to be included
in v is per capita expenditure on 30-day items, which are purchased by almost all households and which
constitute an important part of the total budget. Table 1 shows that in the smaller Rounds all households
purchased 30-day items, which account for about 20 percent of the budget in the rural sector, and 25 percent
in the urban sector (using S1). Table 1 also shows that the systematic difference in average total PCE across
questionnaire types is not reßected in an analogous difference once we look at items for which the recall
period was the same in both S1 and S2.26
In Figure 1 we draw, for every smaller survey and sector, the densities of m, �m, y, and �y, where it
should be remembered that variables with tildes are related to observations obtained from the experimental
questionnaires. All graphs represent kernel estimates obtained using the robust bandwidth proposed by
Silverman (1986) for the estimation of approximately normal densities.27 It is apparent that in all cases
the distribution of �y (estimated using sample S2) lies to the right of the distribution of y (estimated using
sample S1). At the same time, there is no such large and systematic gap between the distributions of m
and �m. Only in the urban sector of the 51st Round does the difference between the two curves appear not
negligible, but even in this case the two densities closely coincide for low values of expenditure, which is the
relevant range when one is interested in poverty counts.
Any formal test for the equality of f(m) and f( �m) should take into account the complex survey design,
which can considerably affect the variance of the estimates. We construct a simple test by Þrst dividing the
range of m into bins of equal length, and then testing the null hypothesis that the proportion of observations
in every bin is the same across the two schedules. The test is based on a Pearson χ2 statistic modiÞed to take
into account the presence of strata and clusters. Under the null hypothesis, once we tabulate observations
across bins and schedules, the proportion of observations in the cell related to the the i-th bin and the k-th26Deaton (2001) studies in more detail the differences in average per capita consumption for different item categories, estimated
using S1 and S2.27The bandwidth is adapted for a biweight kernel, which is the one we use here.
15
schedule (the �joint� proportion), should be the same as the product of the total proportion of observations in
the i-th bin and the total proportion of observations in the k-th schedule (the two �marginal� proportions).28
The test rejects the null when a normalized sum of the differences between joints and products of marginals
is large. Since the differences are normalized dividing them by the joint proportion, or by the product of
the two marginals, one should be careful to choose the number of bins in such a way that the number of
observations per cell is large enough. To avoid the presence of cells with very few observations, for every
Round-sector we divide into bins only the range between the Þrst and the last percentile of the Round-sector
distribution of the relevant variable. Table 2 shows the results, for different number of bins. We do not
consider a large number of bins, again to avoid having only a few observations per cell. In most cases the
null hypothesis cannot be rejected, and the p-values are above 0.2. The null is never rejected at a Þve percent
signiÞcance level when we use 20 bins. Rejection arises, instead, in the rural sector of the 52nd Round, with
10 or 15 bins, and in both sectors of the 51st Round with 10 bins. Note, though, that even when the null is
rejected the p-value is always close to 0.05, so that the null is never rejected if we test using a one percent
signiÞcance level.
Besides m, we also consider the inclusion in v of observed household characteristics (grouped into the
vector X) that are likely to be correlated with y. These are household size and a series of dummies for
land holdings, education of the household head, main economic activity of the household, and whether the
household belongs to a scheduled caste or tribe. For all these variables, we use the same modiÞed Pearson
χ2 test described above. Again, the null hypothesis is that the distribution of every variable is the same
for the two schedules. The p-values of each test are reported in Table 3. In most cases the data strongly
support the null.29 The only exceptions are education of the head in the rural sector of the 53rd Round, and
main economic activity of the household in the rural sector of the 51st and 53rd Round. However, even in
these cases the differences in the relative proportions of observations per cell between schedules is not large,
as we show in the bottom part of Table 3. The overall picture that emerges is that in all the NSS Rounds
we consider, the equality of the distribution of v and �v holds.
Assumption 2a is more complicated to test, since it involves the comparison of two conditional distribu-
tions, rather than two marginals. In particular, the assumption requires that the conditional distribution of
y given v in the auxiliary survey (here the 50th NSS Round) is the same as the one in the target survey (any
of the smaller surveys we use). Since y is the (log) PCE reported when a 30-day recall period is used for all
items, in the target survey the assumption refers to the conditional distribution of the variable as recorded
in S1.
The relatively small sample size makes the direct estimation of the relevant conditional distributions
troublesome, but some evidence on the validity of Assumption 2a can be obtained using the fact that
the equality of the conditional expectations is a necessary condition for the equality of two conditional28 See Stata Reference manuals and references therein for details (Stata Corporation, 2001).29This is not surprising, since the two subsamples are created randomly, and there is no obvious reason why a change in the
recall period should affect reports about household characteristics.
16
distributions. Even the estimation of the regressions, though, is infeasible without speciÞc assumptions as
to their functional forms, or if the dimension of v is not small enough relative to the sample size to allow
nonparametric estimation. But since the most important conditioning variable is m, we can compare across
surveys the regressions of y onm estimated nonparametrically, without imposing any functional form. Figure
2 shows the resulting locally weighted regressions for all Rounds, for the rural and urban sector separately.30
The curves look extremely similar across the different surveys. Since the functional form appears strikingly
linear, we also test for equality of coefficients across different surveys in separate linear regressions. In each
regression, we include observations from a given sector in the 50th Round, together with observations from
the same sector in one of the smaller surveys. Let Dj be a dummy variable equal to one when the j-th
household belongs to the smaller survey. Then each regression has the form
yj = β0 + β1mj + α0Dj + α1Djmj + uj
where uj is the error term. We report the results in Table 4. In all regressions, we cannot reject the
hypothesis that α0 or α1 are equal to zero, but tests for the joint signiÞcance reject the null in most cases.
In all cases, though, the dummies are very small in absolute value, so that both the constant and the slope
remain very similar across surveys. To further check the similarity of the conditional distribution of y given
m, we estimate locally weighted regressions on m of the squared residuals computed from the nonparametric
regressions shown in Figure 2. These regressions, which estimate the conditional variance of y given m, are
plotted in Figure 3. Here the differences among the curves are much larger than the differences among the
conditional expectations in Figure 2. Still, the pattern and the levels are similar in the central part of the
range, where most observations are located.
The results are less encouraging if other covariates are added to v. When we include in the linear
regressions one or more of the variables in X, in most cases we cannot reject the hypothesis that some of the
coefficients differ across surveys, and in all cases a joint test strongly rejects the null of equal coefficients.31
Also, a nonparametric analysis becomes impossible, due to the curse of dimensionality, so that entering all
variables linearly into each regression becomes more controversial. Since, as we show in the next section,
including X in v does not change signiÞcantly our estimates, our preferred estimator will be one that uses
only m, for which the evidence of the validity of Assumption 2a is stronger.
Assumption 2a is only required when one is interested in estimating the marginal density of y in the
target survey. For the estimation of poverty counts, the weaker Assumption 2 is needed, requiring that the
conditional probability of being poor given v is the same in the auxiliary and the target surveys. If we
consider again a case where v includes only m, we can check the validity of this condition estimating locally30 In a locally weighted regression a �local� OLS regression is run at every point where the conditional expectation is evaluated
(see Fan, 1992). The regression is local since at every point we use only observations for which the regressor is inside a
neighborhood of the point itself, deÞned by a bandwidth. The observations are weighted by using a kernel, so that observations
closer to the point have more weight in the regression itself.31These results for different combinations of regressors are not shown here, and they are available upon request from the
author.
17
weighted regressions on m of a dummy equal to one when a household is poor. The resulting curves are
shown in Figure 4. Each graph displays a sector-speciÞc regression for the 50th Round, and compares it with
the analogous regression from one of the smaller surveys. In every case, the two lines are very close, even if,
with the exception of the urban sector in the 53rd Round, the regression for the 50th Round lies everywhere,
or almost everywhere, slightly above the corresponding line from the auxiliary survey. Note, though, that
no systematic pattern in the way the curves move over time is apparent.
All curves suggest that they might be well described by a parametric binary variable model, so we run a
sector-speciÞc logit regression for each target survey, pooling together observations from the target and the
auxiliary survey, and we test the null hypothesis that the coefficients are the same in the two populations.
The results are reported in Table 5, and they again support the assumption that the conditional probabilities
are the same, or that they differ only marginally, in the auxiliary and the target survey. The null hypothesis
that the coefficients for the dummies and the interaction terms are individually equal to zero is never rejected,
with the exception of the urban sector when the 53rd Round is the target survey (even in this case the t-
values are approximately equal to 2, and the coefficients are not large in absolute value). When we test for
the joint signiÞcance of the interaction and the dummy, the null is strongly rejected only when the target
survey is the urban sector in the 52nd Round. When the target is the rural sector of the 51st or 52nd Round,
the null is rejected at the 5 percent signiÞcance level, but not if we use a 1 percent level of signiÞcance.
The overall picture is encouraging, and our experiment suggests that the assumptions needed for the
adjustment procedure to work are supported by the data. In the next section we proceed then with our
experiment, in order to evaluate the performance of the proposed estimators.
5.2 Results
In this section, we use Schedule 2 from the 51st, 52nd, and 53rd NSS Round as target surveys, keeping the 50th
Round as our auxiliary survey. The goal is to use our reweighting procedure to estimate poverty counts and
the density of y for each target population, using the data collected through the experimental questionnaires
in the smaller surveys, and to compare these results with those one would obtain using standard estimators
applied directly to the data recorded in the standard questionnaires from the same small NSS Rounds. If
the reweighting procedure performs well, the two sets of results should be close to each other, since the fact
that the two schedule types were assigned randomly should guarantee that both subsamples represent the
same population.
We analyze Þrst the whole density of y, and then we turn to the computation of poverty counts. Most of
our analysis will be done including only m (the logarithm of per capita expenditure on 30-day items) into
the vector of covariates v. The use of other household-speciÞc covariates makes the validity of Assumption
2 more dubious, and using a univariate v also allows us to compare the results we obtain using a parametric
18
Þrst step with those we obtain when the reweighting function is estimated nonparametrically. We will also
show that including other variables in v does not change the results substantially.
As a Þrst step to compute the reweighting function, we have to estimate �P (τ j = 1), which is the weighted
proportion of individuals from the target survey (see equation 14), and �P (τ j = 1 | vj).32 If v is a vectorof discrete variables, the latter estimate is the fraction of individuals belonging to the target survey, given
the set of those for which the vector of covariates assumes value equal to vj . Since in our case the most
important component of the vector vj is a continuous variable, namely mj , we can more generally interpret
�P (τ j = 1 | vj) as the estimated probability that a household belongs to the target survey once the researcherobserves vj . Such probability can be estimated regressing the binary variable τ j on vj , using a logit or probit
model or, if the dimension of vj is relatively small, using a nonparametric regression. Once we have estimated
�P (τ j = 1) and �P (τ j = 1 | vj), we calculate the reweighting function �R (vj) using (9).When we estimate �P (τ j = 1 | vj) with a parametric regression, we always enter mj (and household size
when it is included in vj) in polynomial form. This is important, since both the logit and probit models
imply monotonicity in the relation between the estimated probability and each regressor. Since here the
dependent variable is a binary variable equal to one when the household belongs to the target population,
we have no reason to assume a priori that the relation between �P (τ j = 1 | vj) and mj is monotonic. In
other words, we cannot rule out that both low and high values of mj are more common in one of the two
surveys, with the values in the central range more common in the other. This would be the case, for example,
if mj had an approximately normal distribution, both in the target and in the auxiliary survey, with the
same mean but different variances. Using a polynomial allows us to relax this monotonicity implication.
Note also that the same concern is not relevant when we enter household-speciÞc variables represented by
categorical variables, since each possible value is entered into the regression as a separate dummy variable
into the regression.
For illustrative purposes, Figures 5 and 6 show the estimated curves when we use S2 from the 52nd
Round as the target survey, for the rural and urban sector respectively.33 The graphs to the left refer to
the estimated conditional probability �P (τ j = 1 | vj), plotted against m, while those to the right are theresulting reweighting functions, plotted with y on the horizontal axis. The graphs in the Þrst row show the
results when we estimate �P (τ j = 1 | vj) with a logit model, using only a polynomial in m as regressor. The
two graphs below refer to the case where we also enter into vj a polynomial in household size, and dummy
variables for education of the head, land holdings, scheduled caste or tribe, and main economic activity of
the household. The Þnal row shows the graphs when we estimate the Þrst step nonparametrically, using a
locally weighted regression of τ j on mj . If the two populations analyzed are identical, knowing vj would not
provide any information about which population the selected household comes from, so that the conditional
probabilities shown to the left would all be approximately ßat at 0.5, and the reweighting functions to the32Remember that τj is deÞned as a dummy variable equal to one when household j belongs to the target population.33The analogous graphs for the 51st and 53rd round are available upon request from the author.
19
right would be approximately ßat at 1. This is actually what happens for most values in the urban sector
of the 52nd Round, when we use a polynomial in m, or a nonparametric regression in the Þrst stage. The
curves are ßat, and upward sloping only for large values of m. In the rural sector, instead, low values ofm are
generally associated with a probability of belonging to the target population above 0.5, and vice-versa. These
results are easily explained looking at Figure 7, where we plot the cumulative distribution functions of m
from both the target and the auxiliary survey, in the rural and urban sector separately. The top panel shows
that in the rural sector the distribution from the 50th Round stochastically dominates the distribution from
the 52nd Round, so that higher values of m have a higher probability in the older survey. The two curves in
the urban sector are instead almost identical, but the distribution from the 50th Round lies generally above
the other for high values of m, which explains why the conditional probabilities in Figure 6 are upward
sloping at high values of m.
Once we have estimated the reweighting functions, we can plug them into equation (15) to estimate the
density of y from S2 over a grid of points. Figure 8 shows the resulting densities for all target surveys. Each
graph shows standard kernel estimates of the densities of y from S1 and �y from S2 (the dashed and dotted
line respectively), together with the density of y from sample S2 estimated using our reweighting procedure
(the continuous line). The results are encouraging, since it is apparent that in all cases the adjusted density is
much closer to the density estimated using sample S1 (which is our benchmark) than the one estimated using
S2. However, even if part of the distance between the adjusted density and the benchmark might be due to
sampling error, it can be noticed that the gap between the two is not negligible in some cases, especially in
the rural sector, where our procedure produces densities that generally lie slightly to the left of the densities
from S1, so that in the rural sector we can expect that our adjustment will slightly overstate poverty. This
is reßected in Figure 9, where we show differences between the estimated cumulative distribution functions
of the logarithm of PCE, computed by numerical integration of the densities shown in Figure 8. In each
graph, the line farther away from zero represents the distance between the cumulative distribution function
of �y (from S2) and the benchmark c.d.f. of y from sample S1. The other line shows the difference between
the adjusted c.d.f. of y from S2 and the benchmark c.d.f. from S1. We are particularly interested in the
distances evaluated at the poverty lines, which we denoted by vertical lines in the graphs. While in the
urban sector the distances are generally very small, this is not always the case in the rural sector, where
the distance evaluated around the poverty line is in all cases approximately Þve percent. Still, our estimator
provides estimates that are much closer to the benchmark than those obtained from sample S2.34
Table 6 reports all the point estimates for the poverty counts. Columns 1 and 2 contain the headcount
poverty ratios straightforwardly calculated using (3), as proportions of individuals living in households with
PCE below the relevant poverty line. Column 1 shows the results one gets using households belonging to34All graphs represented in Figures 8 and 9 appear extremely similar if the reweighting function is estimated with a non-
parametric Þrst step, if we include all household-speciÞc covariates in the logit model, or if we use a probit instead of a logit
model.
20
sample S1, and column 2 shows the much lower estimates one obtains using reports from households who have
been given the experimental questionnaire.35 The next four columns report the poverty counts estimated
using our reweighting procedure, when we use a logit model in the Þrst step. In columns 3 and 4, the poverty
counts are computed using (13), while in columns 5 and 6 we compute them by numerical integration, once
we have estimated the density of y over a grid of points using (15). The estimation of the reweighting function
is done including only a polynomial in m in columns 3 and 5, and including also a polynomial in household
size and dummies for the other household-speciÞc covariates, in columns 4 and 6.36 Finally, the last two
columns contain the point estimates when the reweighting function is estimated nonparametrically, using a
locally weighted regression. Our estimates are much closer to the poverty counts reported in column 1 (our
benchmark) than those in column 2, which are considerably lower, as a consequence of the experimental
design of the questionnaire given to the households in sample S2. The performance of our estimator is
particularly satisfactory in the urban sector, where the adjusted headcount ratios generally differ from the
counts reported in column 1 by 1-2 percentage points. In the rural sector, the performance is not as good,
and in all cases our estimator overstates poverty by 4-5 percentage points, as was also anticipated by the
graphs in Figure 9. Note that it is apparent that the results in columns 3 to 9 are very close to each other,
and no estimator systematically outperforms the others. Also, it is interesting to note that the performance
of the estimators does not worsen when the distance between the target and the auxiliary survey increases.
This provides further support for Assumption 2a, which requires the constancy of the distribution of y given
v across the target and the auxiliary survey. If, for example, the relation between y and m changed quickly
due to changes in relative prices or preferences, one would have expected the performance of the estimator
to worsen once we consider target surveys less close in time to the auxiliary survey.
6 Adjusted Poverty Estimates from the 55th Round of the Na-
tional Sample Survey
Data from the 50th Round of the National Sample Survey, carried out in 1993-94, showed that about a third
of the Indian population was living with a PCE below the poverty line. In rural areas the proportion was
37.2 percent, while 32.6 percent of the urban population was estimated to be poor. After only six years, the
official poverty counts estimated using the 55th Round show an impressive reduction in poverty. Using the
consumption levels reported with a 30-day recall period for food, poverty counts are now approximately 1035These are the same Þgures reported in Table 1. The standard errors in columns 1 and 2 are computed using the survey
commands in Stata, which allow to compute standard errors adjusted for statiÞcation and clustering.36We estimated the standard errors in columns 3 and 4 using 50 bootstrap replication, taking into account the clustered survey
design. This means that in each replication the resampling is done with respect to the clusters, and not to the households.
Once a cluster is included in a bootstrap sample, all households in the cluster are included in the sample (see Deaton, 1997,
Ch. 1).
21
percentage points lower, at 27.1 in the rural sector, and 23.6 in urban areas. Taking into account only the
major Indian states, which are those we will consider in our analysis, we estimate that the unadjusted poverty
counts based on a 30-day recall are respectively equal to 28.4 and 24.5. In a country where the population has
recently surpassed one billion of individuals, these numbers would translate into approximately one hundred
million people out of poverty in less than a decade.
In the previous pages, though, we have described the reasons why we believe that these Þgures are likely
to underestimate poverty: in reporting household consumption of food on a 30-day basis, the respondent will
probably reconcile her answers with those given with a 7-day recall period, which are typically proportionally
higher. Since food, in India, still account on average for a very large part of household budgets, we should
expect PCE with a 30-day recall period from the 55th Round to be higher than the one we would have
observed if the standard recall period had been used in isolation. The reweighting procedure described
in the paper should help recovering poverty counts comparable with those computed from previous NSS
Rounds.
Table 7 contains the sector-speciÞc adjusted estimates for the poverty counts, when we use only m as
conditioning variable. As a robustness check, we use four different auxiliary surveys, all the NSS surveys
between the 50th and the 53rd Round. As usual, all values are deßated using the official sector and state
speciÞc price indexes. Column 1 presents the headcounts obtained when the Þrst step is estimated by a logit
model, while in column 2 we use a locally weighted regression. It is apparent that the choice of the Þrst step
estimator is basically immaterial. The choice of the auxiliary survey, instead, does produce different results,
even if the discrepancies are not large, and in both sectors all estimates are no more than approximately
2 percentage points apart. Even after the adjustment, the poverty counts remain well below their levels
as measured in the 50th Round. The adjusted Þgures for the rural sector are, as expected, larger than
the unadjusted ones, but the gap is very small, and equal to 1-2 percentage points, depending upon which
auxiliary survey is used. It is interesting to note, instead, is that our adjustment does not revise upwards
the estimates for urban areas. These results raise some questions, since in the smaller experimental surveys
carried out between the 50th and the 55th NSS Round the discrepancies in reports due to differences in
the questionnaire between S1 and S2 were very similar in the rural and urban sector. It is unclear, then,
why the adjustment works differently in the two sectors. One possibility is that the questionnaire changes
implemented in the 55th Round affected differently the reports from urban and rural dwellers. In particular,
the latest survey collected data on consumption out of home production differently, and used a 365-day
recall period for durables in both sets of reference periods. While home production is clearly more important
in rural areas, consumption of durables is more relevant in the urban sector. If, and how, these issues
might explain the observed differences in the adjustment should be the object of further study. In any case,
though, our results strongly suggest that the change in the questionnaire affected mostly the 7-day reports,
and not much the standard 30-day reports. In fact, while the adjusted poverty counts are very similar to
the unadjusted ones, it is still true that the gap between the two sets of reports is much lower than the one
22
observed in each of the smaller experimental surveys. Actually, since our empirical experiment show that
in the rural sector the adjustment tends to moderately overestimate poverty, it is possible that, even in the
rural sector, the unadjusted Þgures are comparable with the poverty counts computed from previous surveys.
Note, Þnally, that even if this were the case, the usefulness of our procedure in the present context would
remain unaltered, since without our analysis it would have been impossible to judge the comparability of
the estimates.
7 Caveats and Conclusions
In this paper, we have proposed a new method for estimating comparable poverty counts and comparable
densities of a variable y when such variable is measured differently in different surveys. Our method only
requires that each survey contains data on a set of variables v, correlated with y, whose reports are not
affected by the differences in survey design across the two surveys, and that the conditional distribution of y
given v is the same in the two surveys. Our ultimate objective is to provide tools to estimate poverty counts
using data from the 55th Round of the Indian National Sample Survey (July 1999 - June 2000), which we
couldn�t otherwise use for this purpose, due to important changes in the survey questionnaire with respect
to previous NSS Rounds. In the paper we describe why the evidence suggests that the unadjusted official
poverty counts are likely to understate the extent of poverty.
We use smaller experimental surveys carried out before the 1999-2000 survey to support the validity of
the identifying assumptions needed for our estimators, and to evaluate their performance. In our empirical
experiment the estimators perform well, especially in estimating poverty counts and densities for the urban
sector.
Even after adjusting for the changes in the survey module, we observe an impressive reduction in poverty
in India. Headcount poverty ratios are now about a third lower than in 1993-94, when the previous large
NSS survey had been carried out. According to our estimates, poverty counts in India is now close to 30
percent in rural areas, and 25 percent in urban areas.
It should be stressed that in this paper we do not suggest the superiority of one set of recall period over
the others. Our scope is to provide a method for estimating comparable poverty counts, from data measured
in non-comparable way. Our estimator, then, should not be interpreted as a tool to solve a measurement
error problem, since the variable we are trying to recover, y, might be itself (and surely is) measured with
error. In fact, in our context y is not the �true� PCE, but it is instead total PCE reported on a standard
questionnaire, with a 30-day recall period for all consumption items.
Since changes in questionnaire are implemented frequently in surveys carried out all over the world, the
adjustment procedure proposed here can have many applications, besides its use with NSS data. Our esti-
mators can be useful every time one is interested in comparing distributions of variables recorded differently
23
in different data sources, if one has reason to believe that the needed assumptions hold true in that speciÞc
case. Once again, we remind that such assumptions will have to be accepted as conditions necessary to
achieve identiÞcation, which one cannot test. Then, their credibility should be carefully evaluated on a case
by case basis.
References[1] Bradburn, Norman M., Janellen Huttenlocher, and Larry Hedges, 1994, �Telescoping and Temporal
memory� in Norbert Schwartz and Seymour Sudman (eds), Autobiographical Memory and the Validityof Retrospective Reports, New York, Springer-Verlag.
[2] Datt, Gaurav, 1999, �Has poverty declined since economic reforms?�, Economic and Political Weekly,December 11, pp 3516�18.
[3] Deaton, Angus S., 1997, The analysis of household surveys: a microeconometric approach to developmentpolicy, Baltimore, MD, Johns Hopkins University Press.
[4] Deaton, Angus S., 2001, �Survey design and poverty monitoring in India�, Research Program in Devel-opment Studies, Princeton University, processed.
[5] Deaton, Angus S., and Margaret Grosh, 2000, �Consumption,� Chapter 5 in Margaret Grosh and PaulGlewwe, eds., Designing household survey questionnaires for developing countries: lessons from 15 yearsof the Living Standards Measurement Study, Oxford University Press for the World Bank, Vol 1., pp.91�133.
[6] Deaton, Angus S., and Alessandro Tarozzi, 2000, �Prices and poverty in India,� Research Program inDevelopment Studies Working Paper no. 196, Princeton University.
[7] Di Nardo, John, Nicole M. Fortin, and Thomas Lemieux, 1996, �Labor Market Institutions and theDistribution of Wages, 1973-1992: a Semiparametric Approach�, Econometrica, Vol. 64, 5, 1001-1044.
[8] Elbers, Chris, Jean O. Lanjouw, and Peter Lanjouw, 2001, �Welfare in Villages and Towns: Micro-LevelEstimation of Poverty and Inequality�, mimeo.
[9] Fan, Jianquing, 1992, �Design-adaptive Nonparametric Regression�, Journal of the American StatisticalAssociation, 87, 998-1004.
[10] Gibson, John, 1999a, �Measuring Poverty with Noisy Estimates of Annual Expenditures: Evidence fromPapua New Guinea, Department of Economics�, University of Waikato, 1999.
[11] Gibson, John, 1999b, �How Robust are Poverty Comparisons to Changes in Household Survey Methods?A Test Using Papua New Guinea Data�, Department of Economics, University of Waikato.
[12] Gibson, John, 2001, �Measuring chronic poverty without a panel�, Journal of Development Economics,65(2), pp. 243-266.
[13] Government of India, 1988, National Sample Survey, 43rd Round: July 1987-June 1988: Instructions toField Staff: Volume I, Delhi, National Sample Survey Organization.
[14] Government of India, 1993, Report of the Expert Group on the estimation of proportion and number ofpoor, Delhi, Planning Commission.
[15] Government of India, 2001, Poverty Estimates for 1999-2000, Press Information Bureau, 22nd February.
[16] Government of India, various years, Statistical abstract, Delhi, Central Statistical Organization, Depart-ment of Statistics, Ministry of Planning.
24
[17] Howes, Stephen, and Jean Olson Lanjouw, 1998, �Does Sample Design Matter for Poverty Rate Com-parisons?, Review of Income and Wealth, Series 44, no. 1.
[18] Howes, Stephen, and Jean Olson Lanjouw, 1995, �Making Poverty Comparisons Taking Into AccountSurvey Design: How and Why�, Policy Research Department, World Bank, Washington, D.C., and YaleUniversity, New Haven, processed.
[19] Lanjouw, Jean Olson, and Peter Lanjouw, 2001, �How to Compare Apples and Oranges: PovertyMeasurement based on Different DeÞnition of Consumption�, Review of Income and Wealth, 47 (1), pp.25-42.
[20] Mahalanbois, P. C., and S. B. Sen, 1954, �On some aspects of the Indian National Sample Survey,�Bulletin of the International Statistical Institute, Vol. 34, part II.
[21] Neter, John and Joseph Waksburg, 1964, �A Study of Response Errors in Expenditure Data fromHousehold Interviews�, Journal of the American Statistical Association, Vol. 59, No. 305, pp. 18-55.
[22] Newey, Whitney, and Daniel McFadden, �Large Sample Estimation and Hypothesis Testing�, in Hand-book of Econometrics, Volume IV, R.F. Engle and D.L. McFadden eds., Elsevier.
[23] Pagan, Adrian, and Aman Ullah, 1999, Nonparametric Econometrics, Cambridge, Cambridge UniversityPress.
[24] Parzen, Emanuel, 1962, �On Estimation of a Probability Density and Mode�, Annals of MathematicalStatistics, 33, 1065-1076.
[25] Ravallion, Martin, 1993, Poverty Comparisons: a Guide to Concepts and Methods, LSMS WorkingPaper 88, Washington, D.C., World Bank.
[26] Rubin, David C. and Alan D. Baddeley, 1989, �Telescoping is not Time Compression: a Model of theDating of Autobiographical Events�, Memory and Cognition, 17.
[27] Sachs, Jeffrey D., Ashutosh Varshney, and Nirupam Bajpai, 1999, India in the Era of Economic Reforms,Oxford, Oxford University Press.
[28] Sen, Abhijit, 2000, �Estimates of Consumer Expenditure and Its Distribution: Statistical Prioritiesafter NSS 55th Round�, Economic and Political Weekly, December 16, 2000, 4499-4518.
[29] Silverman, Bernard W., 1986, Density Estimation for Statistics and Data Analysis, London, Chapmanand Hall.
[30] Stata Corporation, 2001, Stata Reference Manuals, Release 7, College Station, Stata Press.
[31] Visaria, Pravin, 2000, �Poverty in India during 1994�98: alternative estimates,� Institute for EconomicGrowth, New Delhi, processed.
25
log(Rupees), Rural
f(m), 51, Sch. 1 f(y), 51, Sch. 1 f(m), 51, Sch. 2 f(y), 51, Sch. 2
1 90
1
log(Rupees), Urban
f(m), 51, Sch. 1 f(y), 51, Sch. 1 f(m), 51, Sch. 2 f(y), 51, Sch. 2
1 90
1
log(Rupees), Rural
f(m), 52, Sch. 1 f(y), 52, Sch. 1 f(m), 52, Sch. 2 f(y), 52, Sch. 2
1 90
1
log(Rupees), Urban
f(m), 52, Sch. 1 f(y), 52, Sch. 1 f(m), 52, Sch. 2 f(y), 52, Sch. 2
1 90
1
log(Rupees), Rural
f(m), 53, Sch. 1 f(y), 53, Sch. 1 f(m), 53, Sch. 2 f(y), 53, Sch. 2
1 90
1
log(Rupees), Urban
f(m), 53, Sch. 1 f(y), 53, Sch. 1 f(m), 53, Sch. 2 f(y), 53, Sch. 2
1 90
1
Figure 1Source: author’s computation from NSS - All India f(m) is the density of log(PCE in 30-day items), while f(y) is the density of log(PCE). All values are in 1993-94 Rupees, deflatedusing CPIAL for the rural sector, and CPIIW for the urban sector. All densities are estimated using the robust bandwidthproposed by Silverman (1986) for approximately normal densities, adapted for a biweight kernel, which is the one we use here.
y=g(
m)
Loca
lly W
eigh
ted
Reg
. -
Ru
ral -
Sch
. 1
log(per capita exp. in 30-day items)
NSS 50, rural NSS 51, rural, Sch.1 NSS 52, rural, Sch.1 NSS 53, rural, Sch.1
2.5 5.6
4.75
6.75
y=g(
m)
Loca
lly W
eigh
ted
Reg
. -
Urb
an -
Sch
. 1
log(per capita exp. in 30-day items)
NSS 50, urban NSS 51, urban, Sch.1 NSS 52, urban, Sch.1 NSS 53, urban, Sch.1
2.8 6.6
4.9
7.4
Figure 2 – Locally weighted regressions of log(PCE) on log(PCE in 30-day items) Source: Author’s computations from NSS, rounds 50-53, all major Indian states. All lines are drawn only for values of the regressor in the interval between the 1st and the 99th centile of the corresponding distribution, to avoid showing the noisy estimates outside those intervals. We use a fixed bandwidth in every regression. The bandwidth is 0.2 in the rural sector of all smaller surveys and in the urban sector of the 50th round, 0.15 in the rural sector of the 50th round, and 0.3 in the urban sector of the smaller surveys.
Loca
lly w
eigh
ted
regr
essi
ons,
Ban
dwid
th =
0.5
Rural sectorlog(m)
NSS 50 - 93-94 NSS 52 - 95-96 NSS 51 - 94-95 NSS 53 - 97
2.5 5.6
.072283
.207077
Loca
lly w
eigh
ted
regr
essi
ons,
Ban
dwid
th =
0.5
Urban sectorlog(m)
NSS 50 - 93-94 NSS 52 - 95-96 NSS 51 - 94-95 NSS 53 - 97
2.8 6.4
.058649
.181788
Figure 3 Locally weighted regression of squared residuals from first stage nonparametric regression of
log(PCE) on log(PCE in 30-day items) Source: Author’s computations from NSS, rounds 50-53, all major Indian states. The vertical lines correspond to the 5th and 95th sector specific percentile of the distribution of the regressor, computed pooling together all four surveys. The residuals are those computed from the nonparametric regressions shown in figure 2.
% b
elow
pov
erty
lin
e -
Ru
ral
log(pce in 30-day items)
NSS 51 NSS 50
2.5 6.50
.5
1
% b
elow
pov
erty
lin
e -
Urb
an
log(pce in 30-day items)
NSS 51 NSS 50
2.5 6.50
.5
1
% b
elow
pov
erty
lin
e -
Ru
ral
log(pce in 30-day items)
NSS 52 NSS 50
2.5 6.50
.5
1
% b
elow
pov
erty
lin
e -
Urb
an
log(pce in 30-day items)
NSS 52 NSS 50
2.5 6.50
.5
1
% b
elow
pov
erty
lin
e -
Ru
ral
log(pce in 30-day items)
NSS 53 NSS 50
2.5 6.50
.5
1
% b
elow
pov
erty
lin
e -
Urb
an
log(pce in 30-day items)
NSS 53 NSS 50
2.5 6.50
.5
1
Figure 4Conditional probability of being poor given log(PCE in 30-day items)
Source: author’s computations from NSS, rounds 50-53, all major Indian States. Locally weighted regressions. The dependent variableis a dummy variable equal to one when the household’s per capita monthly expenditure is below the sector-specific poverty line. Thebandwidth is equal to 0.5 for all curves.
Locally Weighted Regression first step
P(ta
rget
su
rvey
|v)
log(PCE in 30-day items)2 3 4 5 6 7
0
.5
1
Locally Weighted Regression first step
Rew
eigh
tin
g fu
nct
ion
log(pce, all items)3 5 7 9
0
1
2
3
4
5
First step: logit, all covariatesP(
targ
et s
urv
ey|v
)
log(PCE in 30-day items)2 3 4 5 6 7
0
.5
1
First step: logit, all covariates
Rew
eigh
ting
func
tion
log(pce, all items)3 5 7 9
0
1
2
3
4
5
First step: logit, all covariates
P(ta
rget
su
rvey
|v)
log(PCE in 30-day items)2 3 4 5 6 7
0
.5
1
First step: logit, all covariates
Rew
eigh
ting
func
tion
log(pce, all items)3 5 7 9
0
1
2
3
4
5
Figure 5 - Rural sector - 52nd NSS round
Source: author’s computation from NSS, 50th and 52nd round. The graphs in the first row are built using a logit in the first step, with apolynomial in log(pce 30-day items) as regressor. The graph in the second row are computed in the same way but adding among theregressors a polynomial in household size, and dummy variables for education of the head, land holding, scheduled caste or tribe, andmain economic activity of the household. The last row represents the result when we estimate the conditional probability in the first stagewith a locally weighted regression, using a bandwidth equal to .25. In each graph vertical lines are drawn corresponding to the 1st and 99th
percentile of the distribution of the variable on the horizontal axis. The central vertical line drawn in the graphs to the right indicate thesector specific poverty line in 1993-94 Rs.
Locally Weighted Regression first step
P(ta
rget
su
rvey
|v)
log(PCE in 30-day items)2 3 4 5 6 7
0
.5
1
Locally Weighted Regression first step
Rew
eigh
ting
func
tion
log(pce, all items)3 5 7 9
0
1
2
3
4
5
First step: logit, all covariatesP(
targ
et s
urv
ey|v
)
log(PCE in 30-day items)2 3 4 5 6 7
0
.5
1
First step: logit, all covariates
Rew
eigh
ting
func
tion
log(pce, all items)3 5 7 9
0
1
2
3
4
5
First step: logit, all covariates
P(ta
rget
su
rvey
|v)
log(PCE in 30-day items)2 3 4 5 6 7
0
.5
1
First step: logit, all covariates
Rew
eigh
ting
func
tion
log(pce, all items)3 5 7 9
0
1
2
3
4
5
Figure 6 - Urban sector - 52nd NSS round
Source: author’s computation from NSS, 50th and 52nd round. The graphs in the first row are built using a logit in the first step, with apolynomial in log(pce 30-day items) as regressor. The graph in the second row are computed in the same way but adding among theregressors a polynomial in household size, and dummy variables for education of the head, land holding, scheduled caste or tribe, andmain economic activity of the household. The last row represents the result when we estimate the conditional probability in the first stagewith a locally weighted regression, using a bandwidth equal to .25. In each graph vertical lines are drawn corresponding to the 1st and 99th
percentile of the distribution of the variable on the horizontal axis. The central vertical line drawn in the graphs to the right indicate thesector specific poverty line in 1993-94 Rs.
CD
F
Rural sectorlog(PCE in 30-day items)
NSS 50 NSS 52, Sch. 2
2 70
.5
1
CD
F
Urban sectorlog(PCE in 30-day items)
NSS 50 NSS 52, Sch. 2
2 70
.5
1
Figure 7
Source: author’s computations from NSS, 50th and 52nd round. Cumulative distribution functions of logarithm of monthly per capita expenditure in 30-day items, computed by numerical integration from densities estimated using a biweight kernel, and Silverman’s optimal bandwidth for approximately normal distribution (adapted for a biweight kernel).
Rural sectorlog(PCE)
Reweighted, 51-2 f(PCE), NSS 51, Sch.1 f(PCE), NSS 51, Sch. 2
3.5 8 0
1
Urban sectorlog(PCE)
Reweighted, 51-2 f(PCE), NSS 51, Sch.1 f(PCE), NSS 51, Sch. 2
3.5 80
1
Rural sectorlog(PCE)
Reweighted, 52-2 f(PCE), NSS 52, Sch.1 f(PCE), NSS 52, Sch. 2
3.5 80
1
Urban sectorlog(PCE)
Reweighted, 52-2 f(PCE), NSS 52, Sch.1 f(PCE), NSS 52, Sch. 2
3.5 80
1
Rural sectorlog(PCE)
Reweighted, 53-2 f(PCE), NSS 53, Sch.1 f(PCE), NSS 53, Sch. 2
3.5 80
1
Urban sectorlog(PCE)
Reweighted, 53-2 f(PCE), NSS 53, Sch.1 f(PCE), NSS 53, Sch. 2
3.5 80
1
Figure 8
Source: author's computations from NSS, rounds 50-53. Densities of log(PCE). We estimate the two densities denoted Sch. 1 and Sch.2 using standard kernel methods, and using Silverman’s (1986) optimal bandwidth, adapted for a biweight kernel, which is the one we use.The reweighted density is obtained through our procedure reweighting the density from the relevant sector in the 50th round, which isthe auxiliary survey in all graphs. The reweighting function is estimated with a logit first step, using only a polynomial in log(PCE in 30-dayitems) as regressor. Then the reweighted density is estimated using a biweight kernel, with a bandwidth equal to 0.1. The vertical linescorrespond to the sector-specific official poverty lines for 1993-94.
NSS 51 - Rural sectorlog(PCE)
S2-S1 Reweighted-S1
4 5 6 7 8
.06
0
-.22
NSS 51 - Urban sectorlog(PCE)
S2-S1 Reweighted-S1
4 5 6 7 8
.06
0
-.22
NSS 52 - Rural sectorlog(PCE)
S2-S1 Reweighted-S1
4 5 6 7 8
.06
0
-.22
NSS 52 - Urban sectorlog(PCE)
S2-S1 Reweighted-S1
4 5 6 7 8
.06
0
-.22
NSS 53 - Rural sectorlog(PCE)
S2-S1 Reweighted-S1
4 5 6 7 8
.06
0
-.22
NSS 53 - Urban sectorlog(PCE)
S2-S1 Reweighted-S1
4 5 6 7 8
.06
0
-.22
Figure 9
Source: author's computations from NSS, rounds 51-53. Differences between cumulative distribution functions. S2 is the c.d.f. oflog(PCE) estimated using reports from sample S2, while S1 is the corresponding c.d.f. estimated using reports from S1. Both c.d.f.’s areestimated dividing the range of log(PCE) into a grid of 1000 points, and then estimating the weighted proportion of individuals whoselog(PCE) is below every point. The “Reweighted” cdf is the one for sample S2 obtained by numerical integration of a density that weestimated using our reweighting procedure. Here the first step uses a logit model, with a polynomial in log(PCE in 30-day items) asregressor. The vertical lines correspond to the sector-specific official poverty lines for 1993-94.
Table 1 - Summary statistics
All values in 1993-94 Rupees NSS 50 NSS 51 NSS 52 NSS 53Deflator is CPIIW for urban sectorand CPIAL for rural sector
July 93 - June 94 July 1994 - June 1995 July 1995 - June 1996 January-December 1997
rural urban rural urban rural urban rural urban
Schedule S1 S2 S1 S2 S1 S2 S1 S2 S1 S2 S1 S2
Sample size (no. of households) 58922 38712 13606 13415 9283 9214 12253 12047 8870 8749 12313 9214 16418 10555
Mean per capita total monthlyexpenditure (in 1993-94 Rs.)
277.0 455.7 273.4 310.5 461.6 545.9 278.8 328.1 495.3 560.1 294.1 328.2 468.7 547.4(1.49) (4.55) (13.17) (3.45) (18.78) (20.05) (3.91) (3.35) (10.84) (7.54) (4.59) (4.53) (7.55) (8.50)
Proportion of householdspurchasing 30-day items 99.8 99.9 100 100 100 100 100 100 100 100 100 100 100 100
Mean per capita expenditure in 30-day items (in 1993-94 Rs.)
62.0 130.9 54.4 50.8 131.4 133.4 57.9 56.6 130.4 138.0 61.9 58.9 133.1 139.7 (0.83) (1.51) (1.53) (0.80) (7.75) (7.64) (2.46) (1.07) (2.28) (2.81) (1.12) (1.31) (3.61) (3.37)
Mean budget share of 30-day items 21.6 27.3 20.1 16.0 26.3 21.7 20.3 16.4 26.1 22.4 21.1 17.3 27.3 23.4 (0.06) (0.12) (0.22) (0.17) (0.40) (0.34) (0.16) (0.14) (0.17) (0.18) (0.23) (0.22) (0.21) (0.21)
Headcount Poverty Ratio 38.1 33.4 41.8 22.7 36.3 18.5 38.2 18.4 30.7 15.4 35.7 21.1 33.1 17.5
Source: author's computations from NSS. Standard errors in parenthesis. Only the major Indian states are included. All statistics are weighted using inflation factors. S1 is the standard questionnaire, with a 30-day reference period forall items. S2 is the experimental questionnaire, with a recall period equal to seven days for food and other high frequency items, and 365 days for durables, clothing, footwear, educational andinstitutional medical expenses. The category "30-day items" includes: fuel and light, miscellaneous consumer goods and services, rents, consumer taxes and non-institutional medical expenses.The mean budget shares are averages of household-specific ratios between expenditure in 30-day items and total expenditure. The poverty counts are the proportion of individuals living inhouseholds where per capita expenditure is below the poverty line. The poverty lines are the official ones published by the Planning Commission for 1993-94, and are expressed in 1993-94Rupees. The poverty line is 205.7 for the rural sector, and 283.4 for the urban sector.
Table 2 - Tests for equality of distribution of log(pce in 30-day items) across schedulesp-values - design based adjusted F-tests
NSS Round
51st - 7/94-6/95 Rural 26339 0.0281 0.0570 0.0931
Urban 18168 0.0483 0.2698 0.1455
52nd - 7/95-6/96 Rural 23682 0.0426 0.0307 0.0770
Urban 17224 0.4220 0.2770 0.6345
53rd - 1/97-12/97 Rural 20819 0.3555 0.3922 0.5637
Urban 26119 0.4352 0.2231 0.2841
Source: author's computations from Indian NSS, rounds 51-53, all major Indian states.The null hypothesis is that the distribution across bins is the same across the two different schedules. To avoid the presence of cells withvery few observation, for every round-sector we divide into bins only the range between the first and the last centile of the round-sectordistribution. All tests take into account the presence of clustering and stratification, except in the 51st round, where strata are notconsidered, since for schedule 2 the survey does not report them for the rural sector, and it reports them differently in the urban sector. Alltests are feasible only if every stratum contains at least two primary stage units with valid observations, so, when stratification is taken intoaccount, strata with only one valid primary stage unit are dropped.
number of bins
obs. 10 15 20
Table 3 – Tests for equality of distributions across schedules
Rural Urban Rural Urban Rural Urban
Household size 0.200 0.486 0.438 0.569 0.350 0.219Education of head 0.837 0.651 0.577 0.533 0.014 0.208Main activity 0.007 0.442 0.302 0.479 0.009 0.253Scheduled caste/tribe 0.235 0.919 0.333 0.698 0.473 0.541Land holdings 0.738 0.262 0.867 0.491 0.373 0.330
Cross-tabulation of covariates for which the null hypothesis of equal distribution across schedules is rejected at standard significance levels.
Sch. 1 Sch. 2 Sch. 1 Sch. 2 Sch. 1 Sch. 2
0.156 0.126 0.128 0.113 illiterate 0.490 0.4860.266 0.305 0.275 0.319 lit. no schooling 0.050 0.0350.051 0.059 0.068 0.059 lit. below primary 0.114 0.0950.527 0.510 0.530 0.509 primary 0.131 0.141
middle 0.111 0.133secondary 0.058 0.052above secondary 0.046 0.058
NSS 51, Rural NSS 53, Rural NSS 53, Rural
51st round 52nd round 53rd round
Source: author's computations from 51st and 52nd NSS rounds. All major Indian states.The figures are p-values for Pearson chi-squared statistics corrected for the survey design. The null hypothesis is that, for every sector-round pair, thedistribution of the selected variable is the same across the two different schedules. "Education of head" is one of the following: illiterate, literate with noschooling, literate below primary, primary, middle, secondary, above secondary. "Main activity of the household" is one of the following: in the rural sectorself-employed in non-agriculture, agricultural labor, other labor, self-employed in agriculture and others; in the urban sector self-employed, regularwage/salaried, casual labor, others. "Land holdings" are recorded as a categorical variable, with different codes for different intervals: code 1 is for landholdings below 0.01 acres, code 2 for the interval [0.01,0.2) and so on.
Self-empl. in agr. and others
self-employed in non-agricultureagricultural laborother labor
Table 4 - Test for equality of coefficients across different surveys
Dependent variablelog(pce)
coeff. t-ratio coeff. t-ratio coeff. t-ratio coeff. t-ratio coeff. t-ratio coeff. t-ratio
constant 3.185 216.81 3.186 234.76 3.186 234.76 2.915 100.02 2.915 100.81 2.915 100.81
log(p.c.e. in 30-day items) 0.594 161.98 0.594 173.23 0.594 173.23 0.661 107.91 0.661 108.77 0.661 108.77
dummy=1 if round≠50 -0.033 -0.35 0.059 1.51 -0.054 -0.97 0.045 0.80 0.015 0.31 0.003 0.06
(dummy=1 if round≠50) 0.016 0.64 -0.005 -0.51 0.022 1.60 -0.005 -0.39 0.008 0.72 0.001 0.06 xlog(p.c.e. in 30-day items)
Adjusted Wald test - p-value
Source: author's computations from Indian NSS, rounds 51-53, all major Indian states.All tests take into account the presence of clustering and stratification. All tests are feasible only if every stratum contains at least two primary stage units with valid observations, so, when stratification is taken into account, we drop strata with only one valid primary stage unit.
0.7223
Rural Urban
51 52 53 51 52 53
0.0084 0.0002 0.0660 0.00000.0000
Table 5 - Test for equality of coefficients across different surveys
Dependent variableDummy=1 when household is poor
coeff. t-ratio coeff. t-ratio coeff. t-ratio coeff. t-ratio coeff. t-ratio coeff. t-ratio
constant 13.24 77.07 13.24 77.07 13.24 77.07 17.97 51.75 17.97 51.75 17.97 51.75
log(p.c.e. in 30-day items) -3.65 -79.82 -3.65 -79.82 -3.65 -79.82 -4.33 -53.10 -4.33 -53.10 -4.33 -53.10
dummy=1 if round≠50 -0.10 -0.12 -0.14 -0.26 -0.62 -0.99 -1.17 -1.40 0.08 0.12 -1.27 -2.00
(dummy=1 if round≠50) -0.02 -0.11 0.00 -0.02 0.14 0.84 0.24 1.21 -0.08 -0.51 0.31 2.08 xlog(p.c.e. in 30-day items)
Adjusted Wald test - p-value
Source: author's computations from Indian NSS, rounds 51-53, all major Indian states.
0.0254 0.3187 0.1250 0.00000.0170
All tests take into account the presence of clustering and stratification. All tests are feasible only if every stratum contains at least two primary stage units with valid observations, so we drop strata with only one valid primary stage unit. A household is categorized as poor when monthly per capita expenditure is below a sector-specific poverty line.
0.0930
Rural Urban
51 52 53 51 52 53
Table 6 - Headcount Poverty Ratios
(1) (2) (3) (4) (5) (6) (7) (8)
Target survey H s.e. H s.e. H s.e. H s.e. H H H H
51st - 7/94-6/95 Rural 41.8 1.42 22.7 1.19 46.7 0.86 45.8 1.02 46.2 45.3 46.3 46.3
Urban 36.3 1.88 18.5 1.34 36.0 1.64 35.4 1.81 35.4 34.8 35.6 35.3
52nd - 7/95-6/96 Rural 38.2 0.89 18.4 0.74 42.0 0.81 41.7 0.84 41.6 41.2 41.8 41.9
Urban 30.7 0.83 15.4 0.69 32.9 0.72 32.6 0.76 32.3 32.0 32.8 32.5
53rd - 1/97-12/97 Rural 35.7 1.27 21.1 1.06 40.3 0.83 40.2 1.04 39.8 39.8 40.2 40.1
Urban 33.1 0.83 17.5 0.75 31.7 0.90 31.4 0.73 31.2 30.8 31.7 31.3
The standard errors in columns 3 and 4 are estimated through 50 bootstap resamples, taking into account the clustered survey design.
nonparametric first steplogit first step
using indicator using density using indicator
S2S1not reweighted
S2 - reweighted
using density30-day items
only all covariates 30-day items only
all covariates
Source: author's computation from NSS, rounds 50-53. The poverty lines we use are the official ones valid for the 50th round (205.67 for the rural sector, and 283.44 for the urbansector). All values from subsequent rounds are deflated using CPIAL for housheholds living in rural areas, and CPIIW for those living in urban areas. "All covariates" include polynomialsin log(PCE in 30-day items) and household size, and categorical variables for main economics activity of the household, land holdings, education of the head, whether the household headbelongs to a scheduled caste or tribe. Estimates denoted "using indicator" apply the reweighting procedure directly to a binary variable equal to one when the households is categorized aspoor (see eq. 12 in the text). The estimates denoted "using density" are obtained by numerical integration of the density of log(PCE) computed using the reweighting procedure (see eq. 15in the text). In columns 3-6 the point estimates are almost identical if a probit is used rather than a logit. The estimates in columns 7 and 8 are computed estimating the first stepnonparametrically, with a locally weighted regression and a bandwidth equal to 0.1.
Table 7 - Adjusted Poverty Counts - 55th NSS Round (1999-2000)
Auxiliary survey H s.e. (1) s.e. (2) H s.e.
50th - 7/93-6/94 Rural 31.5 (0.39) (0.30) 31.5 (0.47)
51st - 7/94-6/95 Rural 29.9 (0.80) (1.07) 29.7 (0.76)
52nd - 7/95-6/96 Rural 29.5 (0.77) (0.73) 29.5 (0.78)
53rd - 1/97-12/97 Rural 30.4 (1.15) (1.07) 30.3 (0.95)
50th - 7/93-6/94 Urban 25.5 (0.58) (0.43) 25.4 (0.54)
51st - 7/94-6/95 Urban 24.2 (1.01) (1.11) 24.2 (1.06)
52nd - 7/95-6/96 Urban 23.1 (0.58) (0.69) 23.1 (0.58)
53rd - 1/97-12/97 Urban 25.9 (0.67) (0.71) 25.8 (0.69)
Official Poverty Counts, all India, 1999-2000:Standard Questionnaire (30-day recall period for all items) - Rural 27.1, Urban 23.6Experimental Questionnaire (7-day recall period for food) - Rural 24.0, Urban 21.6
Unadjusted Poverty Counts, only larger Indian States, Standard Questionnaire - Rural 28.4, Urban 24.5The official poverty counts in 1993-94 were 37.2 in the rural sector, and 32.6 in the urban sector.
Source: author's computation from NSS, Rounds 50-51-52-53-55The poverty lines are the official ones for the 50th round (205.67 for the rural sector, and 283.44 for the urban sector). Allmonetary values from subsequent Rounds are deflated using state and sector specific official Consumer Price Indexes (CPIALfor households living in rural areas, and CPIIW for those living in urban areas). All figures are obtained using the estimator ineq. (12) in the text. The estimates in column (2) are computed estimating the reweighting function with a nonparametric firststep, using a locally weighted regression and a bandwidth equal to 0.1. The first set of standard errors in column (1) arebootstrapped, with 100 replications taking into account the clustered survey design, while the second set are computed using theexpression for the variance in proposition 3 in the appendix. The standard errors in column (2) are computed using 50bootstrap replications, taking into account the clustered survey design. All estimates are computed for all major Indian states:Andhra Pradesh, Assam, Bihar, Gujarat, Haryana (urban only), Karnataka, Kerala, Madhya Pradesh, Maharashtra, Orissa,Punjab, Rajasthan, Tamil Nadu, Uttar Pradesh, West Bengal, Delhi (urban only).
All major Indian states
logit first stepPCE in 30-day items only
nonparametricfirst step
(1) (2)
1 Appendix - Asymptotic Properties of the Estimators
Let yj be the variable of interest, as measured using a standard survey (in the context of this paper yj is the logarithmof PCE in household j) and let vj be a vector of observed household covariates. Let τ j be a binary variable equalto one when household j belongs to the target population t, and equal to zero when it belongs to the auxiliarypopulation a. Here we disregard clustering and stratiÞcation, so all observations coming from the same populationare independent and identically distributed. So the following assumption holds
(A1) (yj ,vj | τ j) ∼ i.i.d.
In what follows we will consider all observations (yj ,vj , τ j) as being generated by a D.G.P. described by a jointdistribution f (yj ,vj , τ j) . To avoid overburdening the notation, we will use the same notation f for all marginal andconditional distributions derived from f (yj ,vj , τ j) , and we will use differences in the arguments of these distributionsto differentiate them from one another. So, for example, f (yj | τ j = 0) will be the marginal distribution of yj inpopulation a (which can also be seen as the distribution of yj conditional on sampling from population a), andf (yj | τ j = 1) will be the density of yj in the target population t, which we want to estimate.Under the assumptions described in the paper, once the density of yj in the auxiliary survey is identiÞed, we
can use a �reweighting� function R (vj) to compute the density of yj in the target population without actually usingdata on yj from the target survey. In section 3 of the paper we described the conditions necessary for the followingrelation to hold:
(A2) f (yj | τ j = 1) = E [R (vj) | yj , τ j = 0] f (yj | τ j = 0) = m (yj) f (yj | τ j = 0)
where m (yj) ≡ E [R (vj) | yj , τ j = 0] , and the reweighting function is deÞned as
R (vj) =[1− Pr (τ j = 1 | vj)] Pr (τ j = 1)Pr (τ j = 1 | vj) [1− Pr (τ j = 1)] (i)
where Pr (τ j = 1 | vj) is the probability that observation j belongs to the target population t conditional on observingvj , and Pr (τ j = 1) is the unconditional probability that observation j belongs to the target population. One canstraightforwardly estimate the latter probability using eq. (14) in the paper. Since τ j is a binary variable, theformer (conditional) probability is actually the regression of τ j on vj . In what follows we will assume that the
probability is correctly described by a logit model. So, denoting �vj =h1 vTj
iT, in what follows we use the
following assumption:
(A3) Pr (τ j = 1 | vj) = E (τ j | vj;θ1) = e�vTj θ 1
1+e�vTjθ 1, θ1 ∈ RK
Then, if we denote the unconditional probability E (τ j) with θ0, and letting θ =hθ0,θ
T1
iT, the reweighting function
becomes
R (vj ;θ) =E (τ j | vj ;θ1) (1− θ0)[1−E (τ j | vj ;θ1)] θ0 (ii)
where the dependence of the reweighting function upon the K + 1 unknown parameters is taken explicitly intoaccount.Note that, while the estimation of θ requires the use of both samples, we use only observations from population
a to estimate f (yj | τ j = 1) , once the estimates for θ have been plugged in. Under mild regularity conditions, andif (A3) holds, MLE will be consistent for θ (see, for example, Newey and McFadden, 1994, section 3).
A1
Many of the asymptotic results presented here depend upon the difference between the estimated and the truereweighting function. Such difference can be usefully rewritten using a mean value expansion (in what follows wewill use �hats� to denote estimates):
R³vj ; �θ
´−R (vj ;θ) =
KXk=0
∂R³vj ;�θ
´∂θk
³�θk − θk
´(iii)
where �θ lies between �θ and θ. Note that the existence of the derivative is guaranteed by A3.
1.1 Consistency of the estimator for the density
Let J (a) and J (t) be the sets of households from population a and t respectively, and let na and nt be the numberof observations contained in each set. The estimator of the marginal density of y in the target survey is
�f (y | τ = 1) = 1
nahn
Xj∈J(a)
R³vj ;�θ
´K
µyj − yhn
¶(iv)
where K(.) is a kernel with properties to be deÞned below, and hn is the bandwidth. Note that the summation istaken over the observations from the auxiliary survey only. As in common kernel density estimators, each term inthe summation depends upon the sample size through the shrinking bandwidth hn. To prove consistency we will thenmake use of a corollary of a version of the Dominated Convergence Theorem (see Parzen, 1962) which is commonlyused to study consistency of nonparametric estimators (see, for example, Pagan & Ullah, pp. 361-365). We reportthe corollary here:
Corollary 1 - Suppose K(ψ) is a univariate symmetric kernel on < such that sup |K(ψ)| < ∞, R |K(ψ)| dψ < ∞,limψ→∞ |ψ| |K(ψ)| = 0. Let hn be a sequence of positive constants such that limn→∞ hn = 0. Let λ(ψ) be anotherfunction on < such that R |λ(ψ)| dψ <∞. Let r be a positive constant. Then at every point x0 of continuity of λ
λn (x0) =1
hn
Z +∞
−∞Kr
µy − x0hn
¶λ (y) dy → λ (x0)
Z +∞
−∞Kr (ψ) dψ
The following proposition states the consistency of the estimator in (iv).
Proposition A1 - Assume that A1, A2, and A3 above hold. Moreover:
A4 K (ψ) is a symmetric kernel such that:ZK (ψ) dψ = 1,
Z|K(ψ)| dψ <∞, sup |K(ψ)| <∞, lim
ψ→∞|ψ| |K(ψ)| = 0
A5 The bandwidth h is such that limna→∞ h = 0, and limna→∞ nah =∞
A6R |f(y | τ = 1)| dy <∞
A7RE£R2 (v) | y¤ f (y | τ = 0) dy <∞
A8R ¯̄̄Eh∂R(v,θ)∂θk
| yif (y | τ = 0)
¯̄̄dy <∞ and
RE
·³∂R(v,θ)∂θk
´2| y¸f (y | τ = 0) dy <∞ ∀k
A2
A9 Eh∂R(vj ,θ)∂θk
| yiexists Þnite ∀k = 0, ...,K
Then
p lim1
nahn
Xj∈J(a)
R³vj ;�θ
´K
µyj − yhn
¶= f (y | τ = 1)
Proof. We will prove consistency showing that the estimator converges to f (y | τ = 1) in mean square, which inturn implies convergence in probability. The proof proceeds showing that both the bias and the variance convergein probability to zero. It is useful to rewrite the estimator as follows:
�f (y | τ = 1) =1
nahn
Xj∈J(a)
R (vj ;θ)K
µyj − yhn
¶+
1
nahn
Xj∈J(a)
hR³vj ; �θ
´−R (vj ;θ)
iK
µyj − yhn
¶= A (na) +B (na)
where A (na) is the Þrst summation on the right-hand side, and B (na) is the second one. Note that the abovesums are taken only over households from survey a, so that in the following of this subsection all expectations andvariances are conditional on τ j = 0, unless otherwise speciÞed. The following proves that A (na) is asymptoticallyunbiased for the target density.
limna→∞
E [A (na)] = limna→∞
1
hnE
·R (vj)K
µyj − yhn
¶¸using (A1)
= limna→∞
1
hnE
·E
·R (vj)K
µyj − yhn
¶| yj¸¸
= limna→∞
1
hnE
·m (yj)K
µyj − yhn
¶¸= lim
na→∞1
hn
Zm (yj)K
µyj − yhn
¶f (yj | τ j = 0) dyj
using (A2) = limna→∞
1
hn
ZK
µyj − yhn
¶f (yj | τ j = 1) dyj
= f (y | τ = 1)
the last step follows using (A4) and (A6), and noting that all conditions for Corollary 1 apply. The following showsthat the variance of the Þrst term converges to zero:
limna→∞
V ar [A (na)] = limna→∞
1
naV ar
·R (vj)
1
hnK
µyj − yhn
¶¸using (A1)
= limna→∞
1
na
"E
µR (vj)
1
hnK
µyj − yhn
¶¶2−µE
·R (vj)
1
hnK
µyj − yhn
¶¸¶2#
= limna→∞
1
naE
(·R (vj)
1
hnK
µyj − yhn
¶¸2)− limna→∞
1
na
½E
·R (vj)
1
hnK
µyj − yhn
¶¸¾2= lim
na→∞1
naE
(E
"·R (vj)
1
hnK
µyj − yhn
¶¸2| yj#)
− limna→∞
1
naf (y | τ j = 1)2
= limna→∞
1
hnna
Z1
hnK
µyj − yhn
¶2EhR (vj)
2 | yjif (yj | τ j = 0) dyj − 0
= limna→∞
1
hnnaE³R (vj)
2 | y´f (y | τ = 0)
ZK (ψ)2 dψ
= 0
A3
The last step follows from (A5), while the previous step uses (A7) together with Corollary 1. But then we showedthat both the variance and the bias of A (na) converge to zero when the sample size increases. Then
A (na)MS→ f (y | τ j = 1)⇒ p limA (na) = f (y | τ j = 1)
To complete the proof we have to show that p limB (na) = 0.
B (na) =1
nahn
Xj∈J(a)
hR³vj ;�θ
´−R (vj ;θ)
iK
µyj − yhn
¶
using (iii) =1
nahn
Xj∈J(a)
KXk=0
∂R³vj , �θ
´∂θk
³�θk − θk
´K µyj − yhn
¶
=KXk=0
³�θk − θk
´ 1na
Xj∈J(a)
∂R³vj ,�θ
´∂θk
h−1n Kµyj − yhn
¶We know that p lim�θk = θk ∀k = 0, ...K, so p limB (na) = 0 as long as the term in square brackets converges to aÞnite quantity. Through arguments similar to those applied to prove convergence in probability of A (na) to the truevalue of the density, and using (A8) together with the fact that consistency of �θ implies consistency of �θ, one canprove that
p lim1
na
Xj∈J(a)
∂R³vj ,�θ
´∂θk
h−1n Kµyj − yhn
¶= f (y | τ = 0)E
·∂R (vj ,θ)
∂θk| y¸
where the expected value exists by assumption (A9). Then B (na) will converge in probability to zero, since it is thesum of k + 1 bounded terms, each multiplied by a quantity converging to zero in probability.1 So
p limna→∞
�f (y | τ = 1) = p limna→∞A (nA) + p lim
na→∞B (nA) = f (y | τ = 1) + 0 = f (y | τ = 1)
which completes the proof.
1.2 Consistency of the Estimator for the Headcount Ratio
The estimator of the poverty count in the target population is:
�Ht =1
na
Xj∈J(a)
R³vj ; �θ
´1 (yj < z) (v)
Note that no bandwidth is involved in the estimation of the headcount ratio.
Proposition A2 - Assume that A1, A2, A3 hold. Assume also
A10 Eh∂R(vj ,θ)∂θk
1 (yj < z)iexists Þnite ∀k
Then:p lim
1
na
Xj∈J(a)
R³vj ;�θ
´1 (yj < z) = Ht
1Note that for the convergence of �θ the relevant sample size is na + nt, but with simple random sampling from the two populations,both na and nt increase at the same rate.
A4
Proof. First we use again the mean value expansion in (iii), so that (v) can be rewritten as follows:
�H =1
na
Xj∈J(a)
R (vj ;θ) + KXk=0
∂R³vj , �θ
´∂θk
³�θk − θk
´ 1 (yj < z)=
1
na
Xj∈J(a)
R (vj ;θ) 1 (yj < z) +KXk=0
³�θk − θk
´ 1na
Xj∈J(a)
∂R³vj , �θ
´∂θk
1 (yj < z)
(vi)
sinceR (vj ;θ) 1 (yj < z) are i.i.d., to show that the Þrst summation converges to the expected value ofR (vj ;θ) 1 (yj < z)it�s enough to show that this expected value exists, so that we can apply Kinchine�s Law of Large Numbers. Theexistence of the expected value can be proved as follows:
E [R (vj) 1 (yj < z) | τ j = 0] = E [E [R (vj) 1 (yj < z) | yj , τ j = 0] | τ j = 0]= E [1 (yj < z)m (yj) | τ j = 0]
=
Z +∞
−∞1 (yj < z)m (yj) f (yj | τ j = 0) dyj
=
Z +∞
−∞1 (yj < z) f (yj | τ j = 1) dyj
=
Z z
−∞f (yj | τ j = 1) dyj
= Ht
Assumption A10 guarantees that the second summation in (vi) converges in probability to zero, since³�θk − θk
´p→ 0
∀k. So p lim �Ht = Ht.
1.3 Asymptotic Normality of the Headcount Ratio using the Method of Moments
In order to prove asymptotic normality we make use of the method of moments (MM) framework, rewriting theestimator as a two step MM estimator. Then we use results for two-step MM from Newey and McFadden (1994).In particular, we use Theorem 6.1 (p. 2178). To use this result we need consistency of both �H and �θ. In thepreceding section we already proved the consistency of the estimator for the poverty count �H, for which we also usedconsistency of the ML estimator θ. In this section we take explicitely into account the presence of sampling weights,even if we keep abstracting from the presence of strata and/or clusters.2 In general, weights are necessary whenthe sampled units are not selected using simple random sampling, so that different units (in this case, individuals)have a different probability of being selected. So, suppose that the sampling scheme is such that gS (wj) is theprobability that a unit with multiplier equal to wj is selected (so gS (wj) is the probability that such observationis in the sample). By construction, the sampling weights (or inßation factors) are such that wjgS (wj) = gP (wj)
where this last density represents the density of the same unit in the population. In what follows we will use ES todenote expected values of a random variable when observations are selected using a given sampling scheme (that is,a given set of sampling probabilities), while expected values without the subscript S will denote expectations whenobservations are selected from the population with simple random sampling. Note that ES [wjφ (xj)] = E [φ (xj)] ,
2The reason is that here we are not only interested in proving asymptotic normality, but we also want to obtain an estimate for theasymptotic variance of the estimator in (13), which does use the sampling weights.
A5
where φ (xj) is some function of a random variable xj .3
DeÞneβ ≡
hHt θT1 θ0
iT=hHt θT
iTIn what follows it is useful to note that the estimator for the headcount ratio solves the following equation:X
j∈S(a),S(t)wj1 (τ j = 0)
hR³vj ;�θ
´1 (yj < z)− �Ht
i= 0
where wj is the individual sampling weight (not normalized), and the indicator function 1 (τ j = 0) guarantees thatonly observations from the auxiliary survey are being used in the computation. The fact that the sum is taken overall observations (from both the auxiliary and the target survey) is useful once we want to interpret the estimator asa two step MM estimator. The estimator �β, then, solves the following set of sample moment conditions:
0 =1
na + nt
Xj∈S(a),S(t)
mj
³�β´=
1
na + nt
Xj∈S(a),S(t)
mH,j
³�Ht,�θ
´mθ1,j
³�θ1´
mθ0,j
³�θ0´
=1
na + nt
Xj∈S(a),S(t)
mH,j
³�Ht,�θ
´mθ,j
³�θ´
=1
na + nt
Xj∈S(a),S(t)
wj1 (τ j = 0)hR³vj ;�θ
´1 (yj < z)− �Ht
iwj
hτ j −E
³τ j | vj ; �θ1
´ivj
wj
³τ j − �θ0
´
(vii)
Where θ0 is a scalar, and θ1 is K-dimensional.Let M
³�β´≡ ∇�βmj
³�β´, where in general ∇ab indicates the gradient of a vector b with respect to the vector
of variables a. Finally, letDpd = ES [∇dmp (β)] , d = H,θ , p = H,θ (viii)
SoDHθ is the (K+1)-dimensional vector of the derivatives ofmH,j (Ht,θ) with respect to the vector of parameters
θ, evaluated at their true value, and Dθθ is the (K + 1)× (K + 1) matrix of derivatives of the vector mθ,j (θ) with
respect to θ, again evaluated at the true parameter value.In order to apply Theorem 6.1 in Newey and McFadden, we have to Þnd conditions such that the following results
hold:
3To see this we can use the Law of Iterated Expectations:
ES [wjφ (xj)] = ES [wjE [φ (xj) | wj ]]=
ZwjE [φ (xj) | wj ] gs (wj) dwj
=
ZE [φ (xj) | wj ] gP (wj) dwj = E [φ (xj)]
Note also that once we condition upon wj the expected values in the sample and in the population coincide.
A6
1. β ∈ interior of Θ
2. mj (β) is continuously differentiable in a neighborhood N of β (the true value) with probability approachingone
3. ES [mj (β)] = 0, and EShkmj (β)k2
i<∞
4. EShsup�β∈N
°°°M³�β´°°°i <∞
5. ES [M (β)] is nonsingular
The following proposition summarizes the conditions that have to be satisÞed in order to achieve asymptoticnormality.
Proposition A3 - Let �β =h cHt bθT iT
be the solution of equation (vii) above. If �β is a consistent estimatorfor the vector of parameters β, and the following conditions are satisÞed:
A11 β ∈ interior of Θ, where Θ is a compact set.
A12 E£wjv
2ji
¤<∞, i=1,...,K, E [wj ] <∞, E
hwj [R (vj ;θ) 1 (yj < z)−Ht]2
i<∞.
A13 E£E (τ j | vj ;θ1) [1−E (τ j | vj ;θ1)]vjvTj
¤is nonsingular.
A14 �θ0 ≥ C > 0 for �θ0 in a neighborhood of θ0.
A15 Let D ∈ <k be a vector of constants such that¯̄̄�θ1i
¯̄̄≤ |D1i|∀i = 1, ...,K, E
°°vjvTj °°, E he|vj |T |D|i, E [kvjk],Ehkvjk e|vj |T |D|
iexist Þnite4.
Then cHt is asymptotically normal, and√na + nt
³�Ht −Ht
´d→ N (0,π)
where
π =¡DHH
¢−2ES
hmH −
¡DHθ
¢T ¡Dθθ
¢−1mθ
i hmH −
¡DHθ
¢T ¡Dθθ
¢−1mθ
iT=
¡DHH
¢−2 nES£m2H
¤− 2ES £mHmTθ
¤ ¡Dθθ
¢−1DHθ +
¡DHθ
¢T ¡Dθθ
¢−1ES£mθm
Tθ
¤ ¡Dθθ
¢−1DHθ
o
Proof. Here we show that the conditions listed in the proposition are sufficient for assumptions 1-5 above to hold.Assumption 1 in Theorem 6.1 is trivially satisÞed due to A11. Note, in particular, that this assumption excludescases where θ0 is equal to zero or one. Assumption 2 is easily veriÞed by inspection. To prove the Þrst part ofassumption 3 we have to remember the previous discussion on the sampling weights, and also that
f (y | τ = 1) = E [R (v) | y, τ = 0] f (y | τ = 0)4The existence of D is guaranteed by the compactness of the parameter space.
A7
which is a necessary condition for the consistency of �Ht. Then it is relatively staightforward to prove that the expectedvalue of all moments, evaluated at the true parameter values, is equal to zero.
ES [mH,j (Ht,θ)] = ES [wj1 (τ j = 0) [R (vj ;θ) 1 (yj < z)−Ht]]= ES [1 (τ j = 0)ES [wj [R (vj ;θ) 1 (yj < z)−Ht] | τ j = 0]]= 0
since ξ = ES [wj [R (vj ;θ) 1 (yj < z)−Ht] | τ j = 0] is equal to zero. In fact,5
ξ = ES [wjE [R (vj ;θ) 1 (yj < z) | wj , τ j = 0] | τ j = 0]−Ht=
ZwjE [R (vj ;θ) 1 (yj < z) | wj , τ j = 0] gS (wj | τ j = 0) dwj −Ht
=
ZE [R (vj ;θ) 1 (yj < z) | wj , τ j = 0] gP (wj | τ j = 0) dwj −Ht
= E [R (vj ;θ) 1 (yj < z) | τ j = 0]−Ht= 0
Similarly,
ES [mθ1,j (θ1)] = ES [wj [τ j −E (τ j | vj ;θ1)]vj ]= ES [wjE [[τ j −E (τ j | vj ;θ1)]vj | wj ]]=
ZwjE [[τ j −E (τ j | vj ;θ1)]vj | wj ] gS (wj) dwj
=
ZE [[τ j −E (τ j | vj ;θ1)]vj | wj ] gP (wj) dwj
= E [[τ j −E (τ j | vj ;θ1)]vj ]= E {E [[τ j −E (τ j | vj ;θ1)]vj ] | vj} = 0
ES [mθ0,j (θ0)] = ES [wj (τ j − θ0)] = ES [wjE [(τ j − θ0) | wj ]]=
ZwjE [(τ j − θ0) | wj ] gS (wj) dwj
=
ZE [(τ j − θ0) | wj ] gP (wj) dwj
= E [τ j − θ0] = 0
5 In what follows note that once we condition on the sampling weight, E and ES are the same.
A8
The second part of assumption 3 requires that EShkmj (β)k2
i<∞. This is guaranteed by assumption A12.
ES
hkmj (β)k2
i= ES
hw2j
n1 (τ j = 0) [R (vj ;θ) 1 (yj < z)−Ht]2 + [τ j − E (τ j | vj ;θ1)]2 vTj vj + (τ j − θ0)2
oi= E
hwjn1 (τ j = 0) [R (vj ;θ) 1 (yj < z)−Ht]2 + [τ j −E (τ j | vj ;θ1)]2 vTj vj + (τ j − θ0)2
oi≤ E
hwj
n[R (vj;θ) 1 (yj < z)−Ht]2 + vTj vj + 1
oisince (τ j − θ0) , and [τ j −E (τ j | vj ;θ1)] are always included between 0 and 1.
= Ehwj [R (vj ;θ) 1 (yj < z)−Ht]2
i+E
£wjv
Tj vj
¤+E [wj ]
< ∞ by A12
Conditions 4 and 5 require the evaluation ofM (β) , the gradient of the vector of moments. Using equations (ii) and(vii), and assumption A13, it is straightforward to see that
M (β) = ∇βmj (β) =
∂mH,j(Ht,θ)∂Ht
∇θ1mH,j (Ht,θ)∂mH,j(Ht,θ)
∂θ0
0 ∇θ1mθ1,j (θ1) 0
0 0∂mθ0,j(θ0)
∂θ0
=
−wj1 (τ j = 0) wj1 (τ j = 0) 1 (yj < z)R (vj ,θ)v
Tj −wj1(τj=0)1(yj<z)E(τj |vj ;θ1)
[1−E(τj |vj ;θ1)]θ20
0 −wjE (τ j | vj ;θ1) [1−E (τ j | vj ;θ1)]vjvTj 0
0 0 −wj
(ix)
The above matrix is block-triangular, so that A13 ensures that its expectation is not singular, so that assumption 5is satisÞed. The last condition we need to verify is assumption 4. The following proves that assumptions A14 andA15 are sufficient for assumption 4 to hold:
ES
"sup�β∈N
°°°M³�β´°°°# ≤ ES
"|wj1 (τ j = 0)|+ |wj |+ sup
�θ1∈N
°°°wjE ³τ j | vj ; �θ1´ h1−E ³τ j | vj ;�θ1´ivjvTj °°°#
+ES
wj1 (yj < z) sup�θ∈N
1 (τ j = 0)E³τ j | vj ; �θ1
´h1−E
³τ j | vj ; �θ1
´i�θ2
0
+ES
wj1 (yj < z) sup�θ∈N
1 (τ j = 0)E³τ j | vj ; �θ1
´³1− �θ0
´h1− E
³τ j | vj ; �θ1
´i�θ0
kvjk
≤ 2 +E°°vjvTj °°+E sup
�θ∈N
1h1−E
³τ j | vj ;�θ1
´i�θ2
0
+1h
1−E³τ j | vj ;�θ1
´i�θ0kvjk
≤ 2 +E
°°vjvTj °°+E sup�θ∈N
"³1 + e|vj |
T |�θ1|´ 1�θ2
0
+³1 + e|vj |
T |�θ1|´ kvjk�θ0
#
≤ 2 +E°°vjvTj °°+E ·³1 + e|vj |T |D|´ 1
C2+³1 + e|vj |
T |D|´ kvjkC
¸< ∞
A9
where the last step follows by A15, and the previous step follows by A14 and by the compactness of Θ, whichguarantees the existence of the vector D. This completes the proof.
1.3.1 Consistent Estimation of the Variance
First note that, using (viii) and 1 above, the following results hold:
DHH = −ES [wj1 (τ j = 0)]
DHθ = ES
wj1 (τ j = 0) 1 (yj < z)R (vj ,θ)vj
−wj1(τj=0)1(yj<z)E(τj |vj ;θ1)[1−E(τj |vj ;θ1)]θ20
Dθθ = ES
−wjE (τ j | vj ;θ1) [1−E (τ j | vj ;θ1)]vjvTj 0
0 −wj
where all expectations can be consistently estimated using sample equivalents. Note that given the way we havewritten the moments in (vii) all sample means have to be computed using (na + nt) as the total number of obser-vations. For the estimation of the variance π we still need an estimates for E
hm (β)m (β)
Ti. The elements of the
matrix can then be consistently estimated as follows:
�EShm (β)m (β)T
i=
"�ES£m2H
¤�ES£mHmT
θ
¤�E [mHmθ ] �E
£mθm
Tθ
¤ #
=
�ES£m2H
¤�ES£mHm
Tθ1
¤�ES [mHmθ0 ]
�ES [mHmθ1 ] �ES£mθ1m
Tθ1
¤�ES [mθ0mθ1 ]
�ES [mHmθ0 ] �ES£mθ0m
Tθ1
¤�ES£m2θ0
¤
where
�ES£m2H
¤=
1
na + nt
Xj∈S(a),S(t)
w2j1 (τ j = 0)hR³vj ; �θ
´1 (yj < z)− �Ht
i2�ES£mHm
Tθ1
¤=
1
na + nt
Xj∈S(a),S(t)
w2j1 (τ j = 0)hR³vj ; �θ
´1 (yj < z)− �Ht
i hτ j −E
³τ j | vj ; �θ1
´ivTj
�ES [mHmθ0 ] =1
na + nt
Xj∈S(a),S(t)
w2j1 (τ j = 0)hR³vj ; �θ
´1 (yj < z)− �Ht
i³τ j − �θ0
´�ES£mθ1m
Tθ1
¤=
1
na + nt
Xj∈S(a),S(t)
w2j
hτ j −E
³τ j | vj ; �θ1
´i2vjv
Tj
�ES£mθ0m
Tθ1
¤=
1
na + nt
Xj∈S(a),S(t)
w2j
hτ j −E
³τ j | vj ; �θ1
´i³τ j − �θ0
´vTj
�ES£m2θ0
¤=
1
na + nt
Xj∈S(a),S(t)
w2j
³τ j − �θ0
´2Consistency of each term can be proved using arguments analogous to those used to prove consistency of �Ht.
A10