1

Benefits Transfer and Meta Analysis

Professor Anil MarkandyaDepartment of Economics and International

Development University of Bath hssam@bath.ac.uktel. +44 1225 386954

Environmental Economics 2March, 10 2006

2

Benefits Transfer (BT)

• More often than not, we do not have the time and resources necessary to design and implement primary studies such as Contingent Valuation, Travel Cost, Conjoint Analysis and Hedonic Pricing Method.

• Economists suggested to use “Benefits Transfer”: The use of existing estimates of non-market values derived in one context/location to estimate values in a different context/location.

• The idea behind BT is to ‘borrow’ an estimate of WTP from site A (the ‘study site’) and applying it to site B (the policy site’), without the need to do a new valuation exercise in site B

• BT is unreliable (Loomis et al., 1995; Downing and Ozuma, 1996; Brower, 1998; Bateman et al., 1999)

• If it was reliable, researchers wouldn’t do any new study that applies the Contingent Valuation Method, the Travel Cost Method, the Conjoint Analysis, or the Hedonic Pricing Method.

3

Step 1 Define the value(s) to be estimated at the policy site

Step 2

Conduct a literature review to identify relevant valuation data

Step 3 Assess the relevance of the study site values for

transfer to the policy site

Step 4 Assess the quality of the study site data

Step 5 Select and summarise the data available from the study

site(s)

Step 6 Transfer benefit measures from the study site(s) to the

policy site

Step 7 Determine ‘market’ over which benefit estimates are to

be aggregated

4

Step 1

• The first step is to define the value(s) to be estimated at the policy site.

This in turn requires you to identify the specific good or service affected by the policy you want to analyze, or by the environmental damage you are looking at

What type of land is lost? What type of habitat is damaged? What population is affected? What specific recreational activities will be impaired?

5

Step 2

• In the second step you will need to conduct a thorough literature review to identify valuation data relating to the specific good(s) or service(s) identified in Step 1. For example, if wetland habitat is damaged, then you will need to identify studies which value individuals’ WTP to avoid damage to wetlands.

6

Step 3

• Step three involves assessing the relevance (suitability) of the study site values for transfer to the policy site. This requires you to consider a number of criteria. The suitability of the original valuation data to the problem at hand depends primarily on how similar the study site is to the policy site with respect to:

(a) the magnitude of the environmental change; (b) the environmental good/service in question; (c) the socio-economic and cultural characteristics of the affected

population; (d) the availability of substitutes; (e) the assignment of property rights (this will dictate whether WTP or

WTA is the appropriate welfare measure to use).

7

Step 4

• After ascertaining the relevance of the study site values for transfer to the policy site, the fourth step involves assessing the quality i.e. scientific soundness and richness of information of the study site estimates.After all, the estimated values at the policy site are only as accurate as the study site values upon which they are based; measurement error implicit in the original values is compounded when applying benefit measures or valuation functions in the new situation.

8

Criteria for Evaluating the Quality of Candidates for Transfer

Scientific Soundness – the transfer estimates are only as good as the methodology and assumptions employed in the original studies

Specific criteria: Data collection procedures

Sound empirical practices

Consistency with scientific and economic theory

Appropriate and rigorous statistical methods

Relevance - the original studies should be similar and applicable to the ‘new’ context

Specific criteria: Magnitude of impacts should be similar

Baseline levels of environmental quality should be comparable

Affected good(s) or service(s) should be similar

The affected sites should also be similar, where relevant

The duration and timing of the impact should be similar

The socio-economic characteristics of the affected populations should be similar

The property rights should reside with the same party in both contexts

Richness of Information - the existing studies should provide a ‘rich’ data set and accompanying information

Specific criteria: Include full specification of the original valuation equations, including precise definitionsand units of measurement of all variables, as well as their mean values

Explanation of how substitute commodities were treated, where relevant

Data on participation rates and extent of aggregation employed

Provision of standard errors and other statistical measures of dispersion

Source: Desouvsges, Johnson and Banzhaf (1998)

9

Step 5

• The next step is to select and summarise data from the existing valuation studies for transfer. Frequently only a single relevant valuation study exists, in which case selecting a ‘best’ benefit measure to transfer presents few problems. However, when several relevant studies are available, the selection process becomes more problematic. You could still attempt to identify a ‘best study’, but this approach ignores other, potentially valuable, information contained in the studies neglected. In order to take advantage of all the information available, other approaches to transfer have been developed to utilise data from multiple studies.

10

• First, you could develop a range of parameter and benefit estimates from the available studies. For example, you could search the existing studies to identify a low estimate, which would define a lower bound for the transferred data; and likewise use a high estimate from the existing literature to define an upper bound.

• Second, you could collect data from the existing literature and develop simple descriptive statistics of model parameters and benefit estimates - e.g. mean and standard errors - and transfer these data to the policy site.

• Third, you could adjust these measures of central tendency - based on expert (subjective) judgment - and transfer the adjusted data.

• Fourth, in order to take full advantage of the information available from multiple studies, you could undertake some form of meta-analysis to develop a ‘new’ benefit model for transfer.

11

Step 6

• The sixth step involves actually transferring the benefit measures from the study site(s) to the policy site(s).

Most benefit transfer methods utilized to date involve either the benefit value or benefit function approach.

12

Benefit Value Approach

• In the most basic application of the benefit value approach, a scalar-valued ‘best’ estimate (typically, the mean or median WTP per affected unit) is used to represent the results of an existing study, or selection of existing studies, that have been conducted in a specific context. The average consumer surplus per angling trip, for example, might be taken from a travel cost study which assessed the benefits of recreational angling at a specific site. This unit value could then be used to value a change in the quality or provision of recreational angling opportunities at different locations. Specifically, the total cost/benefit of a change in recreational angling opportunities at the policy site is equal to the predicted change in the number of angling trips made to the site multiplied by the average consumer surplus per angling trip.To improve the quality of the benefit value transfer, you could make some adjustment to the scalar-valued estimate. These adjustments are typically ad hoc, and usually reflect the subjective judgment of the analyst.

13

Transferring average WTP from a single study to another site which has no study

• The estimate may be adjusted taking into account of differences in:– Socio-economic characteristics of the relevant populations– Physical characteristics of the study and policy site– Proposed change in the provision between the sites of the good

to be valued– Market conditions applying to the sites (for example, variation in

the availability of substitutes)• A widely used form for adjusted transfer is:

– WTPj=WTPi(Yj/Yi)e

Y income per capita

e income elasticity of WTP, i.e. an estimate of how the WTP for the environmental attribute in question varies with changes in income.

14

Example of BT with Different ‘e’s

WP(J) WP(I) Y(J) Y(I) e

44 100 2000 10000 0.5

20 100 2000 10000 1.0

9 100 2000 10000 1.5

4 100 2000 10000 2.0

1.8 100 2000 10000 2.5

15

• For an income elasticity of WTP that is less than 1 (the typical case) the usual practice of taking a value of one will understimate WTPj when transferring the estimate from a developed to a developing country.

• The adjusting weight generally used is income. But it should be possible to make a similar adjustment for changes in other characteristics. (E.g. population).

16

Benefit Function Approach

• With the benefit function approach, an empirical relationship (function) between WTP and characteristics of the affected population and the resource being assessed, is specified. The entire function is then transferred to the policy site and adjusted to conform as closely as possible with the population and resource characteristics at that site. The adjusted valuation function is in turn used to value changes in the quality or provision of the resource in question – data from the policy site is substituted for the right-hand-side variables in the valuation function. A travel cost demand model for angling trips estimated at the original study site, for example, may be used in conjunction with the average travel costs, income, water quality conditions, etc. at the policy site, in order to estimate the recreational angling dis/benefits of a deterioration or improvement in water quality at that site.

• Benefit function transfer may thus be seen as a way of making the ad hoc adjustments to scalar-valued estimates more systematic, since you can explicitly control for differences between the existing literature and the policy context with respect to e.g. environmental quality, site attributes, and socio-economic characteristics.

17

Even in the case of benefit function transfer - whether based on the transfer of equations from individual studies or ‘new’ models derived from meta-analytical techniques - you may still feel that the parameters in the transferred valuation model are not fully applicable to the policy context. As with the transfer of scalar-values, you may decide that adjustments to the model parameters are required.

18

Step 7• The final step is to determine the so-called ‘market’ over which

impacts at the policy site are aggregated in order to obtain a measure of total cost/benefit. Three interdependent issues that must be considered during the aggregation task:

1) The geographical extent of the affected ‘market’. In some cases this may be defined by geographical (e.g. river catchments) or political boundaries (e.g. counties), it may also be based on the extent of the predicted physical impacts (e.g. the area at risk to increased flooding or coastal erosion).Another possibility is to define the geographical boundaries of the costing analysis as the point where individuals’ WTP in respect of the affected good/service decays to zero. Some empirical evidence is emerging that indicates a negative relationship between WTP for the services provided by a resource and distance to that resource

19

2) Related to the geographical boundaries of the transfer, is the number of affected units - e.g. households, building types, varieties of agricultural products, etc. - within the geographical ‘market’. In some cases the identification of the affected population will be obvious - e.g. all buildings in a floodplain or all households receiving potable water from a particular water supply company. In other cases however, it may be necessary to identify participation rates for certain affected sub-groups - e.g. the number of day trips expected to an affected recreation site per year.

3) In determining the total cost/benefit of a policy or of a variation in the quality of an area it is also important to take possible substitute goods/services into account. Other things being equal, a good or service will have a higher value, the fewer alternatives that are available. Adjusting for the availability of substitutes is particularly relevant in the context of impacts on services provided by natural resources - e.g. recreation activities.

20

Example of Benefit Functions

• You want to see value of improvement in WQ at policy site.

• Benefit Functions are:

• Where N is no of visitors, Q is measure of water quality, Y is income and TC is cost of travel

)/1( 00 QQNN iQ

iii dTCcQbYaWTP

21

Example of Benefit Functions

• Steps in making the calculation are:– Collect data on present visitor profile at site,

including income and travel cost– Estimate change in number of visitors due to

quality improvements from equation (1)– Ascertain profile of additional visitors (how?)– Estimate new WTP of all visitors.

22

How Good Are Benefit Transfers?

• There are two general sources of error in the estimated values: (1) errors associated with estimating the original WTP/WTA measure at the study site(s)(2) errors arising from the transfer of these study site values to the policy site.

Concerning the latter, McConnell (1992) identified five key sources of such error:

1) Choosing the wrong functional form for the value function.2) Omitting important explanatory variables from the value function.3) Measuring the independent variables incorrectly (e.g. income, the

change in water quality).4) Measuring the dependent variable incorrectly.5) Incorrectly specifying the random process that generates the data

(e.g. truncating the number of trips made in a TC model).

23

McConnell also identified key sources of error in calculating the total WTP/WTA at the policy site, including:

1) Incorrect handling of the random components of the value function.2) Aggregation errors in calculating the “group” means, where required,

for the independent variables.3) Errors in calculating the population over which to aggregate

individual estimates of WTP/WTA.4) Errors in calculating the extent of the market for the affected

environmental service at the policy site.5) Clearly, there are multiple sources of error in transferring benefits,

and consequently, care must be taken when undertaking benefit transfer. If however the data quality/suitability checks listed above (in particular, during Steps 3 and 4) are fully adhered to, then these potential sources of error can, to some extent, be limited. Nonetheless, as with all types of decision-support tools, transfer studies are most useful to the end-user when sources of uncertainty are identified and, where possible, quantified.

24

Meta Analysis• Taking the results from a number of studies and analyse them in

such a way that the variations in WTP found in the studies can be explained.

25

• Results from these analyses suggest that differences in study design play an important role in explaining the variance between valuation outcomes.

• Differences in CV outcomes have been analysed in great detail in individual studies by looking at the effect of varying single research design elements on stated preferences. Significant differences have been found in valuation outcomes in individual studies with different survey design.

26

Example: Brower et al., 1999

27

Meta analysis of Cultural Resources, Noonan (2003)

28

29