“Social” Multicriteria Evaluation: Methodological

Foundations and Operational Consequences

Giuseppe MundaUniversitat Autonoma de Barcelona

Dept. of Economics and Economic HistoryEd. B

08193 Bellaterra (Barcelona)Spain

e_mail: giuseppe.munda@uab.es

Structure of the talk

•Why Social Multicriteria Evaluation (SMCE)?

•How such an approach should be developed?

•Conclusions

Complexity is an inherent property

of natural and social systems

addressed

ignored

COMPLEXITY

COMPLEX SYSTEMS

CANNOT BE CAPTURED

BY A SINGLE

DIMENTION/PERSPECTIVE

Complexity: the ontological dimension

the existence of different levels and scales at which a hierarchical system can be analyzed implies the unavoidable existence of non-equivalent descriptions of it

a. b.

c. d.

Orientation of the coastal line of Maine

Complexity: the epistemological dimension

EMERGENT COMPLEXITY

Different dimensions

Different values and perspectives

hard and topologies soft

"The issue is not whether it is only the marketplace

that can determine value, for economists have long

debated other means of valuation; our concern is with the assumption

that in any dialogue, all valuations or "numeraires"

should be reducible to a single one-dimension standard".

(Funtowicz and Ravetz, 1994, p. 198)

AcademicScience

ProfessionalConsultancy

Post NormalScience

Post NormalScience

Uncertainty

DecisionStakes

facts are uncertainvalues in disputestakes highdecisions urgent

facts are uncertainvalues in disputestakes highdecisions urgent

S. Funtowicz, J. Ravetz

Strong comparability

Weak commensurability

Strong commensurability

Weak comparability

incommensurability

•TECHNICAL INCOMMENSURABILITY

•SOCIAL INCOMMENSURABILITY

NAIADE 2 matrices

Multi-, inter-, trans-disciplinarity?

• Multi-: each expert takes his part

• Inter-: methodological choices are discussed across the disciplines– Informing the others about object matter

– Criticism, reflexivity

• Trans-: What is it? ....

MEASURES FOR DEMAND REDUCTION

Metereological drought

Water demands

Socio-economic system

Water supply system

Natural water bodies

Hydrological drought

Water shortage

Economical losses and intangible impacts

MEASURES FOR SUPPLY INCREASE OR DEFICIT RISK REDUCTION

MEASURES FOR DROUGHT IMPACT REDUCTION

Consequences: 1) MULTIDISCIPLINARITY

ROSAMARINA SCANZANO POMAPIANA DEGLI ALBANESI

Demand Zone

Reservoir

Well

Spring Pump Station

Waste Water TreatmentPlant

CT

ME

SR

RG

EN

CLAG

PATP

030'

38

0

0

0

2040

6080

100

km

PALERMO

Western Palermo

Diversion Supply conduit

Purification Plant

MunicipalitiesMunicipalities

Eastern Palermo

Hydropower Plant

RISALAIMI GABRIELE

SCILLATO

Palermo

Irrigation DistrictACQUA DEI Irrigation DistrictPartinico

MONTE TESORO

Rosamarina

RISALAIMI GABRIELE

CICALA

Infiltration Gallery

PRESIDIANA

Irrigation District

Termini

CariniIndustrial District

S.Leonardo R. Eleuterio R. Belice R.Iato R.

Industrial District

Termini

Irrigation DistrictScillato

CORSARI

S.CATERINA

Trabia Aquifer Palermo Aquifer

Municipality

Reservoir Reservoir Reservoir Reservoir

IMERA

MunicipalityMisilmeri

Consequence: 2) PARTICIPATIVE TECHNIQUES

• In-Depth Interviews

• Focus Groups

• Questionnaires

• Institutional Analysis

Application of a MulticriteriaAggregation Procedure

Application of an specific ConflictAnalysis Procedure

Construction of an Actors’ Impact Matrix

Construction of aCriterion Impact Matrix

Choice of set ofEvaluation Criteria

Generation ofAlternative Options

Identificationof the Main Actors

Formulation of the ExplanationHypothesis

Inte

rpre

tati

on o

f t

he

Res

ult

s

InstitutionalAnalysis

MC

DA

VALSE: Structure of the Troina Case Study

Objectives and Methodology of DIAFANIS

1. Why a conflict exists?

2. Which alternatives exist?

3. Which system dimensions can be affected?

4. How alternatives can be evaluated?

5. What means transparency?

Step 1: Evaluation of alternatives

1. Alternatives Generation

Historicalanalysis A1 A2 An

Alternatives

CitizenParticipa

tionInstitution.analysis

Step 2: Diffusion of results

1. Existence of multiple values

2. School visits

3. Citizens meetings 4. International Symposium

Technical andSocial Rankings

3. MCE Algorithm CriteriaSelection

Alternatives Evaluation

CitizenParticipation Mixed Information

Data Collection and Participation

Economical SocialEnvironmental

2. Information Structuring

System Dimensions and Hierarchical Scales

International, National, Regional, Local

Consequences: 3) ETHICS MATTERS

Economic dimension

Social dimension

Environ. dimension

Economic objectives

Social objectives

Environ. objectives

Economic criteria

Social criteria

Environ. criteria

SOCIETY

Weights in a social framework

Political Democracy

Economic Democracy

Sustainability

Precautionary Principle

Consequence: 4)THE AXIOMATIZATION ISSUE

K. Arrow, H. Raynaud (1986): “Social choice and

multicriterion decision making”

Desirable Properties for SMCE

Aggregation Conventions

The idea of social incommensurability implies:

• Multicriteria methods must be as simple as possible to guarantee transparency.

• Weights in this framework are clearly meaningful only as importance coefficients and not as trade-off. As a consequence, complete compensability cannot be implemented.

• Sensitivity and robustness analysis have to check the consequences on the final ranking of only some clear ethical positions and not of all the possible combinations of weights.

• Conflict analysis procedures explicitly looking for social compromises should integrate a SMCE exercise.

• In a policy framework, to have a ranking of all the alternatives is more useful than just to select one alternative only; this implies that dominated alternatives cannot be excluded a priori.

From the idea of technical incommensurability:

• Partial or complete non-compensability is an essential consistency requirement.

• Indifference and preference thresholds should be explicitly taken into account.

• Mixed information of the widest type should be addressed in a consistent way.

• Simplicity, meaning the use of as less parameters as possible, is a very desirable property.

• The hierarchical dimension of a policy problem should be explicitly considered.

Eff.

Altern. Compensab. Weights as

import.coeff. Mix. inf.

MAUT --- --- --- + ELECTRE 2 +++ +++ ++ + ELECTRE 3 +++ +++ ++ + REGIME (H,N,R,1983)

+++ +++ +++ +

REGIME (H,N, 1990)

+++ + - +++

NAIADE +++ ++ --- +++ AHP +++ - --- --- EVAMIX +++ + --- +++ PROMETHEE +++ + + + Martel & Zaras method

+++ +++ + +++

Simpl. Hier. prob.

From. Ind./pref. thresh.

Conf. Anal.

MAUT +++ --- --- --- --- ELECTRE 2 -- --- +++ ++ --- ELECTRE 3 --- --- +++ ++ --- REGIME (H,N,R,1983)

+++ --- ++ --- ---

REGIME (H,N, 1990)

+ --- ++ --- ---

NAIADE --- --- +++ ++ +++ AHP -- +++ ++ --- --- EVAMIX ++ --- ++ --- --- PROMETHEE -- --- +++ +++ --- Martel & Zaras method

--- --- +++ ++ ---

Table 1. Example of evaluation of some multicriteria methods according to proposed desirable properties for SMCE

Is SMCE relevant for the study of Sustainability?

Yang: ECONOMICS

GDP

Yin: ECOLOGY

QUALITY OF PRODUCT

PROCEDURAL RATIONALITY

LEARNING HOLARCHIES

QUALITY OF “SOCIAL” PROCESS

PARTICIPATION

TRANSPARENCY

MULTI/INTER-DISCIPLINARITY

ETHICSRESPONSIBILITY

CONSISTENCY

Social Multicriteria Evaluation

• MCDM (technocratic)• MCDA (technocratic)• non-algorithmic MCE

(loss of the algorithmic component)

• Participative MCE (loss of the algorithmic component)

•Social MCE •(how to integrate mathematical tools with social processes)

MCDMMCDA

MCDMMCDAPMCE

SMCESMCE