COMPARATIVE POLITICS

THE COMPARATIVE MANIFESTO PROJECT

Giorgio Sirtori Prof. Marco Giuliani

CMP’S OBJECTIVE

To measure policy positions of all relevant parties competing in any democratic election in post WWII period for OECD and EU members, Central and Eastern Europe and (in the future) Latin America and South-East Asia

MANIFESTO (ELECTION PROGRAM)

→ the text published by a political party in order to compete for votes in national election

Why manifesto?

Parties’ only authoritative policy statements

Indicators of the parties’ policy preferences at a given point in time

TWO-STEP PROCESS

1. Unitising − cutting text in quasi-sentence

The coding unit is a quasi-sentence, that contains exactly one statement

e.g.: ″We need to address our close ties with our neighbours (107) / as well as the unique challenges facing small business owner in this of economic hardship (402)″

.

2. Coding − find the right code for a quasi- sentence Attribute to each coding-unit one, and only one, category

CMP developed a category system composed of 56 categories, grouped in 7 policy areas, designed to be comparable between parties, countries, elections and across time

.

Manifesto data can be used, and have been used, to provide valid and reliable measurements of party policy position

e.g.: Left-right scale estimates from Mapping policy preferences (Budge et al. 2001)

CMP LEFT-RIGHT SCALE

CMP LEFT-RIGHT MEASURE

Left-right sore = proportion(right - left) %

(R – L)/(R + L) %

Scale ranges from -100 to +100

CMP left-right scale positions for British parties (1945-1997)

COMPARATIVE POLITICS

SHORTCOMINGS OF LOCATING POLITICAL PARTIES USING PARTY MANIFESTO DATA

Irina Dámátár Prof. Marco Giuliani

ESTIMATING LEFT-RIGHT POSITIONS OF POLITICAL PARTIES

The aim of the Comparative Manifestos Project is to place parties on the left-right political spectrum, parting from the information provided within their electoral programmes

Each party’s position is measured as the difference (in percentages) between the right-associated text mentions and left-associated ones; the final scale ranges from -100 to 100

The identity of left and right manifesto coding categories was determined using a series of within-country exploratory factor analysis of a wide range of coding categories

THE COMPONENTS OF THE CMP LEFT-RIGHT SCALE

SHORTCOMINGS No indication of the uncertainty associated with

any CMP estimate - when placing a policy in a certain point on the left-right

dimension, no associated error is indicated

CMP left-right measure consists of pre-defined and fixed scale components

- the constituent elements of the left-right scale are defined in the same way for all countries, at all time periods

but the meaning of left and right can vary in time and

space (e.g.: market regulation is a left issue in the CMP dataset, while in Germany is a valance issue)

- CMP measure may not include relevant variables, that can explain much of the parties’ variation, while considering unrelated ones

The CMP scale combines party positions with party specific measures of the relative salience of the left-right scale, producing a measure that is affected by non-right-left issues mentioned in the party manifesto

- positions of the parties are in part determined by coding categories that are not in the CMP scale

The scores which CMP interpret as positional data may indicate in fact the willingness of a party to move in accordance to changing political circumstances rather than their actual position

- hence, the measurement made by CMP and the score attribute to a certain party lacks precision

. CMP approach identifies position on the basis of

saliency scores; but this can be seen more as a movement of position towards the electorate rather than a party’s true position

- to eliminate this shortcoming, Franzmann & Kaiser

stress upon the distinction between

position issues valence issues matters upon which the electorate’s matters evaluated either

negatively opinions are clearly divided or positively by all voters

- only position issues can structure the policy space -hence, they classify items based on their position or

valance character

.

Using factor analysis to compute party positions leaded to highly difficult interpretations when the number of variables out weighted the number of cases (low measure of sampling adequacy)

Addressing this shortcoming by focusing on

the economic “super-issue” as equivalent to the left-right scale disregards the fact that left and right are not perceived only in economic terms

Nevertheless, in its second stage CMP followed a two step approach: issues were coded as left or right, and the resulting composition was examined through factor analysis -> to see whether these issues did indeed fit in the assumed dimension

- factor analysis is also problematic because aims at a reduction of dimensions

EXPLAINING DIFFERENCES BETWEEN CMP SCALE POSITIONS AND EXPERT

SURVEYS JUDGEMENTS

Computing the position of a party using an expert survey estimate may lead to different results than the ones of the CMP

Why? Although there is no pattern to explain these differences,

there are a number of hypothesis that can be considered:

I-st possibility: party policies change between the time of the election for which CMP scored a party manifesto, and the time when the expert survey was conducted

.

II-nd possibility: one of the two measurements contains significant error

III-rd possibility: measuring different quantities - party manifestos may have policy preferences

not expressed in their election programmes and which hence cannot be captured in the scores provides by CMP

- no single pre-defined scale will accurately characterize the left right dimension of politics in all countries

- “new” dimensions of politics, such as immigration and environment, whose omission from the measurement can cause inaccuracies

(e.g.: until 1980s, in Germany, environmental protection was a valance issue or at least was not ideologically structuring the policy space)

PLEASE NOTE THAT NO SHEEP WERE HARMED IN THE MAKING OF THIS

PRESENTATION

COMPARATIVE POLITICS

EXPERT SURVEYSPARTY POLICY IN

MODERNDEMOCRACIES

Kenneth BenoitMichael Laver

Silvia Merisio Prof. Marco Giuliani

ANALYSING PARTY COMPETITION

Why voters choose to support specific political parties?

Why parties do what they do once the election is over?

Public needsand wants

Policy adopted by political parties

NEED TO ESTIMATE PARTY POLICY POSITIONS

Problem of gathering reliable information

Election Manifestos

How parties behave(e. g. Roll call votes)

Surveying party politiciansor party supporters

Not systematic analysis

Extremely expensive and difficult across countriesExpert

surveys

EXPERT SURVEYS

“Expert” most comprehensive population of country specialists

These experts are asked to locate party policy positions, in the party systems of which they have expert knowledge

On a set of predefined policy dimensions

POLICY DIMENSIONS

Economic Policy Social Policy European integration General policy (environment, immigration, etc.) Specific dimensions for Post-communist countries

(e.g. privatization, religion, etc.) Subjective sympathies/closeness to party General Left-Right dimension

All dimensions are evaluated on a scale ranging from 1 to 20

EXPERT SURVEYS

They provide information on party positions

Wide range of countriesCommon format

Expert surveys are a systematic way to summarize the judgments of

the consensus of experts on the matters at issue

RISKS

The same concept/ dimension can be interpreted in different ways

How to assess the content of validity

Importance of broadly equivalent

dimensionse.g. economic

policyReasonableness of

numbers and comparison of the result with other

method

THE SURVEY

Reference to a previous work by Laver and Hunt (1992)

Survey conducted in 1988-89 24 countriesCore dimensions on economic and social policies

Benoit and Laver (2006)Survey conducted in 200347 countries (all European countries from the former Soviet

Bloc)38 policy dimensions (including specific dimensions for each

country)

Real continuity between the two works

THE SURVEY (2)

Inclusion of countries classified as non-fully democracies by Freedom House;

e.g. Belarus, Russia, Albania, Moldova, Ukraine

Translation in 24 languages; Web or paper format.

SURVEY DETAILS FROM EASTERN EUROPE COUNTRIES

POLICY DIMENSIONS FROM THE EXPERT SURVEYS (NUMBERS PROVIDE THE TOTAL EXPERT RESPONSES ON EACH DIMENSION)

SURVEY DETAILS FROM ITALY

SURVEY DETAILS FROM LITHUANIA

CONCLUSIONS High response rate in all countries

Wide sample of countries which had never been investigated before

Flexible model

Possibility to use expert surveys for different aims and topics

COMPARATIVE POLITICS

SHORTCOMINGS OF EXPERT

SURVEYS

Adelaida Foitik Prof. Marco Giuliani

EXPERT SURVEYS – ADVANTAGES -> POPULARITY

economical way of measuring party positions provide information on party policy positions in a common and

standardised format   can be administered at any time, unlike manifestoes-tied to

electoral calendars as long as experts are willing to respond to surveys, the expert

survey methodology may probe topics that do not surface in manifestos or other data sources e.g. internal dissent within a party

quick and easy compared to other methods(content analysis of party electoral programs or legislative behavioural studies)

they reflect the judgements of experts=> weight and legitimacy

BUT how valid is the information?? ANALYZE: what, how & when

 

THE VALIDITY OF EXPERT JUDGMENTS

Framing survey questions: What is the basis of the judgments that experts offer? Do experts answer questions in the way they were intended? -> Budge (2000): 4types of problems what ‘party’ is being judged by the expert? - Is it the party in the

electorate, the party in government or the party organization? what criteria do experts bring to bear when they judge party

positions? - what do abstract labels like ‘left’ and ‘right’ mean to the expert?

do experts judge the intentions of parties or their behavior? what is the time frame for the judgments that we ask experts to

make?  ->Fundamental question: how do experts interpret the questions in

expert surveys and how do they link substantive knowledge about parties to those questions?

REMARK

Wildly varying considerations when judging party positions

Þ misleading expert judgments

Þ Solution: expert survey design- good questionnaire will attempt to:

avoid ambiguous terms such as ‘party’ and ‘left-right’

give them a more circumscribed meaning indicating precisely what judgment should concern

specify time frames explicitly to limit variation on this dimension

CONT’D

e.g.: experts may be asked to judge the position of the party leadership on issue X during the past year

e.g. in evaluating the left-right position of parties, experts may be asked to describe what ‘left-right’ means in a particular country or what criterion they used to define this dimension

HOWEVER, better expert survey design might not alleviate

all of the validity concerns raised by Budge: problems like telescoping (recalling facts for the wrong time period

before/after they happened or afterwards) are bound to plague such judgments, even when we provide an explicit time frame

Even the most carefully crafted question may still leave an interpretative space for experts that could distort their judgments.

 

I. THE 1999 EXPERT SURVEY OF NATIONAL PARTY POSITIONS ON EU

INTEGRATION

->evaluated by Budge, developed by Ray (1999), objective: to assess the positions of national political parties vis-à-vis European integration, run 1996 - 1999 , 15 EU Member States

NOTE: in order to minimize response variation due to differential scale interpretations by experts: the question specified

the object that was to be evaluated (the party leadership) the time frame for the evaluation (1999) response options were explicitly labeled => the question was designed to put the experts in a common

frame of mind so that they would be judging the same object, on the same dimension, at the same point in time

CONCLUSION OF THIS SURVEY

=> encouraging findings- the correlations among experts are on average very high, both within particular countries and across the whole set

-> substantial convergence in the judgment criteria that experts use: remarkable consistency in responses, agreement in expert placements of political parties with other measures.

=> considerable confidence in the expert data 

II. BENOIT AND LAVER (2006)- instead of using experts’ answers to locate parties within a given

policy space, use them to directly map the ideal points of the respondents

Focus: the Italian case-rely on two expert surveys conducted with the same methodology in a short time span (2003 ,2006)=> possible to check for the robustness of our results

- locate each party on the L-R dimension- all aspects of party policy- policy preferences of parties- 4 substantive policy dimension

(economic policy, social policy, the decentralization of decision making and environmental policy )

- “sympathy scale” asked experts to place all parties on a scale indicating their own closeness to each party’s-> to test for any possible respondent bias by checking whether expert placements of parties on substantive dimensions were correlated with their personal sympathy for a party’s policies

EXPERT PREFERENCES AND BIAS: AN ANALYSIS

- party positions estimated on the basis of survey data are not always reliable -respondents tend to place the parties they like closer to where they locate themselves( = rationalization problem)

=>This is a point well recognized in BL: “the classic problem of a sample bias is not a concern [in expert surveys], even if the experts we consult hold strong political preferences, as long as these preferences do not interfere with their expert knowledge”

RESULTS:=> on a pool of 158 observations (i.e., the number of parties surveyed), 20

biased parties (12.7%) were found. NOTE: almost two-third (65%) of the biased parties are extreme-right ones. HOWEVER: not every extreme-right party is biased (just the 59%) AND not every biased-party is indeed an extreme-right one (one-third of the total)

REMARKS

the consequent level of bias (1.2 out of 10parties), may have nothing to do with the “true” characteristics of the different communities of political experts because information drawn from individuals is quite vulnerable to selection bias

the selection bias is often due to low response rates. Instead of trying to increase the rate of response in the hope of decreasing the level of bias (if a selection bias problem is indeed present), we should attempt to give a more circumscribed policy-meaning to the questions administered, avoiding as far as possible ambiguous terms such as, “Left-Right”

CONCLUSION OF THIS SURVEY a rationalization problem is indeed present: this bias is

relevant because it can jeopardize the usefulness of expert scores for empirical analysis

the ambiguity of a label as Left-Right locations => expert judgments become unreliable

evidence of bias in the expert placements of parties especially against right-wing parties (not necessarily extreme-right ones)

the Left-Right position of a party is less important than the policy preferences of experts (and their variance) to explain the probability of a party being biased

HOWEVER, the bias is less pronounced when we pass from a label as “Left-Right” to less abstract policy dimensions 

CONCLUDING REMARKS Reliance on expert judgments is an attractive option for measuring

complex phenomena such as party positions about policies: they are comparatively straightforward to conduct, but we need to make sure about their validity.

As suggested by Budge’s concerns, those who use expert surveys should be cautious.

We have investigated the shortcomings of the expert survey with particular attention to 2 specific cases: while in the first one the results appear to be encouraging revealing the remarkable agreement among experts about the placement of parties, in the second one there is evidence of bias in the expert placements of parties along the Left-Right dimension

Validity assessment is an exercise that should be a central part of expert survey

 

THANK YOU FOR YOUR KIND ATTENTION