1

Fuzzy Logic or Fuzzy Application?

A Replic to Stockemer’s “Fuzzy Set or Fuzzy Logic? Comparing the Value of

Qualitative Comparative Analysis (fsQCA) versus Regression Analysis for the

Study of Women’s Legislative Representation”

Paper to presented at the 4th

European Conference on Politics and Gender (ECPG)

June 11th

– 13th

2015

Uppsala (Sweden)

Antje Buche (University Erlangen-Nuremberg; antje.buche@fau.de)

Jonas Buche (Goethe University Frankfurt; buche@soz.uni-frankfurt.de)

Markus B. Siewert (Goethe University Frankfurt; siewert@soz.uni-frankfurt.de)

**First draft – please, do not quote or cite without permission by the authors. Comments and

remarks are very welcome.**

Abstract (120 words)

In a recent article in European Political Science (EPS), Daniel Stockemer characterizes fuzzy-

set Qualitative Comparative Analysis (fsQCA) as a “poor methodological choice” because of

its “suboptimal nature” for the study of descriptive women representation in national

assemblies across the globe. However, we demonstrate that his judgments are based on two

"mis-es”: first, a misunderstanding of set-theoretical thinking in general, and specifically

QCA; and second, a misinformed application through-out various steps of the fsQCA, e.g. the

calibration process, the analysis of necessary and sufficient conditions, and the interpretation

of the results. We therefore argue – in contrast to Stockemer – that fsQCA is a valuable tool

for the comparative study of social phenomena providing a fundamentally different analytical

perspective from standard quantitative techniques.

2

1. Introduction

The descriptive representation of women in legislatures is at the core of today’s comparative

political science research (for an overview see e.g. Paxton and Hughes 2014; Krook and

Childs 2010; Wängnerud 2009). While women constitute even more than half of the worlds’

population the global average of female parliamentarians is only around 22.1% (date: April 1st

2015; IPU 2015). A closer look reveals two further aspects: First, the variation across the

globe is immense. While some countries display (nearly) equal gender representation in their

national legislatures, others have only a few or even none female representatives. Second,

parliaments with a high share of women representation are not restricted to countries which

are classically associated with gender equality like the Nordic countries; others as diverse as

Ruanda, Bolivia, Cuba, Seychelles, Ecuador, and South Africa have joined Sweden or Finland

at the top of the rankings. On the other hand, there are several prominent OECD and EU

countries like the United States of America, Japan, Ireland, Malta, or Chile that exhibit a low

share of women in their national parliaments (IPU 2015).

Explicating this cross-national variation in women representation existing scholarship has

identified mainly three sets of factors having a positive or negative effect on the share of

women in legislatures: First, institutional approaches focus on the design of the electoral

system. According to some studies proportional systems and multi-member districts further

equal representation of women while majoritarian electoral systems and single-member

districts seem to have a negative effect (e.g. Rosen 2012; McAllister and Studlar 2002; in

opposition Roberts et al. 2013). Additionally, statutory or party-based gender quotas have

been identified as effective drivers of higher women representation (e.g. Krook 2009; Tripp

and Kang 2008; Dahlerup 2006). A second strand of research emphasizes socio-economic

conditions; from this perspective higher proportions of women in parliaments are strongly

interlinked with equal access to education systems and emancipation at the workplace (e.g.

Viterna et al. 2008; Rosenbluth et al. 2006; Matland 1998). Third, cultural approaches refer to

the importance of models of femininity, gender roles and religious norms within societies as

explanations for descriptive political representation (e.g. Paxton and Kunovich 2003;

Inglehart and Norris 2001). While these bundles of explanatory factors affecting the share of

women in national parliaments are well defined their effects are still largely contested.

Therefore, newer studies have concentrated on the interplay of these factors and varying

patterns across different contexts such as status of development or geographical location (e.g.

Stockemer 2014, 2013; Rosen 2012; Ruedin 2012; Krook 2010).

3

This newer line of research has also led to methodical innovations: For instance, Mona Lena

Krook’s study (2010) of women representation in sub-Saharan Africa and Western Europe

introduced Qualitative Comparative Analysis (QCA; see Ragin 2008; Rihoux and Ragin

2009; Schneider and Wagemann 2012) to the field of gender politics in general, and women

representation specifically. At first sight, QCA seems to fit nicely with the above mentioned

trends in research on descriptive women representation for several reasons: It strives for a

more case-oriented, and therefore, context-specific assessment; it looks for combinations of

conditions and not so much on the average effects of single variables; it focuses on equifinal

configurations that explain various sets of cases; and it introduces the notion of asymmetry

which opens the view for different explanations for high and low shares of women

representation.

In stark contrast to these arguments, Daniel Stockemer (2013) in a recent article comes to the

conclusion “that QCA as a method is a poor methodological choice for explaining the factors,

or combinations of variables, that lead to high or low women’s representation in national

assemblies” (Stockemer 2013: 95) which is unable “to provide any additional insights”

(ibid: 96). He arrives at this statement through the comparison of a fuzzy-set QCA (fsQCA)

and a standard OLS regression model to the same data set of 54 Latin American and Asian-

Pacific countries. In his study, Stockemer applies the two methods separately and sequentially

in order to reveal “whether they yield different results and, if so, which method is more

appropriate for the study of women’s parliamentary representation?” (ibid: 87). In doing so a

range of seven institutional and socio-economic explanatory factors – democratic or

communist status, electoral systems, gender quotas and women suffrage, female economic

activity rate and country’s level of education – are included in the analysis of high and low

levels of women representation in national legislatures.

Somewhat puzzled by Stockemer’s harsh and absolutizing conclusions we argue that his

outright dismissal of fsQCA as a valuable tool for the study of women representation is

largely based on two “mis-es”: a misunderstanding of the Boolean and set-theoretic roots of

QCA and a misinformed application of fsQCA. We tackle both “mis-es” building our

argumentation on two pillars: on the one hand, we engage in a methodological discussion

explicating the core differences between QCA having its origins in a qualitative-empirical,

case-oriented paradigm and standard regression analysis stemming from a quantitative,

variable-oriented research tradition (Ragin 1987, 2000, 2008; Schneider and Wagemann

2012; Goertz and Mahoney 2012; Thiem et al. 2015). On the other hand, we highlight in three

4

steps some major shortcomings of Stockemer’s statements by debating specific elements of

his research design like case selection and operationalization, via replicating his fsQCA and

OLS regression, and through assessing his interpretation of the research findings.

The paper is structured along these two “mis-es”: In the first section we elaborate on the core

differences between the OLS and QCA arguing that both methods are endemic to two

fundamentally different research cultures which cannot – and should not – be evaluated

against each other. The second section discusses the research design and application of

Stockemer’s study outlining problems and alternatives to avoid some of pitfalls related to both

methods. The paper concludes with a short summary of the main arguments.

2. Misunderstanding QCA

The central aim of Stockemer’s study is to assess the value of fsQCA and standard OLS

regression for the question of descriptive women representation. Acknowledging that these

two approaches are based on completely different logics he asks “whether they yield different

results and, if so, which method is more appropriate for the study of women’s parliamentary

representation?” (Stockemer 2013: 87). For the comparison, Stockemer formulates as his

yardstick for evaluating the analytical power of QCA that “[i]f fsQCA helps identify

parsimonious combinations of factors that explain high women’s representation, then this

method is a powerful tool. In contrast, if we find few unique combinations of indicators and

little empirical coverage of these factor combinations, then this method has some serious

limitations when applied to the study of women’s representation” (ibid: 87f).

From our perspective, both the overall goal and the benchmark of Stockemer’s study are

based on two major misunderstandings and therefore are to no avail. On the one hand, the

rationales and technicalities of QCA and standard regressions are so fundamentally different

that it is neither valuable nor feasible to compare results produced by the two approaches

against each other. On the other hand, parsimony is not nor should be central to any QCA.

Quite contrary researchers applying QCA are usually interested in complex and multiple

configurations of explanatory factors.1

The logic behind QCA is deeply linked to Boolean algebra and set-theoretical thinking (Ragin

1987, 2000, 2008; Schneider and Wagemann 2012; Goertz and Mahoney 2012). From these

1 Nevertheless Stockemer has a point here, in the sense that QCA should strive for some form of generalization

across cases through minimization. If single solution terms are so complex that they still include all

explanatory factors this is not achieved. However, we will see below there are many reasons for this like

quality, calibration, or skewedness of the data, and is clearly not per se a problem inherent to QCA.

5

basic principles follow some major ramifications which distinguish QCA from standard

quantitative techniques like regressions:

First of all the central units of QCA and regression analysis are different. While the former

makes use of variables, the later utilizes sets as building blocks for the analysis. This

difference is far from being superficial: A variable stands for latent concept which is

measured directly by one or several indicators. Here the underlying assumption typically is

that the value of the indicator reflects the latent concept – aka the variable – in a linear way.

On the other side, the linkage between sets and concepts is more about assessing the meaning

of a concept. This regularly goes hand in hand with an asymmetric transformation of the data

in order to capture the semantic meaning of the concept – which is called set calibration. Put

differently, through the construction of sets the researcher interprets the data, truncates

unnecessary variation across an indicator, and through the process of assigning set

memberships attaches conceptual meaning to the set value. In the case of women

representation, variable capture the relative share of female parliamentarians while sets are

calibrated in a way to assess high, low, or medium levels of women representation (Ragin

2008: 71-84; Goertz and Mahoney 2012: 139-149; Schneider and Wagemann 2012: 24-31;

Thiem et al. 2015: 8).

Secondly, once sets are calibrated QCA enables the researcher to examine subset-superset-

relations, or in other words, checking for necessity and sufficiency. Looking for necessity

involves the analysis of shared antecedents of a given outcome. In order to be necessary a

condition has to be present if the outcome is present; thus the outcome cannot occur without

the antecedent.2 Sufficiency, on the other side, aims at the analysis of shared outcomes. So the

focus is on the question is the outcome present if a certain (combination of) condition(s) is

present? From this short depiction it should become clear that hypotheses based on necessary

and sufficient conditions are different from those based on standard regressions.3 While the

latter aim at correlational relationships which are symmetric – e.g. the higher or lower the

share of female workforce the higher or lower the proportion of women in legislatures – set

relations, by nature, are asymmetric (Schneider and Wagemann 2012: 83ff; Goertz and

Mahoney 2012: 16-38; Thiem et al. 2015: 11ff).

2 Stockemer defines a necessary condition as “one in which an outcome must be present whenever the initial

condition is present” (2013: 88). This is plainly wrong. It is rather an almost sufficient definition for a

sufficient condition. 3 However, there are approaches to mimic necessity and sufficiency analysis with regressions; see e.g.

Braumoeller and Goertz 2000, 2003; Clark et al 2006.

6

Thirdly, researchers trained in set-theory think in terms of configurations, equifinality, and

asymmetry. Again, this offers a completely different perspective from standard regression

analysis of social science phenomena. In contrast to quantitative techniques QCA does not

focus on the strength and significance of net-effects of single variables. Cases are rather

perceived as configurations of conditions. For instance, a high share of women representation

is explained by the simultaneous presence or absence of certain conditions. Thus, one is

interested in the concurrence of several explanantes that are associated with a given

explanandum.4 Contrary to standard regressions which usually focus on the one model that

fits best, QCA aims at detecting equifinal patterns uncovering multiple configurations of

conditions that explain an outcome like high share of female parliamentarians. Asymmetry

refers to the fact that configurations of conditions contributing to a high proportion of female

representatives can be different from such configurations that explain a low share of women

in legislatures. Again, this is dissimilar to standard regression approaches which assume a

symmetric relationship in the sense if a higher share of women with a high educational degree

leads to a higher share of women in legislatures then a lower level of women with high

education should lead to a lower proportion of female parliamentarians (Schneider and

Wagemann 2012: 5ff, 76-90).

This abbreviated juxtaposition shows that set-theoretic approaches like QCA and standard

regressions like OLS start out from dissimilar assumptions, are embedded in opposite logics,

and apply different procedures. If we accept this the next questions are how is it possible to

compare the results of these two approaches, and how should we assess the value of QCA and

regressions for the study of women representation? The short answer to these questions: we

cannot compare them and we cannot assess them against each other. If two methods use

different building blocks in their analysis, aim at answering different hypotheses, and rivet on

different aspects of the social world, it is not at all surprising that their results also differ. It is

rather the task of the researcher to use the different insights gained through different

analytical perspectives and methodological choices in a constructive way to shed light on

social science problems.

4 It is important to stress that configurations of conditions are not the same as interaction terms. Interactions in

regressions follow an additive or multiplicative logic based on linear algebra. Configurations in QCA, on the

other side, are based on Boolean algebra and therefore are formed applying minimum or maximum rules. See

Schneider and Wagemann 2012: 42-56; 87f; Goertz and Mahoney 2012: 54ff; Thiem et al. 2015: 16ff.

7

3. Misinformed Application

3.1. Debating Case Selection and Operationalization

We start our discussion of Stockemer’s application with his case selection. In his study,

Stockemer analyses 54 Latin American and Asian-Pacific countries. Despite his own

literature review pointing to the fact that “indicators driving women’s representation are often

embedded in a specific regional, social, and/or economic context” (ibid.: 87), he rates the

selection of two different world regions as a “a propitious laboratory to help determine the

usefulness of fsQCA and/or regression analysis” (ibid.: 90). He bases this assertion, for

instance, on a roughly similar level of development of the regions (measured by the region-

wide average GDP), similar “other key explanatory variables in the data set” (ibid.), and the,

on average, comparable number of female parliamentarians. However, cultural differences of

the regions are left out completely. Moreover, the difficulties in the comparison of micro-

states such as Bhutan or Papua-New Guinea with countries likes Australia, Brazil or China are

not discussed at all. While this might not necessarily be problematic, this decision has to have

an influence on the interpretation of the results. One way to avoid problems that stem from

case selection is to run separate analyses separate analyses with more context sensitive

measures, with different calibration thresholds, and probably even partially different

conditions (see e.g. Krook 2010)

However, to put more emphasis on region specific differences comes at a price. If on the one

hand two QCAs and two regressions are conducted, the number of cases decreases to 32 Latin

American or 22 Asian countries, respectively. Thus, to apply an OLS regression for each sub

sample would largely scratch the limits of statistical power (Cohen 1998), while for a QCA

these numbers of cases are not problematic per se (Schneider and Wagemann 2012: 317). One

the other hand, one could include a region specific dummy – or crisp set – to control for the

basic cultural differences. While keeping the number of cases constant, this would lead to a

higher number of explanatory factors. Hence, both approaches might not be a convenient

solution for the problem. This is especially true as the original analysis with 54 cases and 7

explanatory factors is already tangent to the “many variables, small N problem” (Lijphart

1971: 686).

According to this problem, a certain number of cases in an analysis might not be able to

display the diversity that is conceptually included by the number of explanatory factors. With

seven conditions included to the analysis 128 combinations of these conditions are logically

8

possible which outnumber the cases by far. Every combination of conditions that lacks

empirical evidence by a case is a so-called logical remainder (Ragin 2008; Schneider and

Wagemann 2012: 151ff.). For OLS regression the problem of limited empirical diversity

holds too because sufficient variance on the explanatory factors and the dependent variable

cannot be guaranteed. Additionally, the problem of influential observations (outliers) appears

in this context. This means that some observations have a “demonstrably larger impact on the

calculated values of the various estimates (coefficients, standard errors, t-values, etc.) than is

the case for most of the other observations” (Belsley et al. 1980: 11). Given the relatively

small number of 54 cases this problem does occur and weakens the statistical power of the

model, too.

Furthermore, not just the number of cases and explanatory factors but also the

operationalization of the latter is a crucial factor in both QCA and OLS regression analysis;

despite the differences between the operationalization of variables and sets. We will

exemplify some problem associated with operationalization and calibration by focusing on the

two ordinal variables “quotas” and “electoral systems” with three categories each.

Proceeding with OLS, taking these variables into regression contradicts the logic of linear

regression techniques, because dependent and independent variables have to be continuous or,

in the case of the independent variable, dummy variables. Using other types of variables for

linear regression is exclusively possible if some important conditions are fulfilled (Urban and

Mayerl 2011):

(1) The variables must at least have five categories5,

(2) the categories of variables must be in a ranked order,

(3) the distances between the categories have to be equal (both in semantic meaning and

numeric value), and

(4) the categories can be interpreted as intervals for the continuous latent variable.

But, firstly, there is ‘quotas’ (quota clauses in parties), which is coded into zero, 0.5 and one.

Whereas zero means no quota clauses in all parties, 0.5 stands for at least one party that has

quotas. If all parties have quotas (constitutionally implemented quota clauses), the variable

becomes one. Secondly, the ‘electoral system’ is coded one for proportional systems, zero for

majoritarian systems and 0.5 for all others. The two variables violate both the first and the

third condition. They have just three categories and the distance between them is neither

5 Others, e.g. Johnson 2009, even say it has to be seven or more.

9

semantically nor numerically the same. Here, Stockemer should have used categorical

dummies6 or, as a more parsimonious way, recode the data to get at least two dummy

variables. However, this would cause a loss of information. Taking this problem serious, we

use categorical dummies for the replication.

A similar problem with the same variables occurs under the calibration procedure in QCA. In

fsQCA, set memberships vary between full membership (set membership score equals 1) and

non-membership (membership score equals 0). Stockemer states that as the variables

‘electoral system’ and ‘quotas’ already have “values between 0 and 1 [… they] consequently

do not need to be transformed any more (Stockemer 2013: 93). This means that every country

with a mixed party system is calibrated as a 0.5 member in the set of ‘electoral system type’.

The same is true for every country with not all but at least one party with quota applied. But

in QCA the 0.5 anchor defines the transition point between membership and non-membership,

i.e. the “point of maximum ambiguity” (Ragin 2008: 30). Assigning this value to cases

conceptually means that we cannot say anything about the set membership of these cases;

they are neither members nor non-members of the respective set. Technically, these cases

cannot be assigned to only one truth table row but to at least two. These cases display multiple

combinations of conditions ideal typically. Thus, computer software will, first, ‘hide’ these

cases from the truth table which may cause false logical remainders.7 Second, if these cases

are covered by an explanatory conjunction that encompasses the condition with the 0.5

calibration, the software cannot display this since the cases’ membership score in the

conjunction is 0.5, too. Thus, just if the problematically calibrated condition is dropped within

the minimization process, the case can be explained at all. To be clear, none of the cases with

a 0.5 membership in at least one of the conditions is excluded from the analysis by default.

But due to this easily avoidable calibration issue, Stockemer risks to lose the explanatory

power of his model for those 17(!) cases he assigned a 0.5 set membership to in either

‘electoral system type’, ‘quota’ or both. This equals 1/3 of the cases.

Taking this points serious, in the replication we change the calibration for the two most

problematic conditions, i.e. ‘electoral system type’ and ‘quota’ without the allocation of the

0.5 anchor. In order to avoid this membership score we have to decide whether the conceptual

meaning of a 0.5 corresponds more to a (partial) membership or a (partial) non-membership in

6 That means to create as much dummy variables as categories of the former variable. The dummies get the

value one if they apply to the right category and zero otherwise. 7 A false logical remainder is a truth table row that is empty although there are cases representing the given

combination of conditions. This happens only if a set membership scores of 0.5 are assigned.

10

the particular set. For the set ‘quota’ we proceed the following way. As a case was coded 0.5

“in which at least one party represented in the national assembly […] has a party quota”

(Stockemer 2013: 91), the underlying concept of the set is (partially) represented by those

cases. Thus, we recode all 0.5 set memberships in the set ‘quota’ into a 0.668. In comparison,

the coding 0.5 for the variable ‘electoral system type’ includes any “semi-proportional or

mixed system” (ibid.). Here, without case knowledge we cannot allocate the same set

membership score for all of the countries under scrutiny. Thus, we allocate a 0.66

membership in the set ‘proportional electoral system type’ to all countries with a “Mixed

Member Proportional System (MMP)” (Idea 2015) as in this type of mixed system the “List

PR system compensates for the disproportionality in the results from the plurality/majority

system” (Idea 2015). In our understanding, this represents the concept of a proportional

electoral system rather good; i.e. such a case is more in the set than out of the set. In contrast,

a “Parallel System is a mixed system […] where no account is taken of the seats allocated

under the first system in calculating the results in the second system” (Idea 2015).

Consequently, cases with a parallel system of proportional list and majority system are treated

as more out of the set than in the set; i.e. allocated a 0.33 membership.

3.2. Replicating the Analysis

When replicating Stockemer’s analysis with categorical dummies, the results look very

similar to the original analysis (for a comparison see Stockemer’s results in appendix 1).

There are no significant effects for the electoral system and, ceteris paribus, a significant,

approximately ten percentage points higher women’s representation for countries with

implemented quotas (D_Quotas3). But there is one elementary difference between the results.

Stokemer treats the variables as continuous ones. Due to linearity assumption, Stockemer

(2013) has to interpret the coefficient in a way that every change in the independent variable

“quotas” causes a change in the dependent variable “women’s representation”. In Stokemer’s

model, ceteris paribus, countries with quotas for at least one party but not for all (quotas has

the value 0.5), the women’s representation gets approximately five percentage points higher

than in countries without quotas (D_Quotas2). But that is not true. As one can see in table 1,

8 It goes without saying that this decision is highly debatable since a) the memberships scores of particular

country should be different according to the number or importance of parties having the quota, b) the

membership score of 0.66 could be replaced by any other value indicating a partial membership (such as 0.75

or 0.8), and c) the underlying concept could have the (hidden) label ‘all-encompassing quota’ suggesting that

a single party without quota is sufficient for a (partial) non-membership of that particular case in the set

‘quota’.

11

there is no significant difference between these countries and countries without quotas9. Thus,

the effect does just occur between no quota and quotas at all parties proving the weakness of

the ordinal variable for the OLS.

Table 1: Results of OLS regression with categorical dummy variables

B SE Sig

D_Quotas2 3.98 3.01 0.202

D_Quotas3 10.35 2.38 0.000

D_PR2 -2.15 2.90 0.46

D_PR3 -0.38 2.43 0.876

Education 14.79 9.57 0.129

Year of Women Suffrage 0.06 0.09 0.535

Female Activity Rate -0.04 0.08 0.605

Degree of Democracy 0.09 0.05 0.060

D_Communist State 15.51 3.41 0.000

Constant -1.91 7.00 0.787

Furthermore, in his analysis Stockemer argues about the impact of the degree of democracy.

While it is not statistically significant at the 0.05 level, it is at the 0.1 level. This problem is

caused because of the absence of diagnostic tests to confirm the results of the analysis. As

mentioned above regression analyses with small N are very sensitive and one has to account

for influential observations (Jann 2009). A look on the data reveals that there are some

countries with a great influence. This holds especially true for the variable ‘degrees of

democracy’. Here, the effect is mainly driven by New Zealand and Australia which have a

long democratic tradition (129, respectively 106 years of democracy) and a rather high

percentage of women’s representation (33.1%, respectively 26.7%). Figure 1 shows that

without these two countries there is no significant effect for ‘degrees of democracy’ at all10

.

Even if one does not eliminate the two (or at least New Zealand) countries for the analysis,

this figure shows the sensitivity of the regression results. So here might be the overall

question whether OLS regression is an adequate method for these data.

9 When using categorical dummies in a regression analysis one category is used as reference. In this case “no

quota clauses” and majoritarian systems are taken as reference categories. 10

This is also true if only New Zealand is dropped out. Note that the education level becomes now significant

at the 0.1 level.

12

Figure 1: Added variable plot11

for years of democracy

With New Zealand & Australia Without New Zealand & Australia

Regarding the replication of the QCA, one should start with the analysis of necessary

conditions missing so far. As can be seen in table 2, no single condition is necessary due to

the non-perfect subset relations indicated by the low consistency values.

Table 2: Analysis of necessity of single conditions

Condition Consistency

necessity

Coverage

necessity Condition

Consistency

necessity

Coverage

necessity

Education 0.86 0.49 ~12

Education 0.75 0.63

Consolidation

of Democracy 0.27 0.54

~ Consolidation

of Democracy 0.21 0.62

Econ. female

activity rate 0.28 0.48

~ Econ. female

activity rate 0.34 0.86

Women’s

suffrage 0.56 0.54

~ Women’s

suffrage 0.51 0.73

Electoral

system 0.52 0.45

~ Electoral

system 0.49 0.63

Communist

country 0.21 0.78

~ Communist

country 0.04 0.22

Quotas 0.58 0.57 ~ Quotas 0.35 0.51

The analysis of sufficient combinations of conditions with the newly calibrated data does

reveal a solution term with three equifinal combinations consisting of six conditions each (see

11

Added variable plots show the effect of adding another variable to the model. For an introduction see Cook

and Weisberg 1982. 12

The tilde indicates the absence of the respective condition.

Kyrgyzstan

TurkmenistanKazakhstan

Tajikistan

Singapore

Philippines

Thailand

Mongolia

Sri LankaUzbekistan

MalaysiaBolivia

Cuba

EcuadorBhutan

GuatemalaVietnam

Brazil

Mexico

Saint Lucia

Indonesia

Haiti

Pakistan

Nicaragua

Colombia

Chile

Korea. South

Domincan R.

Honduras

Venezuela

Guyana

Cambodia

Papua New GuineaBelize

Bangladesh

El Salvador

China

Nepal

Peru

Panama

Argentina

Uruguay

Paraguay

Laos

Trinidad

Barbados

Suriname

JamaicaBahamas

Japan

India

Costa Rica

Australia

New Zealand

-10

010

20

e(

wo

men

sre

p | X

)

-50 0 50 100e( yearsdemocracy | X )

coef = .0897167, se = .04462221, t = 2.01

Malaysia

Singapore

Philippines

Uzbekistan

TurkmenistanKazakhstan

ColombiaBangladesh

Nicaragua

Tajikistan

Paraguay

Thailand

Bolivia

Kyrgyzstan

Papua New Guinea

Mexico

BhutanHonduras

Sri Lanka

Vietnam

Cuba

Peru

Haiti

Cambodia

Indonesia

Guatemala

Korea. South

Mongolia

Belize

Laos

Venezuela

Ecuador

Chile

Guyana

Argentina

China

Pakistan

BrazilDomincan R.

Nepal

Panama

Bahamas

El Salvador

Trinidad

Saint Lucia

Uruguay

Suriname

Barbados

Jamaica

Japan

Costa Rica

India

-10

010

20

e(

wo

men

sre

p | X

)-40 -20 0 20 40

e( yearsdemocracy | X )

coef = .04526299, se = .06816439, t = .66

13

table 3). While this is somewhat more parsimonious than the result by Stockemer (2013: 101)

which consists of four complex solution paths, the problems of low coverage values and

rather complex conjunctions remain.

Table 3: Analysis of sufficient (combinations of) conditions, conservative solution

Parameters of Fit edu*~dem*~eco*

~pr*com*~q

edu*dem*~eco*

~pr*~com*q

~edu*~dem*~eco

~ws*~pr*com

Consistency 1.00 0.98 1.00

Raw Coverage 0.10 0.08 0.10

Unique Coverage 0.07 0.08 0.06

Cases Cuba, Vietnam Costa Rica, Australia Nepal, Laos

Solution

consistency 0.99

Solution

coverage 0.24

Notes: edu = education, dem = consolidation/longevity of democracy, eco = female economic

activity rate, pr = proportional electoral system, q = quota provisions, com =

communist regime type, ws = women’s suffrage

In order to deal with these problems, the logical remainders can be assessed. The high number

of conditions combined with a rather medium number of cases result in 128 logically possible

combinations of the seven conditions, i.e. truth table rows (27). But, just 29 of them are

covered by empirical cases. Thus, another 99 combinations cannot be included in the analysis

in order to reduce the model complexity. Next to the so called conservative strategy of

excluding all logical remainders, two more solution strategies can be applied. First, the most

parsimonious solution term includes all those logical remainders into the minimization

process that reduce the model complexity (Schneider and Wagemann 2012: 169ff). With 99

possible logical remainders to include, the most parsimonious solution is way less complex in

comparison to the conservative one (see table 4).

Table 4: Analysis of sufficient (combinations of) conditions, most parsimonious solution

Parameters of Fit ~pr*com dem*q

Consistency 0.92 0.94

Raw Coverage 0.18 0.12

Unique Coverage 0.18 0.12

Cases Cuba, Vietnam, Nepal, Vietnam Costa Rica, Australia

Solution

consistency 0.93

Solution

coverage 0.30

Notes: dem = consolidation/longevity of democracy, pr = proportional electoral system, q =

quota provisions, com = communist regime type

14

While the results are clearly more parsimonious, they might not be as trustworthy as before

due to the ‘computerized strategy’ of selecting logical remainders for the minimization

process. However, there is a third solution term in QCA offering the opportunity to select

logical remainders on a theoretical basis, the so-called intermediate solution term (Schneider

and Wagemann 2012: 169f). In contrast to the conservative strategy it allows for the inclusion

of logical remainders, in general. But, contrary to the most parsimonious solution just those

remainders are included that are in line with the directional expectations implied by the

researcher on a theoretical basis.13

Applying this accepted strategy to the data at hand, the

complexity of the initial solution by Stockemer is resolved into two combinations of

conditions (see table 5).

Table 5: Analysis of sufficient (combinations of) conditions, intermediate solution

Parameters of Fit ~pr*com edu*dem*q

Consistency 0.92 0.94

Raw Coverage 0.18 0.12

Unique Coverage 0.18 0.12

Cases Cuba, Vietnam, Nepal, Vietnam Costa Rica, Australia

Solution

consistency 0.93

Solution

coverage 0.30

Notes: edu = education, dem = consolidation/longevity of democracy, pr = proportional

electoral system, q = quota provisions, com = communist regime type

To summarize the QCA replication: We changed the calibration for two out of seven

conditions in order to avoid allocating the 0.5 anchor. The results for the conservative strategy

are quite similar to the one done by Stockemer. Although we are able to reduce the

complexity to some degree the single solution paths still comprise rather complex

configurations. However, by applying the intermediate strategy quite parsimonious and

interpretable results can be easily obtained. What remains is the problem of low coverage

which we discuss in the following section.

3.3. Assessing the Interpretation

Stockemer’s interpretation of the effects resulting from the OLS is correct and clearly

formulated – besides the question if using quotas and electoral system as continuous

variables. One minor issue is the use of the R2 in order to interpret the quality of the analysis.

13

These criteria are so called directional expectations for the theoretically expected contribution of single

conditions to the presence of the outcome (Ragin 2008; Schneider and Wagemann 2012: 168f).

15

Based on the R2 Stockemer concludes “the current model explains 50 per cent of the variance

in the dependent variable” (2013: 96). However, due to "Occam’s Razor“ it would be better to

interpret the adjusted R2 which accounts for the number of independent variables. Even when

the interpretation of the adjusted R2 is not as intuitive as the R

2, it works better by evaluating

the accuracy of fit between data and model. In Stockemer’s analysis the adjusted R2 is 0.42

which still is considerably high..

Shifting our focus towards the fsQCA, the interpretation of the conservative solution paths is

a complex task. As stated above, one factor to explain this is the combination of few cases and

many conditions which subsequently result in a high number of logical remainders. Thereby,

the opportunity for minimization is potentially reduced. Thus, either we reduce the number of

explanatory factors in order to match it with the number of cases or we make use of logical

remainders. Applying the intermediate solution on a solid theoretical foundation is not only a

widely accepted strategy to deal with the almost inevitable problem of limited empirical

diversity in the social science (Schneider and Wagemann 2012: 152f; Schneider and

Wagemann 2013). Moreover, it reduces the complexity of the explanatory model and makes

the results more interpretable without the problem of untenable assumptions one has to face

using the most parsimonious solution. According to these results, either the combination of a

communist state with no proportional electoral system or the combination of quota provisions,

a consolidated democracy and a high level of education is sufficient for a high share of female

deputies.

Figure 2: XY-Plot for the analysis of sufficiency, intermediate solution

16

However, as we already outlined above, the coverage of the intermediate solution is still

rather low. Although being highly consistent with a consistency score of 0.95 only

approximately one third (coverage of 0.3) of the outcome set high women representation can

be explained.14

This is nicely visualized by a XY-plot from which we can deduce three things

(see figure 2).

First, we can see that we do not have cases which strongly contradict the statement of

sufficiency showing the combination of conditions but not the outcome. But the consistency is

not perfect because five countries (Laos, Vietnam, Nepal, South Korea, and Uruguay) have a

slightly greater membership in the configuration of the intermediate solution than in the

outcome set.

Second, in the end only six countries out of twenty being in the set high levels of women

representation are covered by the overall solution (Laos, Vietnam, Nepal, Cuba, Australia,

and Costa Rica). While Stockemer is right that this is not a satisfying result for a case-oriented

method which is interested in explaining as many of its cases as possible, he draws the wrong

conclusions from that: The low coverage does not mean that QCA is a poor methodological

tool for the analysis of women representation in national assemblies. The clustering of cases

in the upper left quadrant of the XY-plot – what we call “equifinality-corner” – rather

indicates that not the most relevant factors (or indicators) have been selected for the analysis

and that others – equifinal ones – have to be found to sufficiently explain high women

representation in Latin American and Asian-Pacific countries.

Closely related to that, third, data allocation is highly skewed. Just ten out of 54 cases are

located outside the left triangle of the XY plot. By that, all other cases do not contradict both

the sufficiency of the intermediate solution for the outcome and for the non-outcome. This

indication of simultaneous subset relations raises the questions whether skewedness stems

from single variables (e.g. communism) and subsequently, whether the explanatory factors

have been selected and calibrated in a proper way (Schneider and Wagemann 2012: 168ff.).

4. Concluding Remarks

The starting point of our discussion was the devastating critique by Daniel Stockemer

dismissing QCA due to its “suboptimal nature” (Stockemer 2013: 88) calling it a “poor

methodological choice” (ibid: 94) to study women representation in national assemblies 14

Coverage of 0.3 does not mean that 30% of the cases are explained by the solution as Stockemer suggests but

that the solution set covers 30% of the outcome set. Instead, explaining 6 of 54 cases equals just 11%.

17

which does not “provide any additional insight” (ibid: 96). However, from our perspective

these judgments stem from a misunderstanding of central ideas and features of QCA, and its

misinformed application. In our opinion, there is nothing “inherently problematic”

(Stockemer 2013: 97) with neither fsQCA nor OLS regressions. Both approaches are rooted

in fundamentally different logics, apply completely diverse techniques, and have their distinct

pitfalls one has to cope with.

If we take these points seriously for the study of descriptive women representation in national

legislatures across the globe we cannot subscribe to Stockemer’s observations. Set-theoretical

thinking differs from standard quantitative approaches in so many ways that it is not possible

to contrast their findings against each other. Thus, questions like “which method is more

appropriate” (ibid: 87) or which one has “greater leverage” over the other (Krook 2010: 887)

are not very expedient and fruitful. It is not about more or less leverage but about different

leverages. In this sense, it is like in the simile of the blind men touching and feeling different

parts of an elephant coming to completely different conclusions about what kind of animal

they have in front of them. Goertz and Mahoney (2012, 2013) have called this the “Rorschach

principle”: Social scientists perceive social phenomena differently if they look at them

through set-theoretic or quantitative lenses. Therefore, the real challenge is to reconcile the

different perspectives and to strive for complementary conclusions.

References

Belsley, David A., Edwin Kuh, Roy E. Welsch. 1980. Regression Diagnostics: Identifying

Influential Data and Sources of Collinearity. Massachusetts: Wiley-Interscience.

Braumoeller, Bear F. and Gary Goertz. 2000. “The Methodology of Necessary Conditions”.

In American Journal of Political Science, Vol. 44, 844-858.

Braumoeller, Bear F. and Gary Goertz. 2003. “The Statistical Methodology of Necessary

Conditions”. In Gary Goertz and Harvey Starr (eds.). Necessary Conditions: Theory,

Methodology, and Applications. Lanham: Rowman & Littlefield, 197-223.

Clark, William R., Michael J. Gilligan, and Matt Golder. 2006. “A Simple Multivariate Test

for Asymmetric Hypotheses”. In Political Analysis, Vol. 14, 311-331.

Cohen, Jacob 1988. “Statistical power analysis for the behavioral sciences”. Hillsdale, NJ:

Erlbaum.

Cook, Dennis R. and Sanford Weisberg .1982. Residuals and Influence in Regression. New

York: Chapman and Hall.

Dahlerup, Drude. (ed.). 2006. Women, Quotas, and Politics. New York: Routledge.

Goertz, Gary and James Mahoney, 2012. A Tale of Two Cultures: Qualitative and

Quantitative Research in the Social Sciences. Princeton: Princeton University Press.

18

Goertz, Gary and James Mahoney. 2013. “Methodological Rorschach Tests: Contrasting

Interpretations in Qualitative and Quantitative Research”. In Comparative Political Studies,

Vol. 46(2), 236-251.

International Institute for Democracy and Electoral Assistance. 2015. Glossary of Terms.

http://www.idea.int/esd/glossary.cfm (last access: May 22nd

2015)

Inglehart, Ronald and Pippa Norris. 2001. Rising Tide: Gender Equality and Cultural Change

Around the World. Cambridge: Cambridge University Press.

Inter-Parliamentary Union. 2015. Women in National Parliaments. http://www.ipu.org/wmn-

e/classif.htm (last access: May 12th

2015).

Jann, Ben. 2009. “Diagnostik von Regressionsschätzungen bei kleinen Stichproben (mit

einem Exkurs zu logistischer Regression)”.In Peter Kriwy and Christiane Gross (Eds.).

Klein aber fein! Quantitative empirische Sozialforschung mit kleinen Fallzahlen.

Wiesbaden: VS-Verlag, 93–125 .

Johnson, Paul E. 2009. Working with Ordinal Predictors. Paper presented at the annual

meeting of the Midwest Political Science Association, Chicago, Illinois.

Krook, Mona Lisa. 2009. Quotas for Women in Politics: Gender and Candidate Selection

Reform Worldwide. New York: Oxford University Press.

Krook, Mona Lisa. 2010. “Women’s Representation in Parliament: A Qualitative

Comparative Analysis”. In Political Studies, Vol.58(5), 886-908.

Krook, Mona Lisa and Sara Childs. 2010. Women, Gender, Politics. A Reader. Oxford:

Oxford University Press.

Lijphart, Arendt. 1971. Comparative Politics and Comparative Method. In American Political

Science Review, Vol. 65, 682-93

Matland, Richard E. 1998. “Women’s Representation in National Legislatures: Developed

and Developing Countries”. In Legislative Studies Quarterly, Vol. 23(1), 109-25.

McAllister, Ian and Donley T. Studlar. 2002. “Electoral Systems and Women’s

Representation: A Long-Term Perspective.” In Representation 39 (1), 227-249.

Paxton, Pamela M. and Melanie M. Hughes. 2014. Women, Politics, and Power: A Global

Perspective. Thousand Oaks, CA: Pine Forge Press.

Paxton, Pamela M. and Sheri Kunovich. 2003. “Women’s Political Representation: The

Importance of Ideology”. In Social Forces, Vol. 82(1), 87-113.

Ragin, Charles C., 1987. The Comparative Method. Moving Beyond Qualitative and

Quantitative Strategies. Berkley: University of California Press.

Ragin, Charles C., 2000. Fuzzy-Set Social Science. Chicago: University of Chicago Press.

Ragin, Charles C., 2008. Redesigning Social Inquiry. Fuzzy Sets and Beyond. Chicago:

University of Chicago Press.

Rihoux, Benoît and Charles C. Ragin (eds.), 2009. Configurational Comparative Methods.

Qualitative Comparative Analysis (QCA) and Related Techniques. Thousand Oaks: Sage.

Roberts, Andrew, Jason Seawright and Jennifer Cyr. 2013. “Do Electoral Laws Affect

Women’s Representation?” In Comparative Political Studies, Vol. 46(12), 1555-1581.

19

Rosen, Jennifer. 2012. “The Effects of Political Institutions on Women’s Political

Representation. A Comparative Analysis of 168 Countries from 1992-2010”. In Political

Research Quarterly, Vol. 66(2), 306-321.

Rosenbluth, Frances M., Robert Salmond and Michael F. Thies. 2006. “Welfare Works:

Explaining Female Legislative Representation”. In Politics and Gender, Vol. 2(2), 165-192.

Ruedin, Didier. 2012. “The Representation of Women in National Parliaments. A Cross-

National Comparison”. In European Sociological Review, Vol. 28(1), 96-109.

Schneider, Carsten Q. and Claudius Wagemann. 2012. Set-Theoretic Methods for the Social

Sciences. A Guide to Qualitative Comparative Analysis. Cambridge: Cambridge University

Press.

Schneider, Carsten Q. and Claudius Wagemann. 2013. “Doing Justice to Logical Remainders

in QCA. Moving Beyond the Standard Analysis”. In Political Research Quarterly,

Vol. 66(1), 211-220.

Stockemer, Daniel. 2013. “Fuzzy Set or Fuzzy Logic? Comparing the Value of Qualitative

Comparative Analysis (fsQCA) versus Regression Analysis for the Study of Women’s

Legislative Representation”. In European Political Science, Vol. 12, 86-101.

Stockemer, Daniel. 2014, “Women’s Descriptive Representation in Developed and

Developing Countries”. In International Political Science Review, 1-16.

Thiem, Alrik, Michael Baumgartner and Damien Bol. 2015. „Still Lost in Translation! A

Correction of Three Misunderstandings Between Configurational Comparativists and

Regression Analysts”. In Comparative Political Studies, online first.

Tripp, Aili M. and Alice Kang. 2008. “The Global Impact of Quotas: The Fast Track to

Female Representation”. In Comparative Political Studies, Vol. 41(3), 338-361.

Urban, Dieter, Jochen Mayerl. 2011. Regressionsanalyse: Theorie, Technik und Anwendung.

Wiesbaden: VS Verlag.

Viterna, Jocelyn, Kathleen M. Fallon and Jason Beckfield. 2008. “How Development Matters:

A Research Note on the Relationship Between Development, Democracy and Women’s

Political Representation”. In International Journal of Comparative Sociology, Vol. 49(6),

455-77.

Wängnerud, Lena. 2009. “Women in Parliaments: Descriptive and Substantive

Representation”. In Annual Review of Political Science, Vol. 12, 51-69.

20

Appendix

Table A1: Table 1: “Results the regression equation” by Stockemer (2013: 95)