Constructing experimental tests by questionnaire Barbara ...However, results of their study contain...

1

Measurement of income inequality re-examined

Constructing experimental tests by questionnaire

Barbara Jancewicz

Mailing address: Schroegera 83/7, 01-828 Warsaw, Poland

Email address: [email protected]

2

Keywords: Income Transformations, Inequality, Inequality Axioms, Questionnaire

Experiments, Transfer Principle

Abstract

Perception of income inequality by ordinary people is a relatively new field of research, but its

importance is rapidly growing. More and more non-specialists are basing their opinions and

decisions on inequality measures, with Gini coefficient, Theil measure and Atkinson index

being the most frequently used ones.

Despite the popularity of inequality measures, underlying assumptions of the most popular of

them do not fully hold, as shown in early research by Amiel and Cowell (1992, 1999).

However, results of their study contain multiple puzzling and inconsistent answers. Further

research by these authors on the topics of inequality, risk, justice and polarisation perception is

still burdened by this problem, even though the questionnaires differ and span multiple topics.

This paper analyses methods used by Amiel and Cowell to elicit respondents’ views about

income inequality. It presents results of a quantitative questionnaire repeated after Amiel and

Cowell, combined with qualitative interviews with selected respondents. The research was

conducted in Poland on 132 sociology and economy students.

Qualitative interviews and subsequent data analysis revealed multiple problems that caused

respondents to answer inconsistently, and solutions to some of them are proposed.

The questionnaire should offer a standardized set of possible answers to verbal questions, since

different structure and adding explanations to some of the answers causes confusion both for

the respondents and the researcher. The article proposes also giving respondents a possibility

of expressing ambivalence or doubt, since some inconsistencies were a result of a forced

answer.

Understandability of the questionnaire can be increased by using natural language and avoiding

abstract examples. This applies also to salaries, which should be presented in a currency

familiar to the respondents and have plausible ranges, otherwise some respondents will

transform them into such on their own.

We should try to limit the number of questions used only to the ones most relevant to the

analysed problem. However verbal and numerical questions show different aspects of

inequality, so using and comparing both types of questions is advised, even though it lengthens

3

the questionnaire and gives an opportunity of conflicting answers. These problems can be only

partially solved by merging possibly conflicting questions into one (e.g. a ranking task of 3

distributions instead of 2 or 3 pairwise comparisons). The number of conflicts can be also

decreased by rearranging the questionnaire so that corresponding questions follow one another,

or using a computer-based interactive questionnaire.

Results presented in this paper allow construction of better questionnaires about income

inequality and related topics.

4

1. Introduction

Income inequality is a commonly discussed and commented on subject in terms of its causes,

effects, costs, fairness etc. People recognize it as an important topic that influences economies,

culture and their lives (Wilkinson and Pickett, 2010; Stiglitz, 2012). But how to exactly define

inequality is a difficult question – in practice we assume definitions imposed by inequality

measures (functions that transform data about people’s incomes into a single number). Experts

proposed many different ways to measure inequality and discussion about which one of them

is the best soon followed. It quickly concentrated on axioms – propositions stating how a

measure should behave under certain circumstances, e.g. when all incomes are raised by 10%.

There are many rivalry axioms that portray different ways of defining inequality. Many

currently used inequality measures, like Gini, Atkinson and Theil, share a small set of axioms,

even though each of these measures is different. Some of these properties are still considered

controversial. Still, there is no easy, objective way to decide which axiom is better, since all of

them have their merits.

The perceived simplicity of inequality measures made discussions about inequality easier, both

for trained experts and for members of the general public, who started taking interest in the

topic. The cost of popularity was lowered depth of understanding of inequality by interested

parties and common misinterpretations. This new situation creates a need to define inequality

measures in a way that corresponds best with people’s intuition and perception of inequality.

There is a lot of research about the perception of income inequality by ordinary people. For

example Decoster and Schokkaert (2002) talk about how the Flemish working population

perceives inequality in their country, Cuena et al. (2004) write about student’s perception of

inequality in leaky-bucket experiments. However, very few scientists focus on how the

understanding of inequality differs from the meaning of income inequality measures, when

looking from the axiomatic perspective. It is the purpose of this article to critically examine

one of the first and most broad studies in this field.

The earliest inquiry in this topic was carried out by Cowell (1985), later in collaboration with

Amiel (1992). They conducted an income inequality questionnaire on 1108 students in USA,

UK, Israel and Germany. While their research has shown that axioms shared by most popular

inequality measures do not have overwhelming support among the respondents, they also found

that many of their subjects’ answers illogical, inconsistent or inexplicable. Further research by

5

Amiel and Cowell (1999, 2002) and later by Ballano and Ruiz-Castillo (1993) and Harrison

and Seidl (1994) also resulted in multiple puzzling answers from the respondents.

The abundance of inconsistent answers can be caused by lack of clear concept of inequality

among the respondents or by the inequality measures themselves being distant from people’s

intuition about inequality. In the first case the only way to reduce misinterpretations is by better

economic education, in the second – there is a need for new and better inequality measures.

However, the most probable cause of so many inconsistent answers is inadequate methodology.

All of the aforementioned researchers were aware of problems and limits of their study. The

field of testing axioms with the population is quite new and scientists are struggling to find the

right way to question respondents while minimizing errors and inconsistencies.

The aim of this article is to provide insight and possible improvements to the methodology of

research on inequality measures with regard to their axioms. The paper also presents and

compares results from Polish replication of research originally conducted by Amiel and

Cowell. It concentrates on the first questionnaire created by Amiel and Cowell, since it

incorporates a variety of ideas on how to elicit views in such a difficult topic and takes an

innovative approach to the problem. Other researchers tried to touch this issue from this or

similar perspective, with mixed success (Devooght (2003) made similar inquiries, though he

based his questionnaire not on axioms, but on the theory of inequality by Temkin (1993); Traub

et al. (2003, 2009) asked respondents to rank distributions in different settings, e.g. self-concern

and social-planner and then tried to interpret their answers from axiomatic perspective). Amiel

and Cowell’s research found many followers, who faced similar problems which have not yet

been solved and this article hopes to aid in finding solutions. After facing difficulties with their

first questionnaire, Amiel and Cowell turned to comparing inequality with risk, polarisation,

welfare and other concepts (2001, 2002, 2007 with Ramos, 2012 with Gaertner). They avoided

the problem of illogical responses by limiting their questionnaire’s verbal section and accepting

the remaining inconsistencies in some answers as normal. They also tried to attribute

inconsistencies to respondent’s individual characteristics (2004 with Slottje). It is the author’s

belief that the original methodology, with some improvements, will yield better results and

fewer inconsistent answers from the respondents.

For the purpose of improving the methodology, the study done by Amiel and Cowell (1992,

1999) was replicated on a group of Polish social studies and economy students. Additionally,

many respondents who gave answers that were difficult to interpret were invited to an in-depth

6

interview to better understand their reasoning. This paper very thoroughly analyses and

compares the results obtained and the questionnaire itself. As a result it proposes several ways

in which the original research methodology can be improved.

The paper is structured as follows. Section 2 describes the combined quantitative and

qualitative approach used in current research. Section 3.1 presents the questionnaire used and

compares results obtained by Amiel and Cowell with those from the Polish inquiry. Section 3.2

discusses general problems with the questionnaire and offers possible solutions, followed by a

conclusion in Section 4.

2. The approach

Problems with interpretation of questionnaire results are common, especially when the survey

is about a complicated and theoretical topic. A result of a quantitative interview is simply a list

of choices made by respondents. Although questionnaires often encourage people to leave

comments or afterthoughts, few respondents really do. As a result, the task of understanding

reasons and motivations of people questioned lays solely on the researchers, with little or no

additional guidance from the respondents.

The lack of understanding of obtained results indicates that we need to take a different

approach. In this case, since our main interest is discovering motivations and reasons for certain

choices made in a questionnaire, the best method seems to be a qualitative interview. We would

like to concentrate on people whose responses were described by Amiel and Cowell as

‘unconventional’, that is why it would be best to interview those who gave such puzzling

replies. In order to discriminate such respondents and check whether our sample seems to

perceive inequality similarly to Amiel and Cowell’s, their study was replicated. The first stage

of the research consisted of an auditory questionnaire conducted in Warsaw School of

Economics and in the Institute of Sociology at the University of Warsaw. The questionnaire

was an exact translation (done by the author) of the one used in the first Amiel and Cowell

research (1992, 1999).

The second part of the research consisted of a series of in-depth interviews with some of the

questionnaire respondents. In the first part of the study all of the respondents were given a

questionnaire with a code assigned to it and a contact card which they were asked to fill if they

agree to participate in the second part of the research. Almost one third (31.1%) of the sample

agreed to an in-depth interview. Among those who also added their questionnaire’s code to the

7

contact card (25.8% of the sample) the interviewees were selected1. The interviewees (as they

will be called in the article) presented as a group almost every type of unconventional or

illogical answer, so different problems could be discussed and analysed.

The sample used by Amiel and Cowell consisted of “(...) upper-level undergraduates who had

some training in economics but who had not previously taken courses that included studying

the measurement of income inequality” (1992, p. 7). The idea was to use subjects “who are

likely to avoid arithmetical mistakes and logical slips” (1992, p. 5). In an attempt to replicate

their research as closely as possible, the Polish sample consisted of undergraduates who did not

frequent any inequality measurement classes. In order to ensure arithmetical abilities, all our

respondents were students who had some training in statistics. Warsaw School of Economics

students also already took a basic course in economics2.

Translating a questionnaire into another language, even when done with care and expertise, is

always another source of additional differences in results. Nevertheless, Amiel and Cowell’s

study also uses the questionnaire translated into 4 different languages,

The first study done by Amiel and Cowell shows only small differences between the answers

of respondents from different countries who received the questionnaire translated into their

native languages. Further research by those authors show mixed results concerning variations

of inequality perceptions among different nationalities, type of education and sex.

Nevertheless, the samples used by the researchers (also the Polish sample used in this study)

were groups of students that frequented a class, so the results obtained are not truly comparable.

However for testing a new area of inquiry and improving methodology, these samples suffice,

especially when combined with qualitative methods of research.

3. Discussion and results

The results of the Polish study is twofold: on one hand we have another set of results of a

questionnaire that was already conducted in various countries; on the other, thanks to

qualitative interviews and methodological analysis, we gain a better understanding of

1 One person, who filled the questionnaire with as many as 5 inconsistencies, two people with 4, one person with

none (for comparison). Other 6 interviewees ranged from 2 to 3 inconsistencies and were selected so that all

possible puzzling answers would be covered. 2 Students at Institute of Sociology take basic economy course slightly later, when they might have already taken

an income inequality class.

8

respondents’ answers. The first part of this section concentrates on questionnaire results, while

the second one gives more thought to more general choices, problems and possible solutions.

3.1. Questionnaire construction and axiom’s support

This section presents 5 main topics that were addressed in Amiel and Cowell’s study, namely:

effect of income transformations and population replication, the principle of transfers,

decomposition by population subgroups and unbalanced enrichment of a society. The results

obtained both in the original and the replicated study are summarised. This section also

describes the questionnaire itself, but is organized according to topic that certain questions

concern instead of the order in which they appeared in the questionnaire.

Income transformations

Scale and translation invariance are two rivalling axioms that deal with the problem of how to

compare two distributions with different income means. Scale invariance assumes that

inequality remains unchanged when all incomes are multiplied by a positive constant, while

translation invariance claims that it is adding a constant that has no effect on inequality level.

In the numerical section respondents were asked to point out of a presented pair of income

distributions the one that they consider more unequal. In question 1 the second distribution was

created from the first by doubling incomes and in question 2 by adding 5 units to each income.

Throughout the article respondents will be labelled supporters of certain axioms if they answer

according to those axioms to selected set of questions. For example, a supporter of scale

invariance axiom should mark “the same” in question 1 (that multiplying incomes preserves

the level of inequality), and “down” in question 2 (that adding 5 units to each income decreases

inequality). Followers of translation invariance should mark “up” in first question (since

multiplying incomes increases absolute differences) and “same” in the second. Someone can

be categorized a supporter of scale invariance when only question 1 is considered, but can be

labelled as non-supporter when both questions 1 and 2 are taken into account, for instance

when he or she answered “the same” to both questions.

Q1. A = (5, 8, 10) B = (10, 16, 20)

Q2. A = (5, 8, 10) B = (10, 13, 15)

Table 1 shows that 37% respondents in the original study and 16% in the Polish one are

supporters of scale invariance, when answers to both numerical questions is considered. In case

of translation invariance the corresponding percentages are 17% and 26%. Many respondents,

especially in the Polish sample (25%), chose to answer “the same” in both questions, which is

9

puzzling since those axioms are considered rivalling. They were constructed as rules used to

compare distributions with different income means. Accepting both is theoretically possible,

but it leads to an enormous amount of distributions that are considered to have the same level

of inequality even though they differ greatly. For example distributions like (1000, 1002, 1003)

and (1000, 2000, 3000) would be considered equally unequal. Amiel and Cowell comment on

their respondents answers: “(…) there is a bias in favour of saying ‘the same’ even when such

a response would appear illogical: perhaps this reflects an innate ‘safety first’ response on the

part of student respondents” (1992, p.12-13).

The follow-up interviews clarified this apparent discrepancy to some extent. Our interviewees

either did not see the conflict between their answers or considered the answer “the same” as

equal to “I don’t know” – this problem is discussed later on in the article. Both pairs of

distributions are similar in some way (“So, they are generally the same, but (…), from what I

see they differ by 5” [resp. no. 6038]), so when they are considered separately it encourages to

answer “the same” in both of them.

Table 1. Income transformation – answers to numerical questions compared.

A&C Add 5 units (q2)

Down (%) Up (%) Same (%)

Double income (q1)

Down (%) 8 2 5

Up (%) 15 3 17*

Same (%) 37** 5 9

Source: Amiel and Cowell (1992)

Poland (N = 131) Add 5 units (q2)


Double income (q1)

Down (%) 6 0 2

Up (%) 15 8 26*

Same (%) 16** 2 24

Source: Own research

* translation invariance supporters

** scale invariance supporters

Bold: puzzling answers

During the follow-up interviews, we asked respondents who answered in both questions 1 and

2 “the same” to rank all three distributions used in those questions instead of comparing them

pairwise: (5, 8, 10), (10, 16, 20), (10, 13, 15). Most interviewees put them at different inequality

10

levels, choosing one axiom over another. What motivated them to do that was the comparison

between the distributions after income changes ((10, 16, 20) and (10, 13, 15)), which wasn’t in

the original questionnaire. From this was established that inviting respondents to consider both

axioms concurrently improves the quality of received data.

Those who answered in accordance with scale or translation invariance in numerical questions

did not necessarily agree with those axioms in general. Three verbal questions were asked to

elicit views about those axioms – one depicted a situation of income multiplication, other two

deducing and adding fixed amount of money to everyone’s income.

Q10. Suppose we double the “real income” of each person in a society, when not all

the initial incomes are equal.

(a) Each person’s share remains unchanged, so inequality remains unchanged.

(b) Those who had more also get more, so inequality has increased.

(c) After doubling incomes more people have enough money for basic needs, so

inequality has fallen.

Q11a. Suppose we add the same fixed amount to the incomes of each person in a

society, when not all the initial income are equal.

(a) Inequality has fallen because the share of those who had more has fallen.

(b) Inequality remains the same.

(c) Inequality has increased

Q11b. Suppose instead of adding we deduct a fixed amount from each person’s income.

Then inequality...

(a) Is the same

(b) Increases

(c) Decreases.

When asked verbally, 45%-47% of respondents agreed with the scale invariance axiom both in

Polish and A&C sample. Table 2 presents that the level of support for translation invariance is

higher among Poles (60% when adding, 50% when deducing income) than among students

questioned by A&C (35% and 28% accordingly)3, but differences between verbal questions

concerning translation invariance is similar.

3 Polish students answered in favour of translation invariance surprisingly often, possibly

studying properties of variance (which is translation invariant) during basic statistical course,

could have influenced their judgement.

11

Table 2. Income transformations – answers to verbal questions.

Sample N Double income (q10) Add fixed sum (q11) Deduct fixed sum (q11)

Down Up Same Down Up Same Down Up Same

A&C 1108 12% 40% 47%** 58% 6% 35%* 7% 64% 28%*

Poland 122; 122; 121 15% 39% 45%** 38% 2% 60%* 4% 46% 50%*

Source: Amiel and Cowell (1992), own research



Analysing answers to two questions simultaneously (as presented in Table 3) leads to

significant fall in axioms support: by 17% and 29% for scale invariance in A&C and Polish

sample accordingly (to 30% and 16%) and by 18% and 40% for translation invariance (to 17%

and 20%). Again there is a significant percentage of responses “the same” to all the questions

concerning income transformation.

Table 3. Income transformation – answers to verbal questions compared.

A&C Add fixed sum (q11a)


Double income (q10)

Down (%) 7 1 4

Up (%) 21 2 17**

Same (%) 30* 3 14

Source: Amiel and Cowell (1992)

Poland (N = 132) Add fixed sum (q11a)


Double income (q10)

Down (%) 4 0 11

Up (%) 18 2 20**

Same (%) 16* 1 29




Bold: puzzling answers

Respondents were asked about support for translation invariance twice: in case of adding and

deducting a fixed amount of money from each income (11a and 11b). Both of these questions

were presented with the same numerical example, yet the order of distributions differed in each

of them. Interviewees who marked conflicting answers in questions 11a and 11b (19% in A&B

and Polish sample) often did realize it, but argued that emotions made those situations different:

“Logically, I should say, that it remains the same, (...) but I have a feeling that this inequality

12

will be higher, because of situations between people. If they lived peacefully at the level of 10

or 1000 or whatever and now they will have to count every penny to somehow manage to

survive, then the atmosphere in those groups and between them will cause the inequality to be

higher.” [resp. no. 1406]. Viewing losses and gains differently is a proven phenomenon

(Kahneman and Tversky 1979) that often influences survey results. Additionally, respondents

noticed that they were asked twice about the same thing and they might have felt tested instead

of asked for an opinion. In this case it seems that asking only one translation invariance

question would suffice since the second one does not increase our understanding of the

respondents significantly.

Amiel and Cowell declared their numerical and verbal questions uncomparable (1992, p. 13),

but comparing those answers gives us additional information about respondents’ perspective

and a more accurate information about the level of axiom acceptance. If someone supports an

axiom, it should show in almost all answers concerning it, not just in the one type of questions.

It seems though that by taking this approach we are left with very few supporters (8 for scale

and 10 for translation invariance in Polish sample4) and many intermediate or undecided ones.

This probably was an additional reason why Amiel and Cowell omitted this comparison. In the

Polish sample answers from numerical and verbal questions about income transformations

differed and most interviewees were unaware of ambiguity in their answers, even though the

questionnaire encouraged reconciling them. Using a computer-based interactive questionnaire

could help in solving this problem without changing the questionnaire structure.

Population replication

The next major axiom investigated by A&C is about population replication. Most commonly

used inequality measures (e.g. Gini, Theil, Atkinson) are normalised in terms of population

size; that means they view people in terms of percentage of population (e.g. one person is

viewed only as 10% of a society of size 10). Population replication states that “cloning” a

society, even multiple times, does not change the level of inequality in it. It is the only axiom

that deals with comparing inequality between differently sized groups. Both questions

concerning this axiom were about doubling population size.

4 When considering answers to questions 1, 2, 10, 11a and 11b

13

Q3. A = (5, 8, 10) B = (5, 5, 8, 8, 10, 10)

Q12. Suppose we replicate a three-person society by merging it with an exact copy of

itself (so that we now have a society of six people consisting of three sets of identical

twins).

(a) The income inequality of the six-person community is the same as that of the

three-person community because the relative income shares remain unchanged.

(b) The income inequality of the six-person community is less than that of three-

person community because in the six-person community there are some people

who have the same income.

(c) The income inequality of the six-person community is greater than that of the

three-person community.

Most respondents supported population replication: 58% agreed with it in the original sample

in the numerical and 66% in the verbal question, while the corresponding percentages in the

Polish sample were 51% and 60% (Table 4)5. The second most popular answer was that

inequality among the bigger population is lower. Interviewees who gave such a reply pointed

out the same thing that appeared in the verbal question: that in multiplied group everyone has

someone in the same situation as them, so no one is alone and ostracized.

Table 4. Population replication - answers to numerical and verbal questions.

Sample N Numerical (q3) Verbal (q12)

Down Up Same Down Up Same

A&C 1108 31% 10% 58%* 22% 9% 66%*

Poland 131 40% 9% 51%* 31% 8% 60%*


* population replication supporters

Questions concerning population replication were rarely a source of conflicting answers. Only

one fourth of the Polish sample (26%) replied to them in an inconsistent way. However, the

verbal question is quite unintuitive, since the idea of cloning or replicating a society seems

rather unnatural. All the answers are also quite long and the first two have explanations attached

to them, but the last one does not, which was also the least popular reply. A question about

joining of two similar societies6 that has no explanations added to the answers would make it

shorter and possibly easier to imagine and answer.

5 When both verbal and numerical questions were taken into consideration the support for population replication

in the Polish sample fell to 45%. 6 Maybe countries, as Amiel and Cowell use the idea of Alfaland and Betaland in their later questionnaires

14

Transfer principle

Amiel and Cowell consider principle of transfer as an axiom that has no obvious alternatives

in the literature (1999, p.47). That is why they analyse it closely, they even dedicated a separate

questionnaire to it in their later study. The Pigou-Dalton’s principle of transfers states that

transferring a small amount of money form a richer person to a poorer one decreases inequality.

Q4. A = (1, 4, 7, 10, 13) B = (1, 5, 6, 10, 13)

Q13. Suppose we transfer income from a person who has more income to

person who has less, without changing anyone else’s income. After the transfer

the person who formerly has more still has more.

(a) Income inequality in this society has fallen. [agree]

(b) The relative position of others has also changed as a consequence of

this transfer. Therefore we cannot say, a priori, how inequality has

changed. [strongly disagree]

(c) Neither of the above. [disagree]

Amiel and Cowell were surprised by the pattern of acceptance of this axiom. They comment

on the results of the numerical question, saying that “nearly two thirds of the sample fail to

agree with the transfer principle” (1992, p. 16). When the axiom was presented verbally 60%

of the original sample supported it (Table 6.). Results from the Polish sample are very similar

38% supports the axiom in the numerical and 55% in the verbal question. It is more significant

though that in the Polish sample answers to verbal and numerical questions are unrelated7

(Table 7.), as if those questions concerned two completely different topics. This suggests that

either those two types of questions show us a completely separate perspective on the problem

or, which is more likely, that one or both of them were in some way misinterpreted by the

respondents.

Table 6. Principle of transfers – answers to numerical and verbal questions.

Sample N

Numerical (q4) Verbal (q13)

Agree Strongly

disagree Disagree Agree

Strongly

disagree Disagree

A&C 1108 35% 42% 22% 60% 24% 14%

Poland 132 38% 42% 20% 55% 23% 22%


7 Independence hypothesis was accepted by Pearson’s Chi-Square test at significance level of 0.05.

15

Table 7. Principle of transfers – answers to numerical verbal questions compared for the Polish

sample.

Transfer principle

Verbal (q13)

Agree Strongly

disagree Disagree

Numerical

(q4)

Down 23% 9% 6%

Up 20% 9% 13%

Same 11% 5% 3%


Note: In the original sample the percentage of those who agreed with the axiom both in

numerical and verbal question was 36% (Amiel and Cowell, 1992, p.17).

The numerical question concerning the principle of transfers is in a way tricky. Interviewees

who disagreed with transfer principle in the numerical question often pointed out that the first

example (1, 4, 7, 10, 13) is very regular in nature, because the gaps between consecutive

incomes are the same “(...) there is exactly a straight line here. (...) and here everything is so

perfectly arranged.” [resp. no. 1616]. The second distribution on the other hand disrupts this

sequence breaking this regularity and leaves the poorest person further away from others. That

is why many respondents viewed this change as inequality increasing.

Asking about the transfer principle by describing it straightforwardly yielded more answers

agreeing with the axiom, but as Amiel and Cowell noted “those whose responses differed as

between question 4 and question 13, the majority did not indicate any desire to go back and

change their response to question 4.” (1992, p.17). The situation in the Polish sample was

similar, so it was asked about it in the interviews: as a result two explanations emerged. First

based on the fact that most interviewees did not notice that a transfer took place in the numerical

question, so they did not take it into consideration while looking at the question for the first

time. Not everyone compared their numerical and verbal answers, so they did not notice the

problem. Most interviewees changed their answer to agreement with the transfer principle,

when the transfer was shown to them. Their second explanation was that the transfer principle

sounds “right”. In theory they understood that the rule was general, but they thought rather

about most extreme and thus obvious examples of its application. The scenario of a poor person

giving to those even poorer did not occur to them. After being confronted with the conflict in

their answers concerning the principle of transfer the interviewees tended to withdraw their

support for this axiom, but they did it rather reluctantly. One of the interviewees even said that:

“I mean, the principle that moving money to the poorer [people] seems fair, except the case

16

here (1,5,6,10,13).” [resp. no. 8932] So she did claim to agree with the axiom, but with this

one exception.

This situation proves that asking about axiom acceptance only verbally or only numerically

does not give us the full picture, since both answers contain different kinds of biases. Asking

both questions does give us additional information and makes data more reliable, but creates a

problem of conflicting answers that needs to be somehow solved.

Decomposability

The last axiom investigated by Amiel and Cowell was decomposability, which states that

inequality in a whole society comes from inequality within groups and differences between

them (e.g. a divide between incomes of men and women). That is why Theil index (which has

this property) is often used for in-depth analysis even though its interpretation is quite

unintuitive.

Three questions in the questionnaire were dedicated to decomposability: two numerical and

one verbal. Again we can see serious discrepancies in the axiom’s support between verbal and

numerical questions, yet this time they are the other way round: the numerical questions suggest

bigger support of the axiom than the verbal one.

Q5. A = (4, 8, 9) B = (5, 6, 10)

Q6. A = (4, 7, 7, 8, 9) B = (5, 6, 7, 7, 10)

Q14. Suppose there are two societies, A and B, with the same number of

people and with the same total income, but with different distributions of

income. Society A is now merged with C, and society B is merged with C’

where C and C’ are identical.

(a) The society which had the more unequal income distribution before the

merger still has the more unequal distribution after the merger.

(b) We can’t say which society has the more unequal income distribution

unless we know the exact distributions.

(c) Neither of the above.

Decomposability is a quite a complicated axiom to explain, so the verbal question is quite

difficult. It requires comparing inequality in four abstract societies. In order to maintain

precision Amiel and Cowell resorted to the use of quite a formal language, so reading the

question properly needs concentration and effort from the respondents. Using a more natural

language and example that is easier to imagine could make it more understandable. For instance

we can describe a situation of two countries to which a small region joins in.

17

Analysing support for decomposability in numerical part requires two questions. The first one

(q5) asks about difference in inequality between two small communities: (4, 8, 9) and (5, 6,

10). The second question (q6) compares those two communities both merged with a two person

group (7, 7). An answer is in accordance with decomposability axiom if it is identical in both

questions. Many respondents did that without realizing (at least at first) what the connection

between those two questions was.

Support for decomposability in the A&C sample was quite strong: 57% in the numerical and

40% in the verbal part. In the Polish sample it was 49% and 27% accordingly (Table 8.), but

only 16% expressed support in both of those questions simultaneously. What is more important

is that the pattern of answers suggests complete lack of relation between those two questions8.

Table 8. Decomposability – answers to numerical and verbal questions.

Sample N

Numerical (q5 & q6) Verbal (q14)

Same Different Agree Strongly

disagree Disagree

A&C 1108 57%* 41% 40% 45% 11%

Poland 132 51%* 49% 27% 69% 5%


* decomposability supporters

Most respondents seem not to notice the way the numerical questions concerning

decomposability were created. Even after reading the verbal questions it requires a moment for

a close inspection of those two pairs to see what exactly happened and how the answers should

look like to be in accordance with the verbal question. Additionally in each case both

distributions are a mirror image of each other, the second example is the result of deducing the

first one from 14, but it is easier to notice in the 3-person society. Some respondents answered

“the same” in the first one, because they recognized this, but gave a different answer to the

second question, where this symmetry was harder to detect. Possibly using a different, less

distinctive example and making the verbal question easier would improve the answer’s

consistency.

Unbalanced enrichment

Temkin (1986) described an example of unbalanced immiserisation where an originally

perfectly equal and rich society gets poorer and poorer (one person at a time) until everyone is

8 Independence hypothesis was accepted by Pearson’s Chi-Square test at significance level of 0.05.

18

equally poor. Amiel and Cowell decided to analyse how respondents judge such a sequence of

events but in a verbal question they assumed a reversed order, namely unbalanced enrichment.

Q7. A = (5, 5, 5, 10) B = (5, 5, 10, 10)

Q8. A = (5, 5, 10, 10) B = (5, 10, 10, 10)

Q9. A = (5, 5, 5, 10) B = (5, 10, 10, 10)

Q15. Suppose there is a society consisting of n people. There is one rich

person and n-1 identical poor people. One by one, some of those who were

poor acquire the same income as the rich person, so that eventually there

are n-1 (identical) rich people and just one poor person. Please circle the

appropriate response

(a) Inequality increases continuously.

(b) Inequality decreases continuously.

(c) Inequality at first increases and then decreases.

(d) Inequality at first decreases and then increases.

(e) Inequality remains the same throughout.

(f) None of the above.

Questions concerning unbalanced enrichment were often described as “difficult” (“the last

three [numerical questions] were, well, a bit hard to answer” [resp. no. 7547]) and giving

conflicting answers to them was common. The first type of conflict that occurred often was in

the numerical questions where three income distributions were compared pairwise ((5, 5, 5, 10)

(5, 5, 10, 10) (5, 10, 10, 10)). There are 27 possible combinations of answers to those numerical

questions and only 13 of them provide a consistent non-conflicting ranking. Over 10% of each

sample gave such an illogical response (11% in A&C sample (1999 p.82) and 12.1% in the

Polish sample). Posing the same question in form of a ranking would reduce error and make it

easier to fill and understand for the respondents.

The second type of conflict, a much more popular one, was between answers in numerical and

verbal questions. The verbal question about unbalanced enrichment had 6 possible answers,

which makes answering hard by itself (“Here for example in 15th [question] there were a lot of

options and one had to read each one very carefully, to be sure” [resp. no. 2088]). Comparing

one’s verbal answer with the numerical one required a lot of effort and concentration; some

interviewees say that even after long consideration they still weren’t sure if they did it right in

the end. Among Polish students those who gave logical response in the numerical questions

56.1% gave a conflicting answer in the verbal part.

19

Table 9. Unbalanced enrichment – answers to numerical and verbal questions.

Sample

Inequality… A&C Polish

Numerical Verbal Numerical Verbal

Increases continuously 8% 7% 4% 5%

decreases continuously 8% 20% 12% 28%

first increases then decreases 26% 19% 19% 16%

first decreases then increases 42% 35% 48% 35%

remains the same 3% 11% 4% 11%

does none of the above 1% 4% 0% 5%

No transitive answer 11% - 12% -

Partial answer 1% 4% 2% 0%

N = 1108 132


Percentages of answers to numerical and verbal questions in the Polish sample are very similar

to those received by Amiel and Cowell (Table 9.). Their later research showed (1999) the

answers to these particular verbal questions are very volatile and depend heavily on the way

the question is posed. Also Devooght (2001, p. 32-35) has shown that in numerical questions

the order in which the distributions is presented (unbalanced enrichment or immiserisation)

influences the results significantly. It suggests that the received results are heavily biased. The

example and possible answers assume continuity in changes of inequality while the groups get

richer one by one. This assumption itself should be verified with the respondents. In the

questionnaire there was no logical way to answer numerical questions that would show

discontinuity, since there were only 3 distributions compared. In the verbal question one could

express such a view only by marking “None of the above” which is an answer generally chosen

reluctantly. One verbal question and three distributions dedicated to such a meaning loaded

example is not enough. It does not fit in the axiom framework, it comes from a completely

different approach to measurement of inequality and so should have a separate questionnaire

dedicated to it.

20

3.2. General problems

Questionnaire understandability

Some respondents, when interviewed after filling in the questionnaire, described it as

“difficult”, “deceitful”, or that even though it was interesting they “didn’t understand many

things”. On the other hand, answers that the questionnaire was “easy” and “understandable”

were also given by few people questioned, but most agreed that filling it in properly required

some reflection “(…) in every question one simply had to think for a moment.” [resp.no.7610].

One question tests one rule

Each question in Amiel and Cowell’s survey was designed to test agreement with exactly one

inequality axioms. This approach simplified questionnaire construction and was meant to also

ease the results interpretation. It also has its limitations that Amiel and Cowell were aware of.

The questionnaire was meant to establish whether certain axioms and as a result measures are

supported by the general public. It cannot tell us what other measures or axioms not asked

about would the respondents prefer. It also does not say which of the axioms are perceived as

most important or which can be abandoned if necessary. Maybe some respondents that were

classified as non-supporters of a certain measure would change their mind if presented with a

measure itself or a whole set of properties that this measure possesses.

Learning by doing

Amiel and Cowell divided the questionnaire into two separate parts: numerical and verbal.

After each verbal question the respondents were asked if answering it influenced their views

about certain numerical ones and were encouraged to note down their new answer. By such

questionnaire structure Amiel and Cowell “(…) attempted to allow for the ‘learning-by-doing’

that inevitably takes place in the course of completing such a questionnaire.” (1992, p. 6).

Judging from answers of interviewed Polish respondents, that goal was partially achieved, since

most interviewees appreciated such construction. Some respondents claimed that filling in the

first part of the questionnaire helped in answering the second part. Others said that it was the

other way around, the verbal questions helped them in answering the first part of the survey, a

situation also noted by Amiel and Cowell. There were also those who said that such

questionnaire construction did not benefit their answers in any way and that while filling in

each part of the survey they had to “think about it all over again” [resp. no. 3129]. The divide

into two parts has one main drawback, discussed later in detail, it makes it harder to compare

one’s numerical and verbal answers. Aside from that problem, starting a questionnaire with a

21

set of examples is a good way to help respondents recognize their own intuitions and think the

problem through on their own.

„I don’t know” answer and clear views about inequality

Amiel and Cowell aimed at reducing non-response to minimum (1992, p. 6), so the

questionnaire had no “I don’t know” (or “It’s hard to say”, “I have no opinion”) option.

The amount of non-responses was indeed very small, but it influenced the quality of received

data, by causing many “the same” and conflicting answers. Our interviews revealed complex

and multiple ways of dealing with a numerical question respondents did not know how to

answer. Among the possible strategies were:

Choose an answer, even if they are not sure of it. “I would mark something. I would

think and mark something with doubt, I wouldn’t be really sure about it.” [resp. no.

3129]

Circle both distributions answering that they are as equal – “Well, there was an option

to circle both” [resp. no. 1616]

Leave the question empty – this option was chosen least often. “I would try to decide,

but if I really didn’t know what to mark, I would leave it empty.” [resp. no. 6038]

Many interviewees reported a problem with answering one or more of the questions.

Sometimes they had no opinion at all or a weak one, sometimes they saw a few possible

perspectives from which one can judge certain distribution and they weren’t sure which one

they agree more with („Generally both answers seem reasonable to me, it depends how one

looks at it.” [resp. no. 6038]). Simultaneously there were questions where interviewees had a

very strong opinion. Looking at the questionnaire results we are unable to tell one from the

other. This problem could be partially solved by adding an “I don’t know”, “It depends on the

situation” or “I have no opinion” option. Another solution, possibly even better, is asking not

only about which of the two distributions is more unequal, but also how much those two

distributions differ in terms of inequality levels. This would make the questionnaire longer, but

it would allow us to identify which changes influenced inequality strongest. Such data would

also allow usage of a more complex analysis like multidimensional scaling or other advanced

exploratory methods.

22

Numerical questions – data presentation

The first, numerical part of the questionnaire was very minimalistic, it consisted only of sets of

numbers with information that they are “distribution of income”. As Amiel and Cowell

describe, “The distributions are presented as vectors, without explicit currency units, and no

hints were provided to the students as to what sort of living standards or welfare levels might

correspond to those numbers.” (1992, p.6). Quite a few respondents felt a need of a more

specific introduction, which was included by Amiel and Cowell in their later questionnaires.

Respondents dealt with the problem by making assumptions of their own, assumptions that the

researchers were not aware of. For example one of the interviewees, declared what level of

income he considered necessary for survival and, what is more important, changing that level

influenced his answers. Respondents had different assumptions about what the vectors

represented. Most people interviewed assumed that an expression like (1,4,7,10,13) symbolizes

incomes of 5 people: a whole population or a small group that consists of 5 people. Others

thought about society and perceived it as an information about incomes of certain social groups,

one person even assumed these groups had different sizes (few wealthy people, many middle

class citizens and so on). Such assumptions might influence one’s responses greatly 9 .

Respondents interpreted income distributions given to them in a variety of ways, since they had

little information about it. Some people treated numbers “just as numbers” [resp. no. 6038]

without any deeper meaning, which turned the questionnaire rather into a mathematical

challenge then real life inequality evaluation. Half of the interviewees visualised those numbers

as incomes in Poland. To do so, they transformed those numbers into more lifelike sums, but

each person used a different transformation. Depending on a person the same number, 1 for

example, meant 1 zloty, 100 zloty or 1000 zloty – respondents multiplied those numbers freely

to make them easier to imagine. However, these people did not always agree with scale

invariance axiom (so some of them did reply that multiplication changes inequality). One

respondent multiplied received numbers by 100 and added 2000 to turn it into “amounts of

money that we really operate with” [resp.no. 6565]10. Those who did not multiply given

numbers to make them more “real” often pointed out that differences between examples

9 In their further research using similar questions Amiel and Cowell did add an introduction informing respondents

that these are incomes of people living in “five regions that are identical in every respect other than the incomes

of their inhabitants. Everyone within a given region receives the same income, but personal incomes differ from

region to region” (Amiel et al. 2007).

10 This person agreed both with translation and scale invariance, so one can interpret it as being in accordance

with other answers, even if that means they were unconventional.

23

weren’t big, making it hard for them to make a distinction (especially in questions concerning

transfers).

Those answers suggest that an inequality questionnaire should use examples that are close to

real-life earnings. This should improve the quality of received data, by shielding us from

situation when respondents transform examples on their own, giving answers to different

questions than those really asked. Unknown assumptions about income necessary for survival

or decent living might be solved by explicitly asking respondents about those values.

Verbal questions

Verbal questions describe an inequality axiom or a hypothetical situation of a society that

undergoes a change. As Amiel and Cowell write: „Among the possible responses presented in

the questions in the second section there is usually one that could be characterized as the

‘traditional’ view found in the literature of measurement of income inequality. Some of other

responses printed in the questionnaire correspond to views that have also appeared in the

literature, some of them were suggested by respondents in a previous pilot questionnaire, and

some of them just seemed to us like a good idea at the time” (1992, p. 7).

Such approach led to different set and phrasing of answers in each question in the verbal part.

In the first four verbal questions (10, 11a, 11b and 12) there were three possible replies saying

that inequality remained the same, decreased or increased, but they came in different order and

wording (as presented in Table 11). The next two questions (13 and 14) had an answer

corresponding to the axiom support, a response expressing that it depends on the exact

distribution (also phrased differently) and finally “Neither of the above”. The last verbal

question had 6 possible answers describing inequality change trend. Summing up, each set of

answers had a different construction.

Table 11. Possible answers to verbal questions 10-12.

Question

10 11a 11b 12

(…) inequality

remains unchanged.

Inequality has fallen

(…). Is the same (…) is the same (…).

(…) inequality has

increased.

Inequality remains the

same. Increases (…) is less (…).

(…) inequality has

fallen.

Inequality has

increased. Decreases. (…) is greater (…).

24

Explanations were attached to some of the answers, but they were distributed unevenly: in

question 10 all of the options given had arguments backing up choice of this answer; in question

11a, only one answer had such an explanation, in 11b none and in 12 two out of three answers.

An argument backing up one of the responses might both increase or decrease its popularity

(or even influence each respondent differently). For example one person admitted to choosing

a different answer than the one he originally intended in the verbal part, because explanation

attached to it was an “improper way of thinking.” [resp. no. 7610].

Verbal questions by nature are longer and require more effort to read, adding explanations and

giving different set of answers to each of them only increases this problem. Many interviewees

complained about it e.g. “Here you had to read those sentences, each one twice, than read

every answer, analyse, (…).For example here in 15th there was a lot of options and you had to

read all of them to make sure.” [resp. no. 2088].

The verbal part of the questionnaire was often described, by the interviewees, as “suggestive”

or, as one person explained it “guided one’s answers in some way, (...).” [resp. no. 6038].

Using a different question structure each time and adding explanations to only some of the

answers causes confusion: it is difficult to answer such questions, the results are heavily

dependent on these factors, and it is hard to compare answers to different verbal questions.

Amiel and Cowell did see this problem, but they neither solved the issue nor investigated what

the shape of influence that question structure and explanations had on respondent’s answers

was (1992 p.7). Discovering how certain explanations influence answers would require a whole

separate research. That is why removing all the explanations and using, as much as possible, a

standardized set of answers, given always in the same order, seems a more reasonable way of

solving this problem.

Comparing and changing numerical and verbal answers

Each set of numerical questions had a verbal counterpart after which the respondents were

asked to rethink their answer to those numerical questions. Asking respondents to change their

original answer is not a common practice, so it came as a surprise or a “very interesting” option

to some of the interviewees. Almost half of the sample (47.7%) used the option of changing

ones’ answers at least once, so it seems that this questionnaire structure did motivate some to

rethink their opinions. Unfortunately not everyone reacted to this possibility in the same way.

Some said that it “made the questions longer and, at least I didn’t pay much attention to it.”

[resp. no. 2088], others just found it slightly confusing or tiresome. Looking at one’s responses,

25

we do not know whether lack of answer change is due to steady views or lack of comparison

between two questionnaire segments. Even if someone changed one of their answers, we still

don’t know if they rethought all responses or just the one that they changed. One of the

interviewees even admitted to “cheating”, as he called it, by changing his answer but marking

the change in the original question, so that no one would notice that he gave a different answer

in the first place.

In general, asking respondents to rethink their answers seems to have improved data quality on

one hand, but lengthened and complicated the questionnaire on the other. This problem can be

solved by restructuring the questionnaire so that each verbal question directly follows its

numerical counterpart, but it would ruin the idea of “learning by doing”. Using an interactive

questionnaire would allow us to keep the divide into numerical and verbal part and remind the

respondent his previous answers if they are relevant to the question they are currently thinking

about. As a result it would make reconsidering, comparing and possibly changing one’s answer

easier and less time consuming.

Numerical and verbal questions - different or the same topic?

Both parts of the questionnaire address the same problems and axioms, but they differ

significantly in possible answer interpretation. Numerical questions compare two static

distributions, while verbal ones judge transformation of one distribution into another, as Amiel

and Cowell put it “(…) an extra factor is being introduced, namely the order in which the cases

are presented” (1992, p.13). It is a significant difference because people evaluate losing and

gaining money differently (Kahneman and Tversky, 1979). Our interviewees had similar

intuitions “(…) when we are talking about getting richer, or about certain process, then this

inequality is getting different” [resp. no. 5414], “(…) because gaining, and taking away, I don’t

know, it may have conditioned my change of answers in some situations.” [resp. no. 6565].

This difference was most visible in inquiries concerning unbalanced enrichment, since the

sequence of distributions was even longer, which could be also the cause of huge differences

between numerical and verbal answers in these questions.

Verbal questions also point respondents’ attention to certain aspects of the situation since they

must describe the situation e.g. phrases “enough money for basic needs”, “those who had more”

and so on might influence respondents’ views. Some interviewees said that the first part of the

questionnaire was more “objective” while the second one introduced “feelings” and

“emotions”. It is normal that our objective and emotional evaluations are different from each

other and thus some of the respondents could give different answers.

26

In general, respondent’s answers to the first, numerical part allows us only to say that in this

example this person acted consistently with certain axiom. Answers to the verbal questions

allow us to conclude whether they support this axiom or not. Amiel and Cowell resigned from

comparing verbal and numerical answers, claiming that they are not about the same thing

(1992, p. 13). While being aware of differences between the types of questions that are asked

is important, it should not stop us from using our data fully. Numerical and verbal questions

are different, they show the problem of inequality from two alternative perspectives and this is

exactly the reason why their answers should be compared.

Unconventional and/or illogical answers

The abundance of unconventional or illogical answers is a huge problem of Amiel and Cowell’s

questionnaire. One of the reasons behind it is that answers to many questions can collide and it

is difficult to give answers that are completely consistent. First of all, there are six opportunities

of discrepancy between numerical and verbal answers, one for each axiom and one for the

unbalanced enrichment example. Depending on the axiom, the level of such inconsistencies

ranged from 13.1% to 56.1% (Table 10). It would be a very valuable result if those conflict

rates could be treated as an indicator of ambiguity of views concerning certain axioms.

However because of the significant differences in question construction we cannot do that,

since we do not know whether these inconsistencies are caused by the axioms themselves or

by the way the question was posed. For example, high inconsistency rates in questions

concerning the example of unbalanced enrichment are partially caused by the fact that there

were six possible answers in the verbal question and even more combinations in the numerical

ones. Relatively small amount of conflicts in replies concerning transfer principle and

decomposition result from the fact that answers to numerical and verbal questions do not match

each other closely11.

Other three opportunities where a respondent could give an illogical answers were: translation

invariance, which received a conflicting answer to verbal questions (there was one question

about adding and another about deducing the same amount of money) from almost one fifth

(19.1%) of the sample; rivalry between translation and scale invariance, they were both

(simultaneously) supported, by 29% of the sample in the numerical part and 41%12 in the verbal

one; examples depicting unbalanced enrichment of a society (three distributions were

11 Only one answer to those verbal questions determine the response to their numerical counterpart, limiting the

scope for conflicting responses. 12 41% when comparing answers to questions q10 and q11a and 35.1% when comparing questions q10 and q11b

27

compared pairwise in numerical section), caused 12.1% of respondents to give an intransitive

answer.

Table 10. Percentages of respondents whose replies to verbal and numerical questions about

the same axiom were directly conflicting in Polish sample.

Percent of

inconsistent

respondents

Scale-invariance (q1, q10) 41.2%

Translation-invariance (q2, q11a) 33.3%

Principle of Population (q3, q12) 26%

Transfer Principle (q4, q13) 31.8%

Decomposition (q5, q6, q14) 13.1%

Unbalanced enrichment (q7,q8,q9,q15) 56,1%


Note: For unbalanced enrichment the percentage is among respondents who gave a transitive

answer in numerical questions concerning this topic.

Overall only 11 people out of 132 surveyed (8.3%) in the Polish sample did not have any

conflicting or unconventional set of answers in their questionnaire. This is an accumulated

result of many problems that were discussed throughout the article: some small, concerning

specific questions, and some more general. Solving even few of them should reduce the amount

of conflicts in the received data and improve accuracy of the measurement.

4. Conclusion

Research results thus far show general lack of support for most common inequality measures.

Among other issues, this outcome might be questioned on the grounds of imperfect research

methodology. Many conflicting and ambiguous answers, respondents who sometimes change

their responses when confronted with the questionnaire again, prove there is a huge scope for

improvement.

This research showed that there are multiple reasons behind unconventional and conflicting

answers. Possibly these problem can be solved by improving the methodology used.

First of all, the questionnaire itself was long and difficult, partially due to the topic it concerned.

For that reason it is important to limit the number of questions and make them as

understandable and easy to answer as possible. Asking verbally twice about the same axiom is

28

often unnecessary. The aim of the questionnaire should be well defined and it should contain

only such questions that fulfil it, e.g. if we intend to verify axiom support then maybe we can

omit questions about other topics, like in the example of a society experiencing unbalanced

enrichment. The money distributions should be presented in an intuitive way, e.g. as bar charts.

They should use explicit currency units and contain incomes that are achievable in the country

where the research takes place. Using a more natural language instead of a formal one and less

abstract examples can also make the questionnaire easier to understand. Verbal questions could

benefit from making the possible answers shorter, standardized and without explanations added

to them. It would make answering and comparing results easier.

The amount of conflicting answers can be significantly cut down by joining possibly

conflicting questions into one. In numerical part it can be done by changing a set of pairwise

comparisons into a small ranking task, which would additionally shorten the questionnaire.

Adding an option like “I don’t know”, “I don’t have an opinion”, or other way to express doubt,

could also reduce inconsistency rate, since some conflicting responses are a result of choosing

an answer that the respondent is not sure about.

While asking both numerical and verbal questions makes the questionnaire long and increases

possibility of inconsistent answers, these questions reflect views on inequality from different

perspectives and this makes it worthwhile to include and compare them. Dividing the

questionnaire into two parts: numerical and verbal does seem to have a “learning by doing”

effect, at least for some of the respondents, as was intended by Amiel and Cowell.

Unfortunately it makes comparing one’s numerical and verbal responses harder and as a result

increases inconsistency rates. Changing the questions’ order, so that the verbal questions follow

directly their numerical counterparts, or using an interactive questionnaire that brings up

previous answers that are relevant to a verbal question, could help.

Reassessing, results obtained Amiel and Cowell’s questionnaire both in the original and in the

Polish study are burdened with high data error, but they bring to light important problems and

questions. Improvements of research methodology proposed in this paper, based on qualitative

interviews and methodological analysis, should enable us to better measure the way a “common

man” defines income inequality.

29

Bibliography

Amiel, Y., Cowell, F. A.: Measurement of income inequality: Experimental test by

questionnaire. Journal of Public Economics 47, 3–26 (1992)

Amiel, Y., Cowell, F. A.: Thinking about Inequality. Cambridge University Press, Cambridge

(1999)

Amiel, Y., Cowell, F. A.: Risk and Inequality Perceptions. Distributional Analysis Research

Programme Papers 55 (2001).

Amiel, Y., Cowell, F. A.: Attitudes towards Risk and Inequality: A Questionnaire-

Experimental Approach. In Andersson, F., Holm, H. J. (eds.) Experimental Economics:

Financial Markets, Auctions, and Decision Making, pp. 85–115. Kluwer (2002).

Amiel, Y., Cowell, F. A., Slottje, D. J.: Why Do People Violate the Transfer Principle?

Evidence from Educational Sample Surveys. Research on Economic Inequality 11, 1–16

(2004)

Amiel, Y., Cowell, F., Gaertner, W.: Distributional orderings: an approach with seven flavours.

Theory Dec. 73, 381-399 (2012)

Amiel, Y., Cowell, F., Ramos, X.: Poles Apart? An Analysis af The Meaning of Polarization.

Review of income and Wealth 56, 23-46 (2010)

Ballano, C., Ruiz-Castillo, U. J.: Searching by questionnaire for the meaning of income

inequality. Revista española de economía 10(2), 233–259 (1993)

Cuena, E. C., Neugebauer, T., Seidl, C.: Leaky bucket Paradoxes in income inequality

perceptions: an experimental investigation. Economics Working Papers 04 (2004).

Cowell, F. A.: A Fair Suck of the Sauce Bottle’or What Do You Mean by Inequality? Economic

record 61(2), 567–579 (1985)

Decoster, A., Schokkaert, E.: The choice of inequality measure in empirical research on

distributive judgements. Journal of Economics 9(1), 197–222 (2002)

Davidovitz, L., Kroll, Y.: Choices in Egalitarian Distribution: Inequality Aversion versus Risk

Aversion. Distributional Analysis Research Programme Papers 43 (1999)

Devooght, K.: Measuring inequality by counting complaints: theory and empirics. Economics

and Philosophy 19(2), 241–263 (2003)

Harrison, E., Seidl, C.: Perceptional inequality and preferential judgements: An empirical

examination of distributional axioms. Public Choice 79(1-2), 61–81 (1994)

Kahneman, D., Tversky, A.: Prospect theory: An analysis of decision under risk. Econometrica

47(2), 263-291 (1979)

30

Stiglitz, J. E.: The Price of Inequality: How Today's Divided Society Endangers Our Future.

Norton Company (2012)

Temkin, L.S.: Inequality. Oxford University Press, New York (1993)

Temkin, L.S.: Inequality. Philosophy and Public Affairs 15(2), 99–121 (1986)

Traub, S., Seidl, C., Schmidt, U.: Lorenz, Pareto, Pigou: Who scores best? Experimental

evidence on dominance relations of income distributions. Economics working paper 04 (2003)

Traub, S., Seidl, C., Schmidt, U. An experimental study on individual choice, social welfare,

and social preferences. European Economic Review 53(4), 385–400 (2009)

Wilkinson, R. G., Pickett, K.: The spirit level: why greater equality makes societies stronger.

Bloomsbury Press, New York (2010)

Constructing experimental tests by questionnaire Barbara ...However, results of their study contain...

Documents

Transcript of Constructing experimental tests by questionnaire Barbara ...However, results of their study contain...