Categorization vs. logic in category conjunction: towards a model of overextension

Dr. Fintan Costello,Dept of Computer Science,University College Dublin.

Fintan.Costello@ucd.ie

Classification in conjunctive categories

The ability to form new complex categories by the conjunction of existing categories is a basic logical operation and an important part of classification and reasoning.

Any worthwhile AI model of categorization should be able to reason about conjunctive categories in a way that mirrors human reasoning about such conjunctions.

In this talk I describe a particular problem which AI models must face in attempting to mirror human conjunctive classification: that of overextension.

I will then describe a computational model that attempts to address this problem, and will demonstrate that model in an experimental test.

Overextension in conjunctive categories: a challenge to logic

In the logical, set-theoretic, view of category conjunction, an item is a member of a conjunction A&B only if it is a member of the single category A and a member of the single category B.

However, people regularly and reliably break this logical rule when classifying items in some conjunctive categories.

They regularly classify items as non-members of a constituent category A (or B) but as members of conjunction A&B. This deviation from set logic is called overextension.

Are ‘blackboards’ furniture? Are ‘blackboards’ school furniture?no yes

Is ‘pool’ a sport? Is ‘pool’ a game that is also a sport?no yes

Overextension and similar effects also occur, for example, with drawings of faces (‘is this person intelligent? No. Is this person happy and intelligent? Yes.’), in economic decisions, and in probability judgements.

Explaining overextensionMost accounts of overextension and similar ‘conjunction fallacy’ effects (e.g. Hampton, 1996, Tversky & Kahnemann, 1983) are based on an idea of representativeness or similarity to a conjunctive prototype. According to this view, for example:

•‘blackboards’ are more similar to the prototype of ‘school furniture’ than to that of ‘furniture’, and so are members of the first but not the second.

This gives a mechanism for how overextension can occur, but does not explain why this apparent error should persist and be widespread in human reasoning.

I will argue that overextension is a logical consequence of a more basic requirement in human reasoning: the need to have an open, as opposed to closed, system of category conjunction.

In an open category system, items can be classified in new, previously-unseen category conjunctions: conjunctions for whose existence there has been no previous support.

Positive and negative evidence for classificationTo classify an item as a member of a category A, we use two sources of evidence : positive evidence (if the item has features that occur in known members of A) & negative evidence (the item has features that occur in known members of other categories: in A’s contrast set).

Classification in a conjunction A&B:

Positive: A Negative:Positive: B Negative:

Membership(A | Membership in A&B Membership(B |

Suppose A&B is a new conjunction – there are no known members of AB. Then A , and B . This means that positive evidence for membership in A is negative evidence for B, and vice versa.

This means that no items will get good classification scores in a new conjunction A&B. This is a closed system of categories.

Get positive and negative evidence

Use both to compute constituent membership

Combine these to get conjunctive

membership

An open system for conjunctive classification

Membership in A&B

In a system like this, positive evidence for membership in A will not count as negative evidence for membership in B, or vice versa. This is an open system: items will be classified in A&B even though A&B is a new conjunction (there are no known members of AB).

Get positive and negative evidence

Use both to compute constituent membership

Combine these to get conjunctive

membershipPositive: A Negative: AB

To produce an open system for conjunctive classification, we must ensure that positive evidence for membership in A or B does not get counted as negative evidence for membership in the conjunction A&B.

Let the contrast set for A&B to be AB: items outside both A and B.

Positive: B Negative: AB

Membership(A | AB)

Membership(B | AB)

An open system produces overextensionIn an open system of classification, when classifying an item in a single category A, positive evidence comes from known members of A and negative evidence comes from known non-members of A (the contrast set for A is). This is logically correct for single categories.

When classifying an item in a conjunction A&B, positive evidence comes from A and B and negative evidence comes from AB (A&B’s contrast set is AB). This is logically correct for ‘open’ conjunctions.

This means that an item X with features that are relatively frequent in single category A but that are very frequent in single category B (and where AB= ) will have negative evidence for memberhip in A, and so will be classified as a non-member of the single category A.

This is an overextension response for the category and conjunction.

However, if X’s features are rare in the conjunctive contrast set AB, X will have very little negative evidence for membership in A&B, and so will be classified as a member of the conjunction A&B.

An example: diagnostic evidence model of classification

I will illustrate these ideas using a simple model of single and conjunctive classification: the diagnostic evidence model.

This model assumes that people represent categories by storing sets of category members in memory.

From these sets, diagnostic features are computed: these features act as evidence to identify category members.

An item’s classification in a single or conjunctive category is a function of the diagnosticity of its features for that category or for the constituent categories of that combination.

A continuous-valued logic is used to combine evidence from multiple diagnostic features, and to combine evidence for membership in the constituent categories in a combination.

Feature diagnosticity: category and contrast set

A new item is placed in a category C if it has diagnostic features for C: features common in stored members of C but rare in its contrast set Kc (the set of stored items outside C).

Diagnosticity of feature x for category C (relative to contrast set KC)

)Kc___()_()___(

)Kc||(inxnumCsize

CinxnumCxD

An item with diagnostic features for a category gives evidence for membership in that category.

A crucial factor in this computation is the contrast set. As discussed earlier, for a single category A the contrast set is the set of items outside A. For a conjunction A&B, however, the contrast set is the set of items that are outside both A and B.

Diagnosticity in combinations: change in contrast set

Single category contrast set

All stored items that are not members of that single category

Conjunctive category contrast set

 All stored items not members of any constituent of combination

If a feature occurs often in a single category’s contrast set, but less often in the contrast set for a combination containing that category, the feature will be less diagnostic for the single category, but more diagnostic for the combination.

This change in diagnosticity is what causes overextension.

If there is no difference in feature occurrence in the contrast set for a single category and a combination, overextension will not occur.

Multiple features: Overall evidence for membership

Continuous-valued logic. Variables between 0(false) and 1(true)

not A = 1-A A and B = A*B A or B=1-(1-A)*(1-B)

Membership in single categories

An item i with features X1 X2 X3 gives evidence for membership in category C if feature X1 or X2 or X3 is diagnostic for C:

E(i|C|KC)= 1-(1-D(x1|C|KC))* (1-D(x2|C|KC))* (1-D(x3|C|KC))

Membership in combined categories

Item i gives evidence for membership in conjunction C1&C2 if i has some features diagnostic for C1 and others diagnostic for C2 (relative to K1&2)

E(i|C1&C2|K1&2) = E(i|C1|K1&2) * E(i|C2|K1&2)

Testing the model: an experiment with single categories and conjunctions

Method: Investigates classification and overextension (logical errors) using a controlled set of patient-descriptions (items), symptoms (features on 3 dimensions) and categories (diseases A, B, and C).

Training phase: 18 participants get a set of patient descriptions (training items) with certain diseases and symptoms, and learn to identify diseases (to criterion).

Test phase: Participants get 5 new patient descriptions (test items) with new symptom combinations. For each test item participants separately rate patient as having disease A, B, C, A&B, A&C, B&C. Each test item therefore occurs 6 times in the test phase (with 6 different rating questions).

Results. Classification scores and frequency of overextension errors are compared with those predicted by the diagnostic evidence model.

Training itemsThese disease categories have a family-resemblance structure: there are no simple rules linking an item’s symptoms and category membership.

Participants learned categories by studying items like these. Different participants got different symptom-words in the training materials, but all had the same symptom distribution.

Participants then classifed new “test” items in categories and category conjunctions.

1 Puffy Flaking Strained Disease A

2 Sunken Flaking Knotty Disease A

3 Sunken Pallid Knotty Disease A

4 Puffy Sweaty Knotty Disease A

5 Puffy Flaking Limp Diseases A&B

6 Puffy Blotchy Twitchy Diseases A&B

7 Red Flaking Knotty Disease B

8 Cloudy Blotchy Twitchy Disease B

9 Red Blotchy Twitchy Disease B

10 Red Jaundiced Knotty Disease B

11 Red Pallid Twitchy Disease B

12 Red Jaundiced Weak Disease C

13 Sunken Jaundiced Twitchy Disease C

14 Red Flaking Weak Disease C

15 Sunken Flaking Twitchy Disease C

16 Sunken Jaundiced Weak Disease C

17 Cloudy Jaundiced Twitchy Disease C

Item Symptoms Category

EYES SKIN MUSCLES

Symptoms rated as member of category or conjunction

EYES SKIN MUSCLES A B C A&B A&C B&C

1 Puffy Jaundiced Weak ? ? ? ? ? ?

2 Sunken Flaking Weak ? ? ? ? ? ?

3 Red Jaundiced Twitchy ? ? ? ? ? ?

4 Red Blotchy Weak ? ? ? ? ? ?

5 Puffy Blotchy Knotty ? ? ? ? ? ?

Participants learned training items and then classified the test items as members or non-members of the categories and conjunctions.

The model was given the training items and used the feature distribution there to compute membership for each test item in each category and conjunction.

Test items

Results

Classification

The model’s classification scores were a good fit to participants’ average classification scores for items as members of single categories (r=.91, <.001, %var=.82) and as members of category conjunctions (r=.85, <.001, %var=.73) in the experiment. No free parameters were used in fitting the model to the data.

Overextension

Nearly half of participants (46%) classified test item 2 as a non-member of category A but a member of disease conjunction A&C. The next most frequent case of overextension occurred for test item 4 30% of participants classified test item 4 as a non-member of single disease category B but a member of disease conjunction B&C.

All remaining cases of overextension were produced by less than 15% of participants.

Model’s predicted overextension

To produce overextension predictions from the model a “membership threshold” was applied to the model’s classification scores, so that if an item was given a score above the threshold in a category or conjunction it was taken as a member; if the score was below that threshold, the item was a non-member of the category or conjunction.

Excluding extreme (very high or very low) threshold values, only one membership threshold value produced an overextension response in the model’s computed classification scores. With this threshold, test item 2 was classified as a non-member of category A but a member of conjunction A&C. This was the item, category and conjunction for which overextension was most frequent in the experiment.

The chance of the model selecting one case of overextension at random, and that being the most frequent case of overextension in the experiment, is 1 in 45 (p < 0.05).

Conclusions

Overextension is an apparent deviation from logic which AI models that aim to reason about and use conjunctive categories must address.

The basic ideas in this talk can be summarised as follows:

Looking deeper, however, overextension is a logical consequence of

(i) the need to use positive and negative evidence in classification,

(ii) the need to have an ‘open’ system of conjunctive categories: one which can classify items in conjunctions for whose existence there has been no previous support.

Take-home message:If you want an open system of conjunctive classification, and you want to use positive and negative evidence in classification, you will have to think carefully about sources of negative evidence. And you will have to resign yourself to overextension and other apparent logical errors.